Design Pattern for Data Entry Applications

Design Pattern for Data Entry Applications M 917 536 3378 maksim_kozyarchuk@yahoo.com

Overview A classical way to develop data entry applications is to follow either an MVC or an MVP pattern.  Using one of these patterns is a significant step forward from a pattern that most UI builders guide you towards, as it allows separation of business logic from presentation logic reducing complexity and enabling unit testing of the business logic separate from the UI.  While these patterns are powerful, more structure is needed when building rich data entry applications.  A typical data entry application has a few responsibilities and complexities. Multiple entry portals for the same data( i.e. quick entry, detailed entry and API upload) Similar fields appearing on multiple entry forms and having either identical or slightly different behavior Field and model level validation rules that need to have a consistent look and feel Context-based field calculation and defaulting rules that can vary in levels of complexity To support these complexities, while maintaining consistent behavior between entry portals and between fields across multiple screens, we should create the following four layers of abstraction Presentation field(s), responsible for defining datatypes, field level validations, calculation rules associated with the fields and mapping to domain objects Product configuration(s), responsible for defining field list for a given product, relationship and dependencies between fields, product level validations as well as the relationship between product and domain objects Presentation configuration(s), responsible for defining a subset of product fields made available for a particular data entry method. This will include display/presentation names of the fields. Domain object(s), responsible for defining structure of the data as it’s stored  in the database, one of it’s responsibilities would be to capture the core structure of the data normalizing diverse product-specific fields down to a relatively small set of core fields Introduction to ReactiveFramework   In this section, I will introduce ReactiveFramework which will provide a strong definition around Fields, Product Model and interface for external systems( UI or API).   To keep this article grounded in practice, I will use an example of FXTransaction entry where a user needs to specify two out of three fields, (primary amount, fx rate, and secondary amount) having the third field calculates automatically First let’s define a few key components.  Field class provides metadata definition behind the field and should uniquely represent a given field across multiple screens. class Field:   def __init__(self, name, datatype, validation_method = None, calculation_method = None):       self.name = name       self.datatype = datatype       self.validation_method = validation_method       self.calculation_method = calculation_method        The Field class has four components Field name that uniquely identifies the field.   Datatype, while python is flexible about datatypes of fields, having a strong notion of datatype for a given data entry field is key to improving overall quality and avoiding boilerplate code for ensuring type compatibility when performing calculations and validations of data.  Developing in Python, we should ensure correct datatypes as the data is entering the system.    Validation and calculation methods will be bound to the relevant product model and used by the reactive framework to validate and calculate values of relevant fields.    The FieldFactory class is the repository of all of the fields in the system, in practice it should load field definition from a configuration source, but for the purpose of this example, I will include a very basic implementation to demonstrate field setup for FXTransaction example described above. class FieldFactory:   FIELDS = [ Field(name = 'action', datatype = str, domain_name = 'action', validation_method = 'must_be_provided'),              Field(name = 'primary_amount', datatype = Decimal, domain_name = 'quantity', calculation_method = 'calc_primary_amount', validation_method = 'must_be_provided'),              Field(name = 'secondary_amount', datatype = Decimal, calculation_method = 'calc_secondary_amount', validation_method = 'must_be_provided'),              Field(name = 'deal_fx_rate', datatype = Decimal, calculation_method = 'calc_deal_fx_rate', validation_method = 'must_be_provided'),              Field(name = 'commission', datatype = Decimal, calculation_method = 'calc_commission', validation_method = 'must_be_provided'),   ]   @classmethod   def getField(cls, field_name):       for field in cls.FIELDS:           if field.name == field_name:               return field        The ReactiveFramework class defines Public API and interactions supported by the Framework.  For now, public interface will have a constructor which accepts Product Model to define fields used by the product and contain implementation of calculation and validation methods.  It will have a set_field method which needs to be called to set value for a particular field, validate method which will return a list of validation errors and get_value method which will allow user to query value of fields defined in the model.  Below is an example usage of the ReactiveFramework for construction and validation of FXTransaction. >>>fxTrade = ReactiveFramework(FXTransaction()) >>>fxTrade.set_value('action', 'Buy') [] >>> fxTrade.set_value('primary_amount', 100) ["commission"] >>>fxTrade.set_value('deal_fx_rate', 1.5) ["secondary_amount"] >>> fxTrade.get_value("secondary_amount" ) 150.0 >>> fxTrade.get_value("commission") 1.00 >>> fxTrade.validate() {} We construct an instance of the ReactiveFramework by passing in an instance of the Product Model, in this case FXTransaction.  We then call set_value on the action field and back and an empty list in return.  Empty list indicates that no other field depends on the action, implying that no calculations took place.  Next we set_value on primary_amount and get back a list containing commission, this indicates that the commission field has been calculated based on the new value of primary_amount. Next we set deal_fx_rate and get back indication that secondary_amount was calculated.  Then we check values of commission and secondary_amount  to verify that they have been set.  Finally, we call the validate method and get back and empty dict indicating that there are no validation errors.   The FXTransaction class defining the the Product Model is below. class FXTransaction:   FIELD_DEPENDS = {       'action': [],       'primary_amount': ['secondary_amount', 'deal_fx_rate'],       'secondary_amount': ['primary_amount', 'deal_fx_rate'],       'deal_fx_rate' : ['primary_amount', 'secondary_amount'],       'commission' : ['primary_amount']   }      def bind_fields(self):       for field_name in self.FIELD_DEPENDS:           setattr(self, field_name, BoundField(FieldFactory.getField(field_name), self) )              def calc_commission(self):       return self.primary_amount.value * Decimal("0.01")      def calc_secondary_amount(self):       return self.primary_amount.value * self.deal_fx_rate.value   def calc_primary_amount(self):       if self.deal_fx_rate.value:           return self.secondary_amount.value / self.deal_fx_rate.value   def calc_deal_fx_rate(self):       if self.primary_amount.value:           return self.secondary_amount.value / self.primary_amount.value          def must_be_provided(self, field):       return "" if field.has_value else "%s is missing" % field.definition.name      FXTransaction defines fields that are part of the Product Model along with calculation and validation methods required by these fields. FXTransaction is loosely coupled to Fields and FieldFactory, bind_fields method validates correct field setup and establishes binding between field definition and model state. When working within the ReactiveFramework, Fields and Product Models are the primary concern of the developer. When adding new screens or new functionality to the screens, the developer would create new or reuse existing fields and create or modify Product Models which define field interactions and dependencies.   A note on implementation, while it’s possible in python to derive dependencies implicitly from the calculation methods, in the opinion of the author, stating them explicitly has the benefit of simplicity, readability and flexibility which outweigh convenience of implicit dependency calculation.

Implementation of the ReactiveFramework ReactiveFramework itself consists of two classes.  ReactiveFramework class which defines the public API that we saw in the previous section and BoundField class which maintains field state and binding to the Model.   BoundField state consists of: The value of the field A flag indicating if the field has a value which is used as part of the calculation framework to determine if a dependent field is ready to calculate.  Only fields that have values for all of their dependencies can be calculated A flag indicating if the field has been set explicitly through the public API.  The framework will not recalculate/replace value of the field that has been set explicitly through the API Below are the classes that make up the ReactiveFramework class BoundField:   def __init__(self, field_definition, model):       self.definition = field_definition       self.value = None       self.has_value = False       self.has_user_entered_value = False       self.calculation_method = self._bind_method('calculation_method', model)       self.validation_method = self._bind_method('validation_method', model)   def _bind_method(self, method, model):       if getattr(self.definition, method):           return  getattr(model, getattr(self.definition, method))   def set_value(self, value, user_entered =True):       self.value = self.definition.datatype(value)       self.has_value = True       self.has_user_entered_value = user_entered   def recalc(self):       if not self.has_user_entered_value:           if self.calculation_method:               self.set_value( self.calculation_method(), user_entered= False )           return True       return False      def validate(self):       if self.validation_method:           return self.validation_method(self)    class ReactiveFramework:   __slots__ = ('model', 'depends_notifty')   def __init__(self, model):       self.model = model       self.depends_notifty  = {}       self._init_depends_notifty()       self.model.bind_fields()      def _init_depends_notifty(self):       for field_name, deps in self.model.FIELD_DEPENDS.items():           for dep_name in deps:               self.depends_notifty.setdefault(dep_name, [])               self.depends_notifty[dep_name].append(field_name)   def _are_dependents_set(self,field_name):       for dep_field in self.model.FIELD_DEPENDS[field_name]:           if not getattr(self.model,dep_field).has_value:               return False       return True      def _recalc_field(self, field_name, recalculated):       if self._are_dependents_set(field_name):           if getattr(self.model, field_name).recalc():               recalculated.append(field_name)               self._recalc_dependents(field_name, recalculated)   def _recalc_dependents(self, field_name, recalculated = None):       if recalculated is None:                   recalculated = []       for field in self.depends_notifty.get(field_name, []):           if field not in recalculated:               self._recalc_field(field, recalculated)       return recalculated          def set_value(self, field_name, value):       getattr(self.model,field_name).set_value( value )       return self._recalc_dependents(field_name)   def get_value(self, field_name):       return getattr(self.model,field_name).value   def validate(self):       result = {}       for field_name in self.model.FIELD_DEPENDS:           errors = getattr(self.model, field_name).validate()           if errors:               result[field_name] = errors       return result Other components of the data entry platform Strong definition of fields and interactions between them that ReactiveFramwork provides, is core but is a small part of an effective data entry platform.  I will discuss other core components, DomainModel binding  and presentation layers in the future posts.