Thoughts on modeling trades

Building a Trade Model by  Maksim Kozyarchuk maksim_kozyarchuk@yahoo.com

	Overview Trade Model lies at the core of any Front, Middle or Back Office platform and has direct implications on it’s capabilities.  A well designed Trade Model will support multiple business functions without duplicating trade data, increasing data quality and keeping operational costs low.  Below is the the list of functions that Trade Model needs to support: Real-time Front office view positions, risk and p&l Support for Middle and Back office workflows such as trade affirmations, reconciliations, margining, balance sheets and more Ability to attribute risk and p&l to trading strategy, sales channel and other criteria Tax lot accounting and books and records quality p&l Support trade allocations Support trade fills Support OTC and Listed products across multiple asset classes Historical positions, risk and p&l Audit trail of all trade events Support higher level transaction groupings such as deals or contracts
	Modeling a Trade A trade is the action of buying or selling a security.  While there is great variety in financial products, large number of trade execution venues, numerous methods for managing p&l and risk and diverse trade life cycles, the basic concept of the trade is quite simple.  The following attributes are sufficient to capture financial and descriptive information about the trade and make up the Trade Model:: Action ( Buy, Sell ) Quantity( may mean different things depending on product ) Price and Currency Security Reference Trade Date/Time Settlement Date Trade Cashflows, representing trade settlement amount, commission and other fees and taxes Descriptive Trade Attributes, indicating legal entities, strategy, sales channel and other attributes Trade Identifiers: Providing ability to uniquely reference the trade, link related trades together and provide reference to for external counterparties Additional Trade Details: These may include execution details, such as order type and venue, trade processing and clearing instructions, calculation agent, trade documentation and allocations In addition to the descriptive fields, to support effective reporting, trade needs to contain attributes describing trade lifecycle with respect to multiple consumers of trade data.
	Descriptive Trade Attributes   Key responsibility of the Trade Model is ability to provide flexible reporting of trade, position and P&L data.   Descriptive Trade Attributes are at the core of this.  To support books and records quality P&L, it’s important that all trades are tagged with correct internal and external legal entities. A combination of these entities will represent the unique bucket to be used for Tax Lot calculation and determination of realized and unrealized p&l.  For a Hedge Fund, typical legal entities on the trade will be the Master Fund and Prime Broker or a Clearing Agent.   In addition to legal entity tagging, trade model needs to allow position aggregation by strategy, analyst, trader, internal portfolio and other arbitrary attributes.   While it’s not necessary to support Tax Lot based P&L at these aggregation levels, we need to support positions, risk and average cost/trading p&l.         Storing descriptive trade attributes directly on the trade, does not support adding custom fields well and carries a significant overhead in data and index sizes for large trade tables.   Instead, we should create a portfolio for each combination of these attributes and keep only the portfolio id on the trade.  Portfolio would in turn have attributes for fund, clearer, analyst and other grouping tags.   Since the portfolio table would be orders of magnitude smaller than the trade table, it should reduce database IO requirements for sql queries, significantly simplify process of adding custom attributes and provide simple, yet sufficiently granular notion of position.
	Trade IDs   Another important responsibility of the Trade Model is ability to represent and maintain complex trade/deal structures in a way it was executed and to enable effective post execution communication with counterparties about the trade.  To support this, trade model needs to support rich yet fairly rigid trade id schema.  First, the trade model itself, only needs two ids, a unique record identifier and trade id which will remain constant across versions of the trade.   All other ids should be maintained in a separate table referencing only the trade id, a vertical table containing ( trade id, alternate id and alternate id type) provides sufficient representation for this.     In practice a given trade may have anywhere from 0 and a dozen of alternate trade ids.  These will include deal/contract ids, counterparty ids, broker ids, trade linking/grouping ids and others.  They will tend to be populated rather sparsely based on product type, execution venue… In some cases they will have 1-to-1 relationship with the trade id, in other cases such as deal linking or grouping, a single alternate id will span multiple trade ids.   While, the number of records in this table will get quite large and will likely exceed the number of records in the trade table, size of the data will be rather small and look ups on this table will be direct with very low IO cost. Trade Lifecycle and Amendments  Trade Model needs to support the notion of trade status.  The primary responsibility of trade status is to determine if the trade is live or not, though it can also be effectively used to provide additional information about the trade lifecycle. For example, initially the trade may be entered in the pre-execution state, then the trade will be executed, following execution a trade would be either confirmed or canceled.  These states allow construction of multiple views of position( i.e, only executed vs pending and executed ).   When designing trade states, it’s important to keep in mind that the primary responsibility of the trade model is to provide rich and flexible view of positions. Furthermore, there are significant complexities with document tracking and confirmation/matching workflows with many of the workflow steps not having impact on validity of position, risk and p&l.   Hence, it’s best to keep complex operational workflows in subsystems that use the trade model without duplicating data.   To support complex operational workflows without the need to duplicate trade data, trade model needs to support multiple trade lifecycles.   The above described trade lifecycle supports front and parts of the middle office workflow. However, there are other operational workflows such as trade reconciliation workflow where the reconciliation team needs to see T+1 positions frozen until all breaks have been accounted for and explained.  Having front or middle office teams amend or cancel yesterday’s trades to correct risk will cause disruptions to the reconciliation team.  Trade Model can support both workflows on the same data by providing versioning and secondary trade status to indicate that the trade is active for recon positions in conjunction with having each trade amendment handled by adding a new row.   Example below: Suppose on 3/25 we executed a trade for 1000 shares of yahoo but it was keyed into the system as 100. as below ID TradeId Security Quantity Price Date Portfolio Trade Status Recon Visible 100 10 YHOO 100 45 3/25/15 123 executed Yes On 3/26 it was discovered that the trade was booked in error and was amended with correct quantity as follows. ID TradeId Security Quantity Price Date Portfolio Trade Status Recon Visible 100 10 YHOO 100 45 3/25/15 123 canceled Yes 101 10 YHOO 1000 45 3/26/15 123 confirm No In this model on 3/26, front and middle office views will see record id 101, since they query positions based on trade status while the recon team view will continue to see record 100 since they query positions based on recon visible flag.   Once reconciliation is complete, recon team will turn on status propagation which will mark record 100 as invisible and record 101 as visible for recons. Cancel/Rebook model of trade amendments described above carries overhead of duplicating trade data though in practice, such overhead will rarely exceed 5-10% of the trade table.   Another design consideration is whether or not to keep Recon Visible flag on the trade model itself or in a separate table.   Since there will be 1-1 relationship between the tables, the trade off is between potential deadlocks and update collisions with other updates to the trade table and the cost of table joins and extra storage needed for new table.   In practice, adding a new secondary trade lifecycle is a very rare event as it adds significant operational complexity.  Therefore, it is acceptable to have secondary statuses directly on the trade table. Historical views and audit trail Trade Model needs to support three functions with respect to historical views and audit trail. Ability to show correct positions for a given historical trade date.  This is naturally supported by the trade model leveraging trade status and trade date fields.   Ability to show what positions looked liked at a particular point time.  It’s important to realize that we only need ability to demonstrate what positions looked like, not what trades looked like.  This distinction allows us to apply the cancel/rebook model only to economic/position impacting changes. To support this view, we need to introduce Effective Date and Cancel Effective Date fields to the trade model.  Effective Date will be set once the trade becomes executed, Cancel Effective Date once the trade becomes canceled.  Extending the previous example we will get the following, allowing us to see 100 share trade with effective date of 3/25 and 1000 share trade with effective date of 3/26 ID TradeId Security Quantity Price Date Portfolio Trade Status Recon Visible Effect Date Cancel Date 100 10 YHOO 100 45 3/25/15 123 canceled Yes 3/25/15 3/26/15 101 10 YHOO 1000 45 3/26/15 123 confirm No 3/26/15 Ability to demonstrate an audit trail of all changes made to a given trade.  Audit goes beyond economic changes.  It’s required to audit every field that has changed on the trade indexed by both ID and TradeID.  Audit data should be managed by a separate subsystem, recording every update to the trade. More to come This article only covers the very basics of the trade model.  Fully functional trade models need to support cash management, trade allocations, trade event processing, order fill management, fast position API, data archiving strategies and much much more.  Those components are beyond the scope of this article and will be covered at some point in the future.

Design Pattern for Data Entry Applications

Design Pattern for Data Entry Applications M 917 536 3378 maksim_kozyarchuk@yahoo.com

Overview A classical way to develop data entry applications is to follow either an MVC or an MVP pattern.  Using one of these patterns is a significant step forward from a pattern that most UI builders guide you towards, as it allows separation of business logic from presentation logic reducing complexity and enabling unit testing of the business logic separate from the UI.  While these patterns are powerful, more structure is needed when building rich data entry applications.  A typical data entry application has a few responsibilities and complexities. Multiple entry portals for the same data( i.e. quick entry, detailed entry and API upload) Similar fields appearing on multiple entry forms and having either identical or slightly different behavior Field and model level validation rules that need to have a consistent look and feel Context-based field calculation and defaulting rules that can vary in levels of complexity To support these complexities, while maintaining consistent behavior between entry portals and between fields across multiple screens, we should create the following four layers of abstraction Presentation field(s), responsible for defining datatypes, field level validations, calculation rules associated with the fields and mapping to domain objects Product configuration(s), responsible for defining field list for a given product, relationship and dependencies between fields, product level validations as well as the relationship between product and domain objects Presentation configuration(s), responsible for defining a subset of product fields made available for a particular data entry method. This will include display/presentation names of the fields. Domain object(s), responsible for defining structure of the data as it’s stored  in the database, one of it’s responsibilities would be to capture the core structure of the data normalizing diverse product-specific fields down to a relatively small set of core fields Introduction to ReactiveFramework   In this section, I will introduce ReactiveFramework which will provide a strong definition around Fields, Product Model and interface for external systems( UI or API).   To keep this article grounded in practice, I will use an example of FXTransaction entry where a user needs to specify two out of three fields, (primary amount, fx rate, and secondary amount) having the third field calculates automatically First let’s define a few key components.  Field class provides metadata definition behind the field and should uniquely represent a given field across multiple screens. class Field:   def __init__(self, name, datatype, validation_method = None, calculation_method = None):       self.name = name       self.datatype = datatype       self.validation_method = validation_method       self.calculation_method = calculation_method        The Field class has four components Field name that uniquely identifies the field.   Datatype, while python is flexible about datatypes of fields, having a strong notion of datatype for a given data entry field is key to improving overall quality and avoiding boilerplate code for ensuring type compatibility when performing calculations and validations of data.  Developing in Python, we should ensure correct datatypes as the data is entering the system.    Validation and calculation methods will be bound to the relevant product model and used by the reactive framework to validate and calculate values of relevant fields.    The FieldFactory class is the repository of all of the fields in the system, in practice it should load field definition from a configuration source, but for the purpose of this example, I will include a very basic implementation to demonstrate field setup for FXTransaction example described above. class FieldFactory:   FIELDS = [ Field(name = 'action', datatype = str, domain_name = 'action', validation_method = 'must_be_provided'),              Field(name = 'primary_amount', datatype = Decimal, domain_name = 'quantity', calculation_method = 'calc_primary_amount', validation_method = 'must_be_provided'),              Field(name = 'secondary_amount', datatype = Decimal, calculation_method = 'calc_secondary_amount', validation_method = 'must_be_provided'),              Field(name = 'deal_fx_rate', datatype = Decimal, calculation_method = 'calc_deal_fx_rate', validation_method = 'must_be_provided'),              Field(name = 'commission', datatype = Decimal, calculation_method = 'calc_commission', validation_method = 'must_be_provided'),   ]   @classmethod   def getField(cls, field_name):       for field in cls.FIELDS:           if field.name == field_name:               return field        The ReactiveFramework class defines Public API and interactions supported by the Framework.  For now, public interface will have a constructor which accepts Product Model to define fields used by the product and contain implementation of calculation and validation methods.  It will have a set_field method which needs to be called to set value for a particular field, validate method which will return a list of validation errors and get_value method which will allow user to query value of fields defined in the model.  Below is an example usage of the ReactiveFramework for construction and validation of FXTransaction. >>>fxTrade = ReactiveFramework(FXTransaction()) >>>fxTrade.set_value('action', 'Buy') [] >>> fxTrade.set_value('primary_amount', 100) ["commission"] >>>fxTrade.set_value('deal_fx_rate', 1.5) ["secondary_amount"] >>> fxTrade.get_value("secondary_amount" ) 150.0 >>> fxTrade.get_value("commission") 1.00 >>> fxTrade.validate() {} We construct an instance of the ReactiveFramework by passing in an instance of the Product Model, in this case FXTransaction.  We then call set_value on the action field and back and an empty list in return.  Empty list indicates that no other field depends on the action, implying that no calculations took place.  Next we set_value on primary_amount and get back a list containing commission, this indicates that the commission field has been calculated based on the new value of primary_amount. Next we set deal_fx_rate and get back indication that secondary_amount was calculated.  Then we check values of commission and secondary_amount  to verify that they have been set.  Finally, we call the validate method and get back and empty dict indicating that there are no validation errors.   The FXTransaction class defining the the Product Model is below. class FXTransaction:   FIELD_DEPENDS = {       'action': [],       'primary_amount': ['secondary_amount', 'deal_fx_rate'],       'secondary_amount': ['primary_amount', 'deal_fx_rate'],       'deal_fx_rate' : ['primary_amount', 'secondary_amount'],       'commission' : ['primary_amount']   }      def bind_fields(self):       for field_name in self.FIELD_DEPENDS:           setattr(self, field_name, BoundField(FieldFactory.getField(field_name), self) )              def calc_commission(self):       return self.primary_amount.value * Decimal("0.01")      def calc_secondary_amount(self):       return self.primary_amount.value * self.deal_fx_rate.value   def calc_primary_amount(self):       if self.deal_fx_rate.value:           return self.secondary_amount.value / self.deal_fx_rate.value   def calc_deal_fx_rate(self):       if self.primary_amount.value:           return self.secondary_amount.value / self.primary_amount.value          def must_be_provided(self, field):       return "" if field.has_value else "%s is missing" % field.definition.name      FXTransaction defines fields that are part of the Product Model along with calculation and validation methods required by these fields. FXTransaction is loosely coupled to Fields and FieldFactory, bind_fields method validates correct field setup and establishes binding between field definition and model state. When working within the ReactiveFramework, Fields and Product Models are the primary concern of the developer. When adding new screens or new functionality to the screens, the developer would create new or reuse existing fields and create or modify Product Models which define field interactions and dependencies.   A note on implementation, while it’s possible in python to derive dependencies implicitly from the calculation methods, in the opinion of the author, stating them explicitly has the benefit of simplicity, readability and flexibility which outweigh convenience of implicit dependency calculation.

Implementation of the ReactiveFramework ReactiveFramework itself consists of two classes.  ReactiveFramework class which defines the public API that we saw in the previous section and BoundField class which maintains field state and binding to the Model.   BoundField state consists of: The value of the field A flag indicating if the field has a value which is used as part of the calculation framework to determine if a dependent field is ready to calculate.  Only fields that have values for all of their dependencies can be calculated A flag indicating if the field has been set explicitly through the public API.  The framework will not recalculate/replace value of the field that has been set explicitly through the API Below are the classes that make up the ReactiveFramework class BoundField:   def __init__(self, field_definition, model):       self.definition = field_definition       self.value = None       self.has_value = False       self.has_user_entered_value = False       self.calculation_method = self._bind_method('calculation_method', model)       self.validation_method = self._bind_method('validation_method', model)   def _bind_method(self, method, model):       if getattr(self.definition, method):           return  getattr(model, getattr(self.definition, method))   def set_value(self, value, user_entered =True):       self.value = self.definition.datatype(value)       self.has_value = True       self.has_user_entered_value = user_entered   def recalc(self):       if not self.has_user_entered_value:           if self.calculation_method:               self.set_value( self.calculation_method(), user_entered= False )           return True       return False      def validate(self):       if self.validation_method:           return self.validation_method(self)    class ReactiveFramework:   __slots__ = ('model', 'depends_notifty')   def __init__(self, model):       self.model = model       self.depends_notifty  = {}       self._init_depends_notifty()       self.model.bind_fields()      def _init_depends_notifty(self):       for field_name, deps in self.model.FIELD_DEPENDS.items():           for dep_name in deps:               self.depends_notifty.setdefault(dep_name, [])               self.depends_notifty[dep_name].append(field_name)   def _are_dependents_set(self,field_name):       for dep_field in self.model.FIELD_DEPENDS[field_name]:           if not getattr(self.model,dep_field).has_value:               return False       return True      def _recalc_field(self, field_name, recalculated):       if self._are_dependents_set(field_name):           if getattr(self.model, field_name).recalc():               recalculated.append(field_name)               self._recalc_dependents(field_name, recalculated)   def _recalc_dependents(self, field_name, recalculated = None):       if recalculated is None:                   recalculated = []       for field in self.depends_notifty.get(field_name, []):           if field not in recalculated:               self._recalc_field(field, recalculated)       return recalculated          def set_value(self, field_name, value):       getattr(self.model,field_name).set_value( value )       return self._recalc_dependents(field_name)   def get_value(self, field_name):       return getattr(self.model,field_name).value   def validate(self):       result = {}       for field_name in self.model.FIELD_DEPENDS:           errors = getattr(self.model, field_name).validate()           if errors:               result[field_name] = errors       return result Other components of the data entry platform Strong definition of fields and interactions between them that ReactiveFramwork provides, is core but is a small part of an effective data entry platform.  I will discuss other core components, DomainModel binding  and presentation layers in the future posts.