To cache or not to cache?
In situations where the source of the data is distant, expensive and/or slow, the storing of frequently accessed data in a central repository close to the data consumers can reduce cost and data access time. The principle of caching is ubiquitous and the concepts are used across industries, applications and designs.
On the surface, caching is a simple concept: a consumer fetches data first from the cache, if data is not available, the consumer or cache fetches it from the source, and populates the cache for the benefit of other consumers. In practice however, caching involves complex mechanisms for holding data, scaling the repositories, managing data lifecycle and so on, all of which are beyond the scope of this post.
For our applications in the world of Market Data, this caching principle can be used to store reference/static data. This, at least in theory, will have two advantages:
- It reduces the cost of data pulled down from vendors
- It centralises storage of data and therefore improves data operation tasks and data quality
Caching and data storage are at the core of EDM solutions on the market today. The two points above are what EDM vendors preach and practical uses have been the subject of numerous debates and conference topics. Themes that generally come up are:
- In a centralised data architecture, who owns the data and who determines the best data ‘quality’?
- Who pays for the data cache / EDM platform?
- Who pays for the data?
Using XMon to understand caching ROI
It seems intuitive that reducing the number of data requests to vendors would reduce associated costs. How much will these costs be reduced by depends on the data vendor’s pricing, data categories pulled down and asset classes requested. Quantifying the saving however, can be a tricky question to answer.
XMon provides a Data Analysis report which can be generated on-demand and answers just that.
XMon analyses all requests for data made over a given period and generates a report to help understand how much cost savings and caching solution would provide and to provide optimisation recommendations in order to reduce data spend, even without a data caching solution.
Understand how much cost savings a data cache solution would provide
- XMon identifies duplicate requests for data, their associated costs and where they originate from.
- The screenshot below illustrates this in action, and suggests that for this particular month, a saving of up to 8% on the data bill would have been achieved through the use of a data cache, solely on the basis of duplicate requests.
Costs attributed to duplicate requests
Optimise data requests to reduce costs
- XMon will make recommendations as to how savings can be made by suggesting optimisations to the data requests sent to vendors. These recommendations can be brought up with the business for validation with the aim of reducing costs.
- The screenshot below shows how XMon identifies low volume / high cost requests, which can lead to data cost optimisations. In the screenshot below, 31.4% of monthly data cost is caused by two data fields. Whether these fields are required or just ‘nice to have’ can be discussed with the data consumers.
XMon identifies low volume / high cost requests
XMon provides the evidence needed for a proper assessment of a caching infrastructure, specifically related to cost reductions and cost optimisations. The XMon Data Analysis report is available to all XMon customers.
Get in touch if you’d like to discuss your requirements for data spend transparency and optimisation or to get a live demo of the product.