CDO Battlescars Takeaways: Applying Data Science to Traditional Market Data Analysis at PFM
In this episode of CDO Battlescars, Sandeep Uttamchandani, Unravel Data’s CDO, speaks with Manish Chitnis, previous CDO at Partner Fund Management (PFM). They discuss his battlescars in Applying Data Science to the Traditional Market Data Analysis at PFM.
Manish has 20+ years of diverse multidisciplinary experience across a wide range of analytics, including architecting data warehouses from scratch, building data-driven apps for risk management, introducing new data architectures, instituting data governance and stewardship, data-hygiene and cleanup, improved data collection, and much more. Here are the key talking points from their chat.
Building a Data Warehouse at PFM
When Manish first joined PFM, he ended up transforming his job from risk management into more of a data science role.
- He worked to construct a data warehouse almost from scratch and reduce the amount of time it took to get the correct data points. He then cleaned that up to have usable information.
- By building a new data warehouse, Manish reduced the amount of time it took to produce critical reports from about 20 days to less than one day, which saved money and resources.
What Type of Data Does PFM Work With?
Most of the data at PFM is generated by their trading behavior. While raw data is not very complicated, there is a lot of metadata that can be generated off of the core data.
- At PFM, data is also generated from newer technologies, such as geolocation data, credit card data sets, information about foot traffic, and information about transactions.
- There is also data that comes from more traditional sources, including from financial statements, and information about book value, market capitalization and liquidity, return on investment, and stock price volumes.
- At PFM, all of the data is on-premises, but they are in the process of migrating that data to the cloud. Manish does not see many use cases where data should be on-prem since security is stronger on the cloud.
Thinking Like a Data Scientist
When first building a data science platform, one hurdle is likely to be building a data-driven culture, where everyone can think like a data scientist and looks at data as something that can benefit them.
- To Manish, “thinking like a data scientist” means not just focusing on the existing processes, but thinking about the kind of consistent innovation that is possible in the field.
- Domain knowledge gives you the ability to determine what questions to ask and what hypotheses to present when looking at data. Domain knowledge also allows you to understand the limitations of certain processes.
- In areas where the data scientists at PFM are not domain experts, they choose to outsource, hiring intermediaries who can interpret the data and do the needed domain specific cleaning.
While we’ve highlighted the key talking points here, Sandeep and Manish talked about so much more in the full podcast. Be sure to check out the full episode and the rest of the CDO Battlescars podcast series!