Designing a data platform is no easy task, particularly when there are new technologies, techniques, and approaches appearing every week. At Cox Auto UK we have been on a journey from manually deployed Hadoop clusters to a full platform as a service setup using Azure Databricks.
This journey hasn’t always been smooth however and we’ve learned some things along the way! In this talk, we examine how we have made design choices while evolving our platform, our decision to open source some of our work, and what our past, present, and future look like. James Fielder, Senior Data Engineer, at Cox Automotive presents The Evolution of a Data Platform.
Also, see the Cox Auto case study, on the Unravel Data blog.
DataOps Resource
Unravel’s guide tells you what it takes to do DataOps. 