Agile Analytics Lab Case Study


Financial Services

Overview:  We recently worked with a large financial services company that was challenged with massive amounts of data stored across several different legacy databases, all managed by different teams. The company partnered with Fulcrum using our Agile Analytics Lab to experiment with modern big data solutions. Their goal was to find the best way to organize all their data into a single location and provide business users with new insights across these data sets. Their internal IT team was back-logged and wasn’t up-to-date on the latest big data tools and so our lab, staffed with top data scientists and big data technology experts, was able to deliver a solution much more quickly and cost effectively than they could have done on their own.

Download an overview of our Agile Analytics Lab

Get started today with a free consultation


Our Solution


As is the case with most large financial services companies, our client had massive amounts of information stored in disparate databases managed by different teams. These datasets included purchase history, customer information and product information. There was no way for anyone in the company to view data across these systems and because of the size of the data sets, a unified view was not feasible with a single machine or a standard database.

Fulcrum created reusable data processing pipelines, loaded the data into Hadoop and readied it for analysis. Once the central repository was established, we then cubed the data using Hive query language with Spark as the compute engine. This allows fast querying of metrics of interest across all combinations of relevant variables. The output table is in HBase, a key value store, and is queried with Impala which is made for rapid response and returns the results quickly. Finally, a Tableau dashboard was created to view measures and dimensions so the business end users can utilize the results.

Aggregating this information in one location allows business users a holistic view of their data that was not previously available. Best of all, we were able to accomplish this without requiring time from their internal IT team. All Fulcrum requested was a file transfer onto our secure servers and our team assumed the burden of data management and processing. Through an iterative process, we tested multiple solutions in parallel and were quickly able to determine the best technologies for this use case. The client was up and running quickly and is now able to unlock new insights and new value from their data. This set them up for greater exploitation of their data assets. We’re helping then rapidly develop and tune machine learning models, while providing training and support to their analytics team on cutting edge technology such as Tensorflow, Keras, and Theano for deep learning. The lab proved to be the perfect instrument to bring agile analytics to this financial services company.