Automating Customer Lifetime Value Analysis in the Financial Sector

Sector: Finance
Duration: May 2024 – January 2025

Work Delivered

• End-to-End Databricks Integration: Designed and implemented an automated data workflow using Databricks, encompassing ingestion, transformation, modeling, and dashboarding.

• AWS Infrastructure Setup: Established a secure, scalable Databricks workspace within AWS, aligned with the client’s cloud strategy.

• Automated Pipelines: Created automated data pipelines to handle new client data, reducing manual input and turnaround time.

• Machine Learning Model: Built and integrated custom CLV models capable of handling varying transaction behaviors across clients.

• Airflow Orchestration: Deployed Apache Airflow to orchestrate and monitor the full data lifecycle, from ingestion to dashboard publication.

People

• Managing Director, Customer Growth – LinkedIn Profile
  Sponsored the project and served as the primary subject matter expert. Although not technical, she played a key role in guiding the CLV strategy and outcomes.

• Product Manager – Acted as the consistent point of contact, coordinating feedback and approvals.

• Support Engineers – Occasionally assisted with AWS access and environment setup.

Pressure

• Reliance on Excel made processes tedious and error-prone

• Manual steps led to inconsistent results and delays

• No data lineage to track how insights were generated

• Each client required a full rework of the same manual process

• Delivering value to clients was slow and inefficient

Impact of Automation:

• Enabled faster iterations and reduced turnaround time

• Significantly decreased human errors

• Created a consistent, reliable, and traceable system

Program/Project

• The project was driven by the client's product manager, who coordinated feedback and approvals.

• Technical decision-making, architecture, and implementation were fully owned by the consulting team.

• The client focused primarily on the output (CLV insights and reports), not on the underlying technology stack.

• Minimal technical support was needed from the client’s side, other than provisioning access and clarifying domain concepts.

Problems and Pains (Pre-Project)

• Slow delivery of insights to clients

• Frequent data quality and cleansing issues

• Lack of centralized data management and history

• No automation, causing repetitive and time-consuming processes

Quantified Impact of Pre-Project Pain Points:

• Client delivery time dropped from months to days

• Data cleaning time reduced from days to a few hours

• Manual data logs replaced by a centralized, queryable data repository

• Laid the groundwork for data governance and compliance

Promises

• Implemented an error-free, fully automated process from data ingestion to dashboard delivery

• Created a reusable and modular system that can easily be scaled to serve new clients

Problems and Pains (In-project)

• Understanding the CLV model and its business logic was challenging

• Translating domain-specific finance insights into technical implementations required extensive collaboration

• The main sponsor, while highly knowledgeable, was not technical, which extended the onboarding of domain logic

How These Were Overcome:

• Conducted structured working sessions at each phase to align business understanding with technical implementation

• Developed prototypes early and iterated with feedback

• Created documentation and visual guides to bridge knowledge gaps

Payoffs

For Stakeholders:

• Reduced client onboarding time from 2-3 months to just 3 weeks

• Gained access to centralized historical data, enabling long-term value tracking and future AI integration

• Simplified client management by standardizing the delivery process

For the Company:

• Increased client satisfaction and confidence due to timely and consistent delivery

• Enhanced ability to scale operations and take on more clients without increasing internal workload

• Positioned the company for future data-driven innovation