Real-Time Analytics Platform for Optimizing Subscription Services

In the fast-paced subscription economy, timely and actionable insights are crucial for maintaining competitive advantage. Our enterprise company is embarking on a strategic initiative to develop a real-time analytics platform that will revolutionize our approach to customer engagement and subscription optimization. The project will involve building a state-of-the-art data engineering infrastructure leveraging technologies like Apache Kafka for event streaming, Apache Spark for real-time data processing, and Airflow for orchestrating complex data workflows. The platform will support data mesh architecture principles, enabling decentralized data ownership and enhanced scalability across teams. We aim to implement MLOps practices to automate the deployment of machine learning models, ensuring continuous improvement in customer experience. Data observability tools will be incorporated to monitor data quality and lineage, guaranteeing reliable and accurate insights. The project will span 16-24 weeks and will require collaboration with cross-functional teams, including product managers, data scientists, and subscription service experts. The ideal candidate will have a proven track record in building scalable data platforms and experience with technologies such as Snowflake, BigQuery, and Databricks.