Real-time Data Infrastructure Enhancement for Optimal Energy Distribution

As a leading enterprise in the Solar & Wind Energy industry, we are seeking a comprehensive upgrade of our data engineering infrastructure to support real-time analytics and decision-making. The project involves designing and implementing a data architecture that leverages Apache Kafka for event streaming, Spark for processing, and dbt for data transformation. By integrating these technologies with our existing BigQuery and Snowflake environments, we aim to create a data mesh architecture that provides granular insights into energy production and distribution patterns. Our objectives include minimizing energy wastage, optimizing grid operations, and enhancing predictive maintenance capabilities through MLOps practices. The project will span 12-20 weeks, with a strong emphasis on data observability to ensure high-quality data flows. The successful implementation will allow our organization to respond swiftly to demand changes, reduce operational costs, and improve energy sustainability. We are looking for skilled professionals with experience in the mentioned technologies and a proven track record of delivering complex data engineering projects. This initiative is vital for maintaining our competitive edge in a rapidly evolving market that increasingly values real-time data capabilities.