Real-time Data Integration Platform for Renewable Energy Asset Management

As a scale-up company in the renewable energy sector, we are focused on optimizing the performance and maintenance of our solar and wind energy assets. Our current data processing capabilities are limited, affecting our ability to react swiftly to changes in system performance and environmental conditions. We require a robust real-time data integration platform that can ingest, process, and analyze data from multiple sources, including sensors, weather stations, and operational logs. The project involves implementing a data mesh architecture using technologies such as Apache Kafka for event streaming, Apache Spark for real-time data processing, and Snowflake or BigQuery for data warehousing. The platform should also incorporate MLOps practices to deploy and monitor machine learning models that predict maintenance needs and optimize energy output. Data observability and lineage capabilities are required to ensure data quality and traceability throughout the pipeline. The successful candidate will work closely with our in-house data science and IT teams to deliver an efficient and scalable solution that meets our operational and strategic needs. This project not only enhances our asset management but also aligns with our goal of maximizing energy efficiency and reducing downtime.