Data Engineering Solution for Real-Time Production Monitoring in 3D Printing

As a growing SME in the 3D Printing & Additive Manufacturing industry, we aim to enhance our production monitoring capabilities through an advanced data engineering solution. Our current challenge lies in the lack of real-time visibility into our production processes, which hinders our ability to make swift decisions that could significantly impact efficiency and productivity. We require a freelancer with expertise in modern data engineering technologies such as Apache Kafka for event streaming, Spark and Databricks for complex analytics, and Snowflake or BigQuery for scalable data storage. The project will also involve implementing data observability practices to ensure data quality and leveraging tools like Airflow and dbt for efficient data pipeline management. The solution should facilitate real-time insights into key production metrics, enabling our teams to proactively address issues and optimize operations. The desired outcome is a robust system capable of integrating seamlessly with our existing infrastructure and providing actionable insights to drive strategic decisions.