Real-Time Data Infrastructure Enhancement for Livestock Performance Monitoring

Our scale-up company in the Livestock & Dairy industry is looking to improve our real-time data infrastructure to better monitor livestock performance and optimize decision-making processes. As we scale, the need for a sophisticated data pipeline that supports real-time analytics and provides actionable insights is critical. We aim to implement a robust system that integrates real-time data collection from IoT devices, environmental sensors, and manual inputs to provide a comprehensive view of herd health and productivity. The project will involve designing a scalable and efficient data pipeline using technologies like Apache Kafka for event streaming, Spark for data processing, and Snowflake for data warehousing. MLOps practices will be integrated to automate predictive analytics models that anticipate livestock health issues or productivity dips. The objective is to provide the operations team with real-time dashboards and alerts, enabling proactive interventions and optimal resource allocation. We are looking for a data engineer with strong experience in building data infrastructure using modern tools such as Apache Kafka, Spark, Airflow, and dbt. Familiarity with data observability and governance concepts, as well as experience with cloud platforms like Databricks or BigQuery, will be valuable.