Real-Time Data Pipeline for Tenant Experience Optimization in Smart Buildings

As a rapidly growing PropTech startup, we aim to leverage data to enhance tenant experiences in smart buildings. We need a talented data engineer to design and implement a real-time data pipeline that aggregates and analyzes data from IoT sensors, tenant feedback, and building management systems. The solution should enable property managers to access insights on tenant preferences and predict maintenance needs, ultimately improving tenant satisfaction and reducing operational costs. The core components of the project include data ingestion using Apache Kafka for event streaming, transformation and processing with Spark, and data orchestration through Airflow. The processed data should then be stored in a Snowflake or BigQuery data warehouse for further analysis and visualization. Additionally, the pipeline should incorporate data observability principles to ensure data quality and reliability. This project is crucial for gaining a competitive edge in the smart building market by offering a superior tenant experience.