Real-Time Data Pipeline and Analytics System for Enhanced Travel Insights

As a rapidly growing player in the travel technology space, we are committed to leveraging cutting-edge data engineering solutions to gain competitive advantages. We are looking for an experienced data engineer to develop an end-to-end real-time data pipeline and analytics system. The project will involve setting up Apache Kafka for event streaming to capture and process travel transaction data, customer behavior patterns, and booking trends. Apache Spark will be utilized to handle large-scale data processing, ensuring data is transformed and ready for real-time analysis. We will integrate Snowflake as our cloud-based data warehouse for its scalability and performance capabilities, allowing for efficient storage and querying of structured and semi-structured data. Additionally, Airflow will be employed for orchestrating and scheduling data workflows, while dbt will ensure the transformation and modeling of data meet business requirements. This system will provide our stakeholders with real-time insights, enabling them to enhance customer experiences, personalize travel recommendations, and optimize pricing strategies. With a budget of $15,000 - $50,000, we seek a solution delivered within an 8-12 weeks timeframe, reflecting a high level of urgency due to upcoming travel peak seasons.