Real-time DeFi Transaction Monitoring and Analytics System

We are a dynamic startup in the Cryptocurrency & DeFi sector, aiming to elevate the transparency and efficiency of our platform through cutting-edge data engineering. Our project involves building a robust real-time transaction monitoring and analytics system that can handle vast amounts of cryptocurrency transaction data. We require an experienced data engineer to architect a solution using Apache Kafka for event streaming, and process this data using Spark for real-time analytics. The system will leverage Airflow for orchestrating data workflows and dbt for transforming data in the warehouse. Snowflake or BigQuery will serve as the central data repository, enabling efficient storage and retrieval for complex query operations. The project demands the integration of data observability and MLOps to ensure data quality and reliability, and to facilitate the deployment of machine learning models for predictive analytics. The outcome will be a comprehensive dashboard providing insights into transaction patterns, potential fraud detection, and user behavior analytics.