Responsibilities

  • Design, develop, and maintain cloud-based ETL and ELT pipelines using Azure Data Factory for enterprise data integration needs.
  • Build scalable data ingestion workflows to move data from hybrid sources including on-premise systems, Azure SQL, and cloud platforms into data lakes and warehouses.
  • Develop and manage ADF pipelines, triggers, integration runtimes, and mapping data flows to support automated processing.
  • Implement orchestration and scheduling mechanisms for event-driven and recurring data workflows.
  • Perform data transformation and cleansing using ADF Data Flows to prepare structured datasets for reporting and analytics.
  • Write and optimize SQL queries for data extraction, transformation, validation, and performance improvement.
  • Utilize Python and PySpark for advanced processing, automation, and data engineering requirements.
  • Monitor pipeline execution, troubleshoot failures, and ensure reliability, scalability, and operational efficiency of data systems.
  • Collaborate with analytics, BI, and engineering teams to deliver high-quality and business-ready data solutions.


Relevant Jobs