Responsibilities

  • Expert in designing and optimizing data pipelines using Azure Data Factory (ADF), Microsoft Fabric, OneLake, Lakehouse Explorer, and PySpark.
  • Implement and maintain Medallion Architecture layers, ensuring optimal performance across data zones.
  • Develop flexible, well-structured data models aligned with business use cases across Lakehouses and Warehouses.
  • Optimize PySpark-based ETL processes for efficient data transformation and compute utilization.
  • Enable seamless integration with Power BI to deliver impactful visual analytics and dashboards.
  • Collaborate with business stakeholders and BI teams to align data products with reporting needs.
  • Implement enterprise-level data governance, access controls, and data security protocols across pipelines and storage layers.
  • Continuously monitor and optimize pipeline performance in large-scale Microsoft Fabric environments.
  • Deep understanding of data governance, cloud security best practices, and performance tuning.