Responsibilities

  • Design, develop, and manage scalable data pipelines using Google Cloud services like BigQuery and Dataflow.
  • Implement data ingestion, transformation, and storage solutions while ensuring high performance and reliability.
  • Optimize and maintain data workflows using real-time streaming tools, including Apache Beam and Cloud Composer. 
  • Develop ETL and ELT processes to ingest, transform, and store structured and unstructured data from diverse sources in cloud storage.
  • Ensure data security, governance, and compliance by implementing access control and encryption policies.
  • Monitor, debug, and optimize data pipelines for seamless analytics and business operations.
  • Collaborate with data scientists and analysts to deliver reliable, high-quality datasets for decision-making.
  • Automate data processes using Terraform, Python, or other scripting languages for operational efficiency.
  • Stay updated with GCP advancements and implement best practices to enhance data engineering workflows.