Senior Data Engineer

Senior Data Engineer
£78,000 per annum
Hybrid role - 2 days in London office
Key Responsibilities:

• Collaborate with cross-functional teams to develop robust data pipelines.
• Design, build, and maintain scalable data processing systems, adhering to industry-standard development practices.
• Partner with product and platform teams to understand data requirements, aligning cost optimization strategies with business objectives.
• Ensure adherence to data governance policies, aligning with best practices in data engineering and query cost optimization.
• Develop advanced Python scripts to automate and optimize data processes.
• Apply in-depth knowledge of data engineering to improve data handling, storage methodologies, and overall system efficiency.
• Provide strategic insights to senior stakeholders, aligning data initiatives with broader business goals.
• Stay ahead of emerging technologies and trends in the data landscape to maintain cutting-edge systems.
• Collaborate closely with infrastructure and DevOps teams to support data engineering requirements.

Essential Qualifications:

• Strong proficiency in Python programming, database design, and SQL.
• Proven experience in building batch and streaming data pipelines with tools like BigQuery, Dataflow, and Pub/Sub.
• Expertise in SQL for data extraction, transformation, and loading (ETL) from various data sources.
• Hands-on experience with job automation and orchestration tools such as Apache Airflow.
• Proficient in Data Build Tool (DBT).

Preferred Skills:

• Familiarity with CI/CD tools like Jenkins or GitLab.
• Experience using Cloud Logging and Cloud Monitoring.
• Knowledge of Google Cloud Platform (GCP) security and privacy tools, including Identity and Access Management (IAM), encryption, key management, Data Catalog, and Data Loss Prevention (DLP).
• Proficiency with GCP and GitLab to build and advance data systems.

..