Senior/Lead Data Engineer (SC Cleared)

Job DescriptionRole title: Senior & Lead Data Engineer contracts (SC Cleared)Duration of role: 6 months (potential to extend)Location: London (happy to be mostly WFH but would need to attend team meets)Day rate: £550-600 per day - INSIDE IR35UK based candidates with full UK residency only pleaseAny extra Clearance requiredSC requiredParagraph about team/project the person is joiningData engineers in the Data Modelling and Engineering team build and maintain data pipelines to this government body's Analytical Platform from their operational services. Their analytics engineers build dimensional models on top of these data pipelines to provide useful and usable data to downstream users, including analysts, performance teams and data scientists. Over time, this is reducing dependence on manual, legacy processes that are time-consuming, error-prone and inefficient.With new microservices being developed by their digital services to replace existing operational databases substantial data and analytics engineering support is required to maintain critical feeds of data to downstream applications.Current models have been developed in a piecemeal manner for specific reporting requirements. Given the current scale of change, they would like to take a step back to develop a more comprehensive data model which reflects business processes and analytical needs. To build such a model they would take advantage of data linking technology and new data sources / services.They are looking for an experienced, collaborative, Data Engineer to join the team to work with upstream digital teams to accelerate bringing additional microservice data onto their Analytical Platform.RequirementsExcellent python, SQL and jinja skillsExperience with cloud platforms, ideally AWSExperience of orchestrations tools, ideally airflowExperience of building CI/CD workflows, ideally with GitHub ActionsFor Lead position - Experience of leading, enabling and supporting a team of data engineers, guiding them to apply best practicesExperience of ensuring practices are up to date with latest developments in data engineeringExperience of designing and developing data pipelinesExperience of working with business stakeholders and cross-functional teamsDesirableExperience of infrastructure as code, ideally TerraformExperience of using dbt, including implementing dbt best practices