Data Engineer

Data Engineer Remote - UK with some travel to the office (London, Ipswich, Manchester, Birmingham or Belfast) The contract runs initially to March 25 with scope for extension Inside IR35 (Umbrella) Who are we? We're a software development company building the world's Elastic Workforce, reinventing work and challenging the assumption that a local team = the best team. We help businesses deliver technical projects better than ever before through our platform and on-demand Elastic Teams™. What's in it for you? Our mission is to create freelance jobs with more benefits than permanent. Want to know more? read: About this role We're working with a key partner in the telecoms industry to build a cutting-edge SIEM for our network to ensure full compliance with the Telecommunications Security Act and enhance our security maturity – key priorities for our leadership. After a successful “return to green” in May, we identified crucial roles to drive the project forward. Maintaining our cyber security against external threat actors is a constant challenge. You will work on key cyber capabilities, using state-of-the-art technologies, to provide the threat-hunting teams with the data they need to keep the group, its customers and the UK safe. Your Responsibilities Team Collaboration : Work within a high-performing team of engineers, delivering cutting-edge security tools and solutions. Log Source Onboarding : Take the lead on log source onboarding for Network SIEM, managing the integration of critical data sources. Technical Leadership : Provide expert guidance and liaise with log source platform owners and ASGs to ensure smooth and efficient integration. Elastic Ingest Pipelines : Develop custom Elastic ingest pipelines to effectively parse log data from a variety of sources. Cross-Functional Collaboration : Work closely with the wider SIEM team to ensure successful log source ingestion and ongoing functionality. Design Involvement : Contribute to SIEM design, particularly in areas that enhance log source ingestion and data management. Timely Delivery : Ensure all tasks are completed on time and within budget, maintaining high standards throughout. Documentation : Keep documentation up-to-date, reflecting any changes or improvements in SIEM processes. Continuous Improvement : Actively drive continuous improvement, refining processes and methodologies to enhance team performance. About You We’re looking for passionate technologists who enjoy working in collaborative agile teams. You’ll need to be a clear, concise & engaging communicator with people on your team. We enjoy the big picture and the detail; we want people who excel at both Log Source Data Onboarding : Experienced in onboarding and managing data from various log sources. ETL Expertise : Strong skills in data extraction, transformation, and loading (ETL) processes to manage and optimise data pipelines. Log Augmentation : Proficient in enhancing and enriching log data for better analysis and insight. Teamwork and Collaboration : Demonstrates excellent teamwork, working effectively with stakeholders and team members. Technical Tools : Skilled in Elasticsearch, Kibana, Beats/Elastic Agent, and data collection/filtering through Logstash. Kafka : Solid understanding of Kafka concepts for managing real-time data streams. Regular Expressions : Knowledge of regular expressions for efficient data parsing and manipulation. Additionally, any experience in the below would be welcomed Experience in SIEM or similar cyber security tool development. Agile Delivery : Familiarity with agile methodologies and project delivery. Elastic Fleet : Experience with Elastic Fleet for centralised agent management. Elastic Common Schema (ECS) : Knowledge of ECS for standardising data. Telecoms Security Act (TSA) : Understanding of the Telecoms Security Act and its implications for security compliance. About us Distributed is proud to be an equal opportunities employer. Employees and contractors, as well as prospective employees and contractors, will all be treated equally and fairly. Distributed is committed to ensuring no less favourable treatment is experienced by any current or prospective employee because of any of the protected characteristics under the UK Equality Act 2010 or equivalent local equality legislation. By submitting your application you give us permission to store and use the information from your CV and your answers to application questions.