Job Summary
We are looking for a Senior Data Engineer to join our dynamic team. This role is ideal for someone with a deep understanding of data engineering and a proven track record of leading data projects in a fast-paced environment.
Key Responsibilities
- Design, build, and maintain scalable and robust data pipelines using SQL, Python, Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark.
- Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms.
- Implement continuous integration and continuous deployment (CI/CD) practices for data pipelines to improve efficiency and quality of data processing.
- Work closely with data architects, analysts, and other stakeholders to understand business requirements and translate them into technical implementations.
- Oversee and manage a team of data engineers, providing guidance and mentorship to ensure high-quality project deliverables.
- Develop and enforce best practices in data governance, security, and compliance within the organisation.
- Optimise data retrieval and develop dashboards and reports for business teams.
- Continuously evaluate new technologies and tools to enhance the capabilities of the data engineering function.
Qualifications
- Bachelor's or Master’s degree in Computer Science, Engineering, or a related field.
- 10+ years of industry experience in data engineering with a strong technical proficiency in SQL, Python, and big data technologies.
- Extensive experience with cloud services such as Azure Data Factory and AWS Glue.
- Demonstrated experience with Databricks and Snowflake.
- Solid understanding of CI/CD principles and DevOps practices.
- Proven leadership skills and experience managing data engineering teams.
- Strong project management skills and the ability to lead multiple projects simultaneously.
- Excellent problem-solving skills and the ability to work under tight deadlines.
- Strong communication and interpersonal skills.
- Excellent understanding of data architecture including data mesh, data lake and data warehouse.
Preferred Qualifications
- Certifications in Azure, AWS, or similar technologies.
- Certifications in Databricks, Snowflake or similar technologies
- Experience in the leading large scale data engineering projects
Working Conditions
- This position may require occasional travel.
- Hybrid work arrangement: two days per week working from office in London