Data Engineer
Advert Reference Number: 267
Job Location: Milton Keynes, Remote/Hybrid
Department: Data Technology
Salary: £45,585 to £54,395
Closing Date: 27th September 2024
Weekly Working Hours: 37
Contract Type: Permanent
About the Role
We believe that where you start in life, shouldn’t limit where you go.
The Open University is a unique institution created 50 years ago to open up education for all. We are open to anyone, anywhere, changing millions of lives across the world and always looking for new ways to help people succeed. Our pioneering use of innovative methods, such as our BBC partnership, means we reach more people than any other university.
We work across all four nations of the UK and we are the most popular choice for online, distance, flexible study.
Digital Services bring trusted design, data and technology know-how to the delivery of lifelong learning. Digital Services are responsible for the digital information, systems and technologies used by staff and students across The Open University. Covering all aspects of digital technology at the OU, from sourcing and maintaining systems, through to providing frontline IT support for staff and students.
The effective use of data is key to the successful delivery of the Open University Strategic goals. Reporting to the Digital Integration Hub Lead Data Engineer as part of the Chief Data Office this is a key data engineering role appointment to build and run the Data Technology Capability for the Open University. The successful candidate will be an expert in data engineering with relevant experience with data engineering tools in Azure, programming languages, technology operations, architectures, integration patterns and how to implement these successfully.
The role will be responsible for technical development in Azure Data platform and writing data pipelines for ingestion of data into a data platform.
Key Responsibilities
The Azure Data Engineer will be responsible for:
- Lead, influence, and be a contributor across our entire technology team while acting as an area specialist for your team.
- Deliver end to end data transformation pipelines handling ingestion of both batch and streaming data to deliver OU’s digital services data platform.
- Implement the necessary infrastructure which allow self-healing and observable monitoring of data ingestion, and consumption.
- Take on technical ownership of a workstream, be the subject matter expert and managing risks and issues.
- Define, develop, and maintain artifacts like technical design or user documentation and look for continuous improvement in software and development process within Agile.
- Resolve problems and roadblocks as they occur with help from peers. Follow through on details and drive issues to closure.
- Performance tuning of code and review of fellow developer’s code to confirm adherence to best practices.
- Mentoring of more junior members of the team
- Work with a team of data engineers to ensure the timely delivery of project deliverables that meet requirements.
- Support technology innovation initiatives, through proof of concepts, across the OU
The primary focus for the initial phase of the role will be to:
- Deliver critical data engineering solutions to enable OU to achieve its key strategic programmes.
- Work with Data Architecture function on current and future solutions for reporting and analytics
- Unlock the data platform capabilities to facilitate the delivery of analytics and data science use cases.
- Deliver the data platform following Microsoft technology roadmap.
Skills and Experience
Essential:
The successful candidate will have a proven track record in:
- Experience in communicating with stakeholders at all levels knowing how to manage stakeholders’ expectations and facilitate discussions across high risk or complexity or under constrained timescales.
- Establishing enterprise-scale data integration procedures across the data development life cycle with in-depth knowledge of data engineering on cloud platform in Microsoft Azure.
- Ability to manage a team and motivating them to ensure that data services work efficiently at an enterprise level.
- Implementing cloud-based data engineering solutions with practical experience in relational databases, ETL tools and programming languages (SQL, Python, pySpark )
- Establishing data engineering standards, design patterns and keeping abreast of the best practices in industry
- Technical ability to predict and advise on future technology roadmap for the organisation to deliver business benefits.
- Investigating emerging trends in data-related approaches, performing horizon-scanning for the organisation and introducing innovative solutions.
Desirable:
- Certified as Cloud Data Engineer in Azure
- Knowledge and practical experience of cloud-based development in Azure
- Expert knowledge in Data Factories, Data Lake, Synapse and SQL Database
- Azure Databricks/Spark Cluster understanding/management
- Azure ADF to build pipelines & configuring with GIT for CI/CD
Essential Requirements
As part of the application process, you will be expected to submit your CV and a Supporting Statement (maximum of 1000 words) that demonstrates how you meet the essential/desirable criteria listed above. Flexible working patterns will be considered.
It is anticipated that a hybrid working pattern can be adopted for this role, where the successful candidate can work from home and the office. However, as this role is contractually aligned to our Milton Keynes office it is expected that some attendance in the office will be required when necessary and in response to business needs.
*Early Close: We reserve the right to close adverts sooner than the closing date.
The Open University is committed to equality, diversity and inclusion which is reflected in our mission to be open to people, places, methods and ideas. We aim to foster a diverse and inclusive environment so that all in our OU community can reach their potential. We recognise that different people bring different perspectives, ideas, knowledge, and culture, and that this difference brings great strength. We strive to recruit, retain and develop the careers of a diverse pool of students and staff, and particularly encourage applications from all underrepresented groups. We also aspire to make The Open University a supportive workplace for all through our policies, services and staff networks.