The Data Engineer will support the client's OCM Self-serve platform. They will be a part of a team with Tech Lead, Business Analyst and a Data Engineer.
Role and Responsibilities:
- Technical Operationalizing and maintaining data pipelines deployed to the Client's investment entities and cultivate a business relationship that assists in meeting business demands
- Translating requirements into solutions
- Deploying ETL/ELT pipelines to achieve a high level of reliability, scalability and security
- Sourcing, transforming and delivering structured and unstructured assets for analytical use
- Optimizing performance of ETL pipelines and data sources
- Participating in solution build, delivery, support and troubleshooting
- Provisioning and managing Azure resources, including security and cost
- Monitoring pipelines. Improving reliability. Automating audit
- Defining and executing technical test plans, performing unit, performance and integration tests and automating regression tests
- Organizational Operating in an Agile development environment
- Communicating effectively with other engineers and various other partners such as product managers, data & business analysts, IT infrastructure specialists, security specialists, enterprise architects and members of senior management
- Leading change and communicating impacts to partners and fellow team members
- Identifying, defining and implementing opportunities for improving existing processes
- Exhibiting the ability to work on multiple projects simultaneously and ensuring timely delivery
- Participating in establishing sound data management practices for the team and the organization
- helping shape and scale the future of our development and production infrastructure
Qualifications:
- Professional Experience:.
- 3+ years solving sophisticated technical data projects using the Microsoft Azure data stack or equivalent work experience. Financial industry application is preferred
- A bachelor’s degree in a scientific or engineering field of study or equivalent work experience
- Microsoft Data engineer associate level certificate will be considered as an asset
- Industry Knowledge of best practices. Previous experience in developing and deploying operational procedures, tuning guides and best practices documentation
- Relationship Building. Consistent track record of building deep technical relationships. Experience in aligning expectations across various partners
- Problem-Solving. The ability to trace data lineage / workflows and resolve technical issues with minimal direction
- Teamwork. Motivated and keen to work in a collaborative environment with a focus on team success
Preferred Skills:
- Enterprise-scale technical experience with cloud and hybrid infrastructures, architecture designs, and technology management
- Experience in handling data in Python using libraries like Pandas, Numpy
- Proficient with SQL
- Experience with data architecture, modeling, data warehousing
- Breadth of technical experience and knowledge across the Azure Data Platform, with depth / Subject Matter Expertise in two or more of the following Data Platform resources:
- Azure Databricks, experience with Python
- Azure Functions and/or Web Apps using C#.Net
- Azure Managed SQL Server
- Azure Data Factory
- Proven experience automating CI/CD pipelines using Azure DevOps or equivalent work experience
- Experience with containerization and container deployment to Kubernetes
- Experience developing star schema reporting pipelines using Kimball or Data Vault v2.0 methodologies