Skills: Expertise in SQL, database management, and performance tuning. Experience with Python is highly preferred. Strong problem solving and critical thinking abilities. Attention to detail and commitment to data accuracy.
Candidate Value Proposition
- The successful candidate will have the opportunity to network, build valuable connections, and gain job experience.
Job Description:
As a Data Engineer, you would play a critical role in development and maintenance of new and existing Azure data pipelines for various analytical use cases inclusive of AI/ML/GEN AI. You would collaborate with other highly skilled data engineers, data scientists in designing, developing, and deploying data pipelines, ETL processes, and other data integration solutions to extract, transform, and load data from various sources into SQLMI. This data engineer has accountability as a data domain subject matter expert, building automated capabilities and data products to accelerate advanced analytics.
Position Responsibilities:
• Work with data professionals to transform business requests into value-add data products and capabilities.
• Automate, re-build, and re-imagine data processes to increase efficiencies in business processes.
• Design, develop, and maintain scalable ETL/ELT pipelines.
• Develop and uphold the data architecture, encompassing data models, schemas, and data dictionaries.
• Integrate data from diverse sources, ensuring quality and consistency, and prepare it for analytical and operational uses.
• Adhere to data governance policies and processes to safeguard data security, privacy, and compliance.
• Demonstrate the ability to learn and apply best practices in big data and cloud computing to ensure scalable, efficient, and effective solutions.
Must have skills:
- 3+ years of experience as a Data Engineer.
- Advanced Experience in automating data pipelines and working with data transformation tool
- Advanced experience with programming languages including Python, SQL, and Spark, C#
- Solid understanding of data warehousing concepts and relational data modeling.
- Strong working knowledge of SQL queries, Databricks, and SQL Server.
- Advanced experience with cloud technologies, particularly Azure, and Data as a Service (DaaS).
Nice-To-Have Skills:
- Familiar with DevOps and CI/CD pipelines in the automation of ETL workloads.
- Experience in data processing, performance analysis, tuning, and capacity planning.
- Understanding of Agile Scrum methodologies and experience working in an Agile team, including familiarity with collaboration tools such as Teams and JIRA.
- A commitment to continuous learning from both successes and failures, with an openness to change and improvement.
- Familiar with production release and operational support models.
Education:
- University degree in quantitative-based field (e.g. Computer Science, Mathematics, Physics, and Engineering).
Additional Soft skills;
- Excellent communication and interpersonal skills.
- Strong analytical, problem-solving, and solution-oriented abilities.