Job Title: Data Engineer
Location: Westmont, IL (On-site)
Duration: 12 Months
Implementation Partner: Blue.cloud
End Client: To be disclosed
Role Overview
The client is seeking a Data Engineer with strong experience in PySpark, Azure, and Python to support and enhance their NextGen platform. The selected candidate will be responsible for integrations, bug fixes, expansions, and improvements to the existing system.
Mandatory Qualifications
- Strong PySpark experience: Essential for developing and optimizing data pipelines and transformations.
- Extensive Python experience: Proficiency in core Python programming and scripting.
- Azure Databricks: Demonstrated expertise in using Databricks for data engineering tasks.
- Azure Data Factory & Azure Data Lake: Hands-on experience with these services for orchestrating and managing data workflows.
- Proven ability to build re-usable frameworks, libraries, and data platform capabilities.
- Experience with REST API and GraphQL APIs: Knowledge of API integration and management.
- Experience with Data Quality (DQ) capabilities: Building re-usable DQ capabilities is highly desirable.
- Ability to work independently and demonstrate system design thinking.
Preferred Qualifications
- Experience working in a data lake environment.
- Background in building re-usable Python-based frameworks.