Description
Looking for a Data Engineer who has a passion for building scalable and high performing data pipelines and data services.
Responsibilities
- Design, implement, optimize, and maintain data pipelines and reports using Python, SQL, Java, Spark, PySpark, Java
- Collaborate with data analysts, and engineering teams to gather requirements, and build data pipelines/services
- Work closely with operation teams to support the operation of data deliverables
- Ensure data integrity and quality through rigorous validation and testing procedures.
Key Qualifications
- Proficiency in building data pipelines with Python and Java
- Proficiency in large scale data processing technologies such as Spark, Kafka and Flink
- Familiarity with RDBMS and solid experience with SQL for data manipulation and reporting
- Experience building scalable solutions in cloud computing environment such as AWS EKS and S3
- Experience in data analytics with strong analytical and problem-solving skills
- Good to have HDFS knowledge.
- Strong communication skills, team collaboration skills, and managing multiple tasks effectively