DWS is seeking multiple, experienced
Hadoop Data Engineers / Senior Hadoop Data Engineers for a major multi-year program of work in Sydney. Working in an embedded team with our client, a major BSFI giant, you will be working towards the migration of their petabyte scale on-premises Hadoop data warehouse to a new AWS solution.
This is a career defining opportunity, to work with arguably the biggest name in their field on a massive digital transformation project that will be the anchor to all subsequent work across the enterprise in future.
Type: Max term
Length: up to 2 years
Salary: $170-190K (inc super), depending on experience
Location: CBD Sydney, NSW
Style: Hybrid, approx. 3-4 days in the office
Start: December 2024
Essential Skills And Experience
- Significant professional experience and expertise with Apache Hadoop
- Recent experience in an enterprise BFSI (banking, financial services, insurance) environment
- Expertise in data migration strategies
- Proficiency in HDFS, Hive and YARN
- Worked on major enterprise scale data migration projects
Desirable Skills And Experience
- Experience with Apache Spark
- Worked with AWS: EMR, S3, Glue, Lake Formation, Redshift
- AWS Certification(s): Associate level Developer or Data Engineer
- Other AWS Certifications at Professional / Specialty level
- Experience in data governance, security, compliance
- Experience with Ab Initio / Scala / Sqoop scripts
What You’ll Be Doing
- Design and implement data migration strategies from on-premise Hadoop to AWS
- Develop and optimize Hadoop jobs for data processing and migration
- Ensure data quality and integrity throughout the migration process
- Collaborate with stakeholders to understand and document data requirements
- Perform data mapping and transformation tasks as part of the migration strategy
- Implement and manage data governance and security processes
- Troubleshoot and resolve data migration issues and performance bottlenecks
- Monitor and optimise data workflows for performance and cost efficiency
- Develop and maintain documentation for data processes and architecture
- Coordinate with cross-functional teams to align migration activities with project goals
- Conduct performance tuning and optimization of data processing tasks
- Design and implement automated data pipelines
- Collaborate with AWS data engineering teams as needed
Applicants must hold Australian Citizenship, Permanent Residency (PR), or other permanent work rights - DWS are unable to sponsor or support visas.
Working At DWS Offers You
- Ongoing professional and career development
- Paid learning and certifications (chosen entirely by you), with completion bonuses
- An active social club that host regular, subsidised events
- Corporate discounts on a range of services and products, including health insurance, gym memberships, Apple products and much more
About Us
Established in 1991, DWS is a professional Technology Consultancy and part of HCLTech. With offices across Australia, we have over 700 employees in Melbourne, Sydney, Adelaide, Brisbane and Canberra; providing services to a broad range of blue-chip corporate clients, as well as Federal, State and Local Government agencies. We provide a wide range of services including; IT consulting, managed application services, project management, customer driven innovation, digital transformation, data and business analytics, strategic advisory and productivity, and robotic process automation.
Ready for a new challenge? Click
APPLY now!
www.dws.com.au
DWS encourages applications from Aboriginal and Torres Strait Islander peoples.
No agencies please - our in-house talent acquisition team have this covered.