"All candidates must be directly contracted by ASK Consulting on their payroll and cannot be subcontracted. We are unable to provide sponsorship at this moment".
Job Title: Sr. AWS Data Engineer
Location: South San Francisco, CA - Hybrid
Duration: 6 months
Pay rate: $100- $105/hr on W2 AI
Job Description:
We are seeking a highly skilled and motivated Cloud Data Engineer to join our growing data team. The ideal candidate will have a strong background in developing data pipelines, implementing data models, and executing best practices for developing data products. This role requires expertise in AWS, GitLab CI/CD, dbt, Snowflake, SQL, Python, Git, and DevOps, as well as hands-on experience with orchestrators such as Airflow and Automate Now, and data governance technologies like Monte Carlo and Collibra.
Key Responsibilities:
- Develop Data Pipelines: Design, develop, and maintain robust, scalable, and efficient ETL/ELT pipelines to support diverse data sources and large-scale data processing.
- Data Modeling: Create and maintain data models and data architecture to ensure data integrity and optimum performance.
- Best Practices Implementation: Apply industry best practices in data warehousing, data governance, and data lifecycle management.
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to gather requirements and deliver high-quality data products.
- Automation and CI/CD: Implement CI/CD pipelines using GitLab for automated testing, deployment, and integration of data workflows.
- Database Management: Manage and optimize Snowflake databases for performance, scalability, and cost-effectiveness.
- Coding and Scripting: Write efficient SQL queries and Python scripts to extract, load, and transform data.
- Version Control: Utilize Git for version control, ensuring traceable and manageable code changes.
- Orchestration: Utilize orchestration tools like Airflow and AutoMateNow to manage and automate data workflows.
- Data Governance: Implement and manage data governance and data observability solutions using technologies like Monte Carlo and Collibra.
- Monitoring and Optimization: Monitor data pipeline performance and troubleshoot issues to ensure reliability and efficiency.
- Documentation: Maintain comprehensive documentation for data pipelines, models, and processes.
Qualifications:
- Education: Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field.
Experience:
- Minimum 5 years of experience in data engineering or a similar role.
- Proven experience with AWS cloud services related to data processing, such as S3, Redshift, Lambda, Glue, and Data Pipeline.
Technical Skills:
- Proficiency in designing and implementing CI/CD pipelines with GitLab.
- Experience with dbt (data build tool) for transforming data within the warehouse.
- Strong knowledge of Snowflake with hands-on experience in data warehousing solutions.
- Expertise in SQL for data querying, manipulation, and optimization.
- Proficiency in Python for scripting and automation tasks.
- Familiarity with DevOps practices and tools.
- Experience with orchestrators like Airflow and Automate Now.
- Knowledge of data governance and observability tools such as Monte Carlo and Collibra.
- Strong working knowledge of Git for version control.
Soft Skills:
- Strong problem-solving and analytical skills.
- Excellent communication skills, both written and verbal.
- Ability to work collaboratively in a fast-paced, dynamic environment.
- High attention to detail and commitment to producing high-quality work.
Preferred Qualifications:
- AWS Certified Data Analytics Specialty or AWS Certified Solutions Architect.
- Experience with real-time data processing and streaming technologies (e.g., Kafka, Kinesis).
- Experience in a regulated industry
- Experience in a manufacturing environment
About ASK: ASK Consulting is an award-winning technology and professional services recruiting firm servicing Fortune 500 organizations nationally. With 5 nationwide offices, two global delivery centers, and employees in 42 states-ASK Consulting connects people with amazing opportunities
ASK Consulting is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all associates.