Job Title: GCP, Vertex AI Data Engineer
Location: Remote
Duration: FTE
Job Summary:
We are looking for highly skilled Data Engineers with experience in Google Cloud Platform (GCP) and Vertex AI to join our team. The ideal candidates will be responsible for designing, developing, and maintaining robust data pipelines and architectures, enabling seamless data integration and processing for AI and machine learning initiatives. These roles will require a strong understanding of big data technologies and the ability to optimize data workflows for performance, scalability, and reliability.
Key Responsibilities:
- Design and Develop Data Pipelines: Create and manage scalable data pipelines using GCP and Vertex AI to support data ingestion, processing, and transformation.
- Collaborate with Data Scientists and AI/ML Engineers: Work closely with data scientists and machine learning engineers to integrate and prepare data for model training, testing, and deployment.
- Optimize Data Workflows: Ensure data pipelines are optimized for performance, scalability, and reliability to handle large datasets and complex data environments.
- Data Security and Compliance: Implement data security measures and ensure all data processes comply with industry standards and regulations.
- Continuous Improvement: Monitor data pipelines and workflows continuously, identifying opportunities for improvement and implementing changes to enhance efficiency and reliability.
Qualifications:
- Proven experience in data engineering, with a strong focus on building and managing data solutions on GCP and Vertex AI.
- Expertise in SQL, Python, and big data technologies such as BigQuery, Dataflow, and Dataproc.
- Solid experience in data modeling, ETL processes, and cloud storage solutions.
- Ability to work effectively with large datasets and complex data environments, ensuring data integrity and consistency.
- Strong analytical and problem-solving skills with a proactive approach to identifying and addressing potential data issues.
- Excellent verbal and written communication skills, with the ability to explain technical concepts to non-technical stakeholders.
Preferred Qualifications:
- GCP certifications such as Professional Data Engineer or Professional Machine Learning Engineer are a plus.
- Familiarity with machine learning frameworks and libraries, and experience working in AI/ML-driven environments.
- Understanding of DevOps practices and experience with CI/CD tools and infrastructure automation.
- Experience in implementing data governance frameworks and practices.