Machine Learning Platform Engineer
$220,000 + $500,000 equity (4 year vest + annual refreshers)
Series E, AI company - 1000 employees
Hybrid - 3 days per week in the office in SF (Union Square)
Our client is a leading late-stage startup in the AI space, boasting a team of over 1,000 talented professionals. They are at the forefront of AI innovation, and they're looking to expand their engineering team to support their machine learning initiatives.
We seek a highly skilled Machine Learning Platform Engineer who is passionate about working with machine learning teams and has a strong background in big data technologies. As a Machine Learning Platform Engineer, you will be crucial in building and optimizing data pipelines, enabling seamless data flow for machine learning applications.
Key Responsibilities:
- Collaborate closely with machine learning teams to understand their data needs and provide robust data solutions.
- Develop and maintain data pipelines using Apache Spark for large-scale data processing.
- Implement real-time data processing solutions using streaming technologies like Apache Kafka or Apache Flink.
- Optimize and manage feature stores to ensure efficient data access for machine learning models.
- Perform feature engineering to prepare data for training and serving machine learning models.
- Ensure data quality, reliability, and performance across all data engineering processes.
Key Qualifications:
- Proven experience in data engineering, with a focus on big data technologies.
- Expertise in Apache Spark and big data feature engineering.
- Hands-on experience with real-time data processing and streaming technologies (e.g., Apache Kafka, Apache Flink).
- Strong understanding of feature stores and their optimization.
- Ability to work effectively within machine learning teams and contribute to their success.
- Excellent problem-solving skills and attention to detail.
Preferred Qualifications:
- Experience in AI or machine learning projects.
- Familiarity with cloud-based data solutions (e.g., AWS, GCP, Azure).
- Strong programming skills in languages such as Python or Scala.