About Us
MLCommons is a non-profit dedicated to making machine learning (ML) better for everyone. We are funded by our 60+ member organizations such as Alibaba, Google, Intel, and NVIDIA, as well as researchers at academic institutions such as Harvard University and University of Toronto. We support 15+ working groups focused on benchmarks (e.g., MLPerf™), data, and research. We built and operate MLPerf™, the industry-standard benchmark suite for measuring ML performance for systems of all sizes from the smallest IoT device to the world’s largest supercomputers. We have also released some of the world’s largest open and freely available datasets that enable ML for speech technologies in 50 different languages and add tremendous diversity to image recognition.
Job Description:
MLCommons had embarked on a new project to build a new AI safety benchmarking platform. This is a multi-organizational effort to establish a public framework and toolset for the creation, operation, and presentation of benchmarking scores that measure existing and future generative AI models and systems. Read about our recent v0.5 POC release here: https://mlcommons.org/2024/04/mlc-aisafety-v0-5-poc/
We are looking for a senior system developer to be a core part of the team to build our AI Safety Platform. You will play an essential role in designing, developing, documenting, maintaining, and supporting the systems and tools that allow users to create, modify, view and share AI tests, benchmarks, results and scores. Our primary customers are creators of models needing testing, consumers and users of models, and regulators in scientific and commercial domains.
Responsibilities:
- Systems Design and Creation: Design, develop, and maintain RESTful services, CLI tools, and python packages that comprise the AI Safety Testing Platform. Assist in the design, documentation, and support of public python APIs for core objects.
- Collaboration: Collaborate with internal and partner organizational teams to integrate backend systems with external services and components. Provide support for our tooling and APIs for technical users.
- Optimization: Identify and resolve performance bottlenecks and scalability issues to ensure the system operates efficiently.
- Security: Implement robust security measures to protect data and maintain compliance with industry standards.
- Documentation: Document code, APIs, and system architecture for reference and collaboration purposes.
- Troubleshooting: Investigate and resolve technical issues, bugs, and errors in a timely manner.
- Code Review: Participate in code reviews to maintain code quality and foster a collaborative development environment.
Desired Qualifications:
- Bachelor's degree in Computer Science, Software Engineering, or a related field (or equivalent work experience).
- Strong proficiency in Python programming.
- Solid understanding of RESTful architecture and API design principles.
- Strong professional verbal and written communication skills. You should be able to write specs and understand requirements from others.
- Knowledge of database systems (SQL and NoSQL) and data modeling.
- Excellent problem-solving and debugging skills.
- Familiarity with with software development best practices (e.g. testing frameworks, CI/CD systems, source control)
- Ability to work effectively in a remote, asynchronous work environment in North American time zone hours.
- Self-motivated, detail-oriented, and able to work independently.
- Experience with cloud platforms (e.g., AWS, Azure, Google Cloud).
Benefits:
- This is a contract position for at minimum 9 months with continuances possible.
- Compensation commensurate with experience.
- 100% remote work environment.
- Flexible working hours.
- Opportunity to work on high-visibility, high-impact systems in a diverse, collaborative team.
Application Process:
To apply, please submit your resume and a cover letter detailing your relevant experience and why you are a great fit for this role. Include any relevant portfolio or GitHub links, if available. We look forward to hearing from you!