Job Title: DataBricks Architect
Location: Normandale, Minneapolis, Minnesota
Job Type: C2C
Job Overview
We are seeking an experienced
DataBricks Architect to lead the design, implementation, and optimization of the Databricks platform. This role involves collaborating with cross-functional teams to develop technical solutions that meet business requirements while ensuring platform performance, scalability, and reliability. The DataBricks Architect will also mentor team members, enforce security and compliance measures, and troubleshoot issues to maintain an efficient platform.
Key Responsibilities
- Architect the design, implementation, and optimization of the Databricks platform for performance, scalability, and reliability.
- Collaborate with teams to understand business needs and provide tailored technical solutions.
- Design and implement data pipelines, ETL processes, and data models.
- Ensure security compliance and implement privacy measures for data protection.
- Troubleshoot and resolve performance issues, errors, and inconsistencies in data workflows.
- Provide leadership to the team, mentoring junior members and fostering a collaborative environment.
Technical Expertise
Key Skills & Qualifications:
- Strong experience with the Databricks platform and its related technologies.
- Proficient in programming languages like Python, Scala, or SQL.
- In-depth knowledge of Apache Spark and its ecosystem.
- Experience with cloud platforms like AWS or Azure.
- Familiarity with Snowflake (nice to have).
- Solid understanding of data engineering best practices, including data modeling and integration techniques.
Leadership & Problem-Solving
- Proven ability to lead teams and mentor junior staff.
- Strong analytical and problem-solving skills, with expertise in troubleshooting.
- Excellent communication and collaboration abilities.
Minimum Qualifications
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in data engineering or architecture roles.
- Hands-on experience with the Databricks platform and Apache Spark.
Preferred Qualifications
- Master’s degree in Computer Science, Engineering, or a related field.
- Experience within the healthcare or medical device industry.
- Certifications in Databricks or related technologies.
- Familiarity with data governance and data quality frameworks.
Focus Areas
- Architecting the Databricks platform for performance and scalability.
- Team leadership and mentoring.
- Building data pipelines, ETL processes, and data models.
- Ensuring security and compliance.
- Troubleshooting and resolving issues for smooth platform operation.