Role: Azure Databricks Architect
Location: Cheektowaga, NY 14225 (100% Onsite)
Contract
Azure Databricks Architect
Job Description
An Azure Databricks Architect is a specialized role focusing on designing, implementing, and optimizing data solutions using the Azure Databricks platform. Here's a typical job description for this role:
Total Experience: 12+ years
Job Summary
We are seeking an experienced Azure Databricks Architect to join our team. The ideal candidate will have a deep understanding of the Azure Databricks platform, Big Data technologies, and the broader Azure ecosystem. You will be responsible for designing and implementing scalable, high-performance data architectures and providing strategic technical direction to ensure successful delivery of complex data projects.
Key Responsibilities
Architectural Design:
- Design and architect end-to-end data solutions using Azure Databricks, ensuring alignment with business requirements and technical constraints.
- Develop scalable and optimized architectures for data processing, storage, and retrieval.
- Integrate Azure Databricks with other Azure services such as Azure Data Lake, Azure Synapse Analytics, and Azure Machine Learning.
Technical Leadership
- Provide technical leadership and guidance to development teams on best practices for using Azure Databricks.
- Mentor and train team members on Azure Databricks and related technologies.
- Conduct code reviews and ensure adherence to established architectural standards and guidelines.
Solution Implementation
- Lead the implementation of data pipelines, ETL processes, and data transformation tasks on Azure Databricks.
- Collaborate with data engineers, data scientists, and other stakeholders to build and deploy data solutions.
- Ensure the reliability, scalability, and performance of data platforms.
Optimization And Troubleshooting
- Optimize data workflows and performance tuning within Azure Databricks.
- Troubleshoot and resolve complex technical issues related to data processing and integration.
- Monitor system performance and implement improvements as needed.
Stakeholder Management
- Collaborate with business stakeholders to understand data requirements and translate them into technical solutions.
- Communicate complex technical concepts clearly to non-technical stakeholders.
- Work closely with project managers to ensure timely delivery of projects.
Qualifications
Technical Skills:
- Extensive experience with Azure Databricks, Apache Spark, and Big Data technologies.
- Strong proficiency in Azure services, including Azure Data Lake, Azure Synapse Analytics, and Azure Data Factory.
- Solid understanding of data warehousing, ETL/ELT processes, and data architecture principles.
- Experience with programming languages such as Python, Scala, or Java.
- Knowledge of SQL and NoSQL databases.
Experience
- 5+ years of experience in data architecture, data engineering, or related roles.
- Proven track record of designing and implementing large-scale data solutions in the cloud, specifically on Azure.
- Experience with machine learning and AI integration in data pipelines is a plus.
Soft Skills
- Strong analytical and problem-solving skills.
- Excellent communication and interpersonal skills.
- Ability to work in a fast-paced, collaborative environment.
- Strong leadership abilities and experience leading technical teams.
Preferred Certifications
- Microsoft Certified: Azure Solutions Architect Expert
- Microsoft Certified: Azure Data Engineer Associate
- Databricks Certified Associate Developer for Apache Spark
Educational Background
- Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.