Job Title: Databricks Architect

Location: San Jose, CA (Hybrid)

Fulltime and Contract

Requirements

Key Responsibilities:

Work on client projects to deliver Microsoft Azure based Data engineering & Analytics solutions

Provide hands on technical leadership for relevant technical skills, in multiple projects if required

Design and implement high performing scalable analytics solutions on Azure following well-architected framework guidelines

Design pipelines to ingest & integrate structured/unstructured data from heterogeneous sources

Collaborate with delivery leadership to deliver projects on time adhering to the quality standards

Mandatory Skills

In-depth knowledge Data processing techniques and handling large data sets

Hands on experience using different storage services on Azure for storing data, like ADLS, SQL DB, SYnapse, Delta tables, Hive

Expert level hands on experience using Databricks in at least 4 projects, creating notebooks using Scala/Python

Expertise using Pyspark and Spark SQL for processing data for various use cases

Expertise in code/performance optimization, and troubleshooting PySpark notebooks

Good understanding of Delta lake and how it can be used in conjunction with ADLS and other storage

Experience working with atleast 1 big data project using map-reduce

Experience with Databricks scheduling features to automate Databricks jobs

Problem-solving skills along with good interpersonal & communication skills

Self-starter who can pick up any other relevant Azure Services in the Analytics space

Good-to-Have Skills

Databricks Certified Data Engineer

Knowledge of HDInsight, and Azure DevOps is a big plus

DP 203 - Microsoft Certified Azure Data Engineer

Knowledge in Power BI

Exposure to Databricks on AWS

AVIS TECH LLC jobs