We're seeking a data engineering rockstar with a knack for mentoring, a strategic mind, and strong communication skills. You'll not only design, develop, and maintain robust data pipelines using Databricks and your ETL expertise, but you'll also play a key role in building our Databricks practice.

Here's what sets this role apart:

Technical Guru and Mentor: You'll be an expert in Databricks, ETL tools (like Informatica), Python, and data modeling concepts. You'll guide junior team members, foster a culture of knowledge sharing, and participate as an expert and learner in team tasks.
Sales Champion: You'll collaborate with the sales team, attending pre-sales calls to showcase your technical expertise and demonstrate the value of Databricks solutions to potential clients. Your insights will be instrumental in closing deals and expanding our Databricks practice.
Data Visionary: You'll have a strategic understanding of how Databricks can empower businesses, translating client needs into compelling technical solutions.

Day-to-Day Technical Skills:

Design, develop, test, deploy, and maintain complex data pipelines using Databricks.
Write and execute data unit/integration tests for data quality using Python and relevant testing frameworks.
Build and maintain scalable data pipelines for both streaming and batch requirements using Spark.
Develop API integrations to support data pipelines and future growth.
Utilize cloud technologies like Azure and AWS to manage large-scale data.
Implement data cleansing, transformation, and validation routines using techniques like data profiling and data quality checks.
Create data models that are clear, concise, and perfectly capture the essence of your client's data.
Ensure data security by championing secure coding practices and adhering to best practices.

Additional Responsibilities:

Partner with the sales team to develop compelling Databricks solution demos and presentations for pre-sales calls.
Articulate the technical benefits and ROI of Databricks to potential clients in a clear and concise manner.
Help craft technical proposals that address client pain points and showcase the value proposition of Databricks.
Collaborate with sales leadership to identify new market opportunities and expand our Databricks client base.

Languages: Python, SQL (likely Spark SQL)
Technologies: Databricks, Apache Spark, Cloud platforms (Azure, AWS)
Data Engineering: ETL tools (Informatica), data modeling, data cleansing, data transformation, data quality checks, data pipelines (batch & streaming), API integrations
Version Control: Git
Scripting: Shell scripting (likely Bash) may be required
Other: Data profiling, secure coding practices, Jupyter Notebooks

bitsIO Inc. jobs

Databricks Consultant