Overview

As SilverCreek continues to stay at forefront of leveraging data to drive innovation and business growth, we are currently seeking a Data Engineer with a specialization in Databricks to join our dynamic team. The ideal candidate will have a strong foundation in data engineering principles, extensive experience with Databricks, and a passion for continuous learning.

Responsibilities

Design and Implement Data Solutions: Architect and implement high-performance data solutions using Databricks, ensuring efficiency, reliability, and scalability.
Data Processing and Analysis: Utilize Spark streaming and Delta Live Tables for real-time data processing and analysis.
Optimization: Conduct storage (Delta) and cluster optimizations to enhance performance and reduce costs.
Development Practices: Adhere to and promote good development practices within Databricks environments, including effective use of GIT and Databricks DevOps principles.
Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions that support data-driven decision-making.

Qualifications

Experience: Minimum of 2-3 years of hands-on experience with Databricks.
Technical Skills: Proficient in Spark streaming, Delta Live Tables, Python programming, and the ability to code and troubleshoot complex issues.
Architecture Understanding: Solid understanding of Medallion architecture and its application in data solutions.
Optimization Skills: Experience with storage (Delta) and cluster optimizations.
Development Practices: Familiarity with development practices in Databricks environments, including version control with GIT and some exposure to Databricks DevOps.
Education: Bachelor’s degree in Computer Science, Engineering, Information Technology, or equivalent combination of education and experience.
Cloud Platforms: Experience with major cloud service providers (e.g., AWS, Azure, GCP), understanding their data services and how to integrate them with Databricks.
Database Management: Strong background in SQL and NoSQL database technologies, data modeling, and ETL processes.
Data Warehousing: Experience with building and maintaining data warehouses, including knowledge of data warehouse design and dimensional modeling.

Desirable Attributes

Big Data Technologies: Familiarity with a range of big data technologies beyond Databricks, such as Hadoop, Kafka, and others.
Attention to Detail: Exceptional attention to detail, ensuring every technical deliverable meets the highest standards.
Analytical Mindset: Strong analytical abilities to tackle complex reporting challenges and optimize solutions.
Effective Communication: Ability to bridge the gap between non-technical stakeholders and the technical team, ensuring clarity and understanding across the board.
Proactive Approach: A self-starter always looking for opportunities to enhance processes and improve outcomes.

Join Us

We value long-term relationships in all spheres of business. When it comes to taking care of our energetic and highly skilled roster, we take no shortcuts. We offer a wide array of benefits that are both integral and exceptional. On top of our extensive benefits program, we are committed to providing the tools and guidance needed to increase ease of performance and elevate the individual career potential of each employee. Our complimentary equipment allowance includes a smartphone of your choice that comes with a fully paid plan, and all team members are provided with the latest Asus laptops and monitors to bolster maximum efficiency. We are strong proponents of balancing work and life while rewarding those who go the extra mile - all overtime hours are paid on top of salary.

As an organization dedicated to staying ahead of the cutting-edge advancements in technology, SilverCreek provides an extensive training program that gives employees free access to renowned and professional online learning platforms LinkedIn Learning and Pluralsight. With an annual 40 paid hours of training, you will be provided an opportunity to take your career to next level through the pursuit of further skill development.

SilverCreek jobs

Data Engineer