Position:- Senior Databricks Engineer
Location:- Raleigh, NC / Charlotte, NC (Onsite / Hybrid)
Job Description
Skillsets for data engineer:
- Azure:::Azure Resource Group, ADLS Gen 2 storage , Container, event hub, IOT hub, stream analytics, Power BI/ Tableau.
- Databricks: : Databricks lakehouse architecture (Bronze, Silver and Gold) using Delta Lake (ACID, schema enforcement, timetravel, schema evolution) , databricks workspace, pyspark, auto loader, python, SQL , spark cluster( all purpose , job) ,driver/worker, workflow/job/task, dataframe , dataset, azure blob file system secure( ABFSS)/DBFS, transaction log , delta table, delta live table, materialized view, metastore, databricks access connector, unity catalog, schema, DBFS Databrick CLI, databricks performance tuning.
- Data Modeling: Bigdata Modeling 3NF , Dimensional Modeling (Fact, Dim)
- Other: GitHub, CD/CI (Azure Dev/Ops)
Roles & Responsibility
- Design, develop, test, deploy, and maintain enterprise-level applications using the Snowflake platform
- Work with a variety of stakeholders to understand requirements and deliver solutions
- Take ownership of a project and see it through to completion
- Experience with the Snowflake platform
- Experience with enterprise-level applications
- Ability to work independently as well as part of a team
- Ability to take ownership of a project and see it through to completion
- Strong analytical and problem-solving skills
- Working experience on Snowflake cloud data warehouse
- Able to write advanced SQL queries
- ETL --IICS knowledge
- Have working experience on DATA Warehousing Projects
- Understanding of Dimensions, Facts and Datamart Terminologies.
- Must have minimum 5 years experience in Informatica PowerCenter using various transformations, ETL Framework and Design/Development.
- Develop ETL Solutions using IICS Data Integration/Application Integration handling large data volumes
- Must have working knowledge in handling various source/target systems including XML/Flat files/Databases using PowerCenter/Informatica MFT/ IICS File Ingestion.
- Experience in working Agile/Sprint projects and exposure to Azure DevOps is a plus
- Experience in change data capture, Data warehousing, SQL, SQL server and Unix shell scripting
- Prior DataStage and Azure cloud platform experience is an advantage.""