Position: Data Engineer
Location: Houston, TX (Onsite)
Duration: Contract
Experience: 10+ Years
Job Description:
Skills | Must-Haves:
- .NET and C# experience
- Spark experience
- experience with large data sets
- PowerBI experience
- experience using CI/CD DevOps pipelines
Job Description Skills/Domain: Microsoft Azure, Synapse, Spark, Python, Angular, C#, .NET, DevOps, Azure Function, Microservice/API Development, Power BI
Roles and Responsibilities:
- Promotes and adheres to Client's architectural standards and guidelines and works closely with the Enterprise Architects to develop target architectures and technology standards
- Provides explicit architecture consulting to project teams, across corporate functions, understands and frames the business problem to be solved. Identify available problem-solving approaches and methods
- As part of an Agile delivery approach, works with the product owner to write epics and stories to be delivered and participate in Agile SAFe delivery via standups, sprint planning and PI planning.
- Develop solutions by utilizing Microsoft Synapse, .Net and Spark to retrieve data from on-premises systems, manipulate large datasets, and perform intricate data wrangling including logging, error handling and rerun capabilities
- Implement role based and OBO authentication to enhance security management for the business.
- Develop CI/CD DevOps pipelines using ansible roles to enable continuous deployment and regression tests.
- Develop Data Manager Application using Angular and NgRx.
- Design synapse data warehouse using data modelling guidelines to enable optimal performance for Power BI reporting and develop Power BI reports
- Develop Rest APIs following REST guidelines for interacting with source systems and ingesting data into Azure Data Manager for Energy.