We are seeking a highly skilled and experienced Solution Architect to join our team. The ideal candidate will possess a strong background in designing and implementing data integration solutions, with a particular focus on Azure ADF Pipelines, APIs, and integration architecture. This role requires expertise in data management, data architecture, and large data store technologies, including Data Lakes, Data Warehouses, and Data Hubs. The successful candidate will have recent experience developing reference data architecture, data modeling, data profiling, and data quality analysis, and will be proficient in XML, JSON, and Python.
Key Responsibilities:
- Architect and Design Solutions: Develop end-to-end data and integration architecture solutions that support business goals and requirements. Design and implement robust, scalable, and secure data integration solutions using Azure Data Factory (ADF) Pipelines, APIs, Databricks, and other relevant technologies.
- Integration Design and Development: Lead the design and development of API-based integrations and data pipelines to connect various systems and data sources. Ensure seamless data flow between disparate systems, applications, and platforms.
- Data Management and Governance: Establish and implement data management practices, including data governance, data quality standards, data profiling, and metadata management. Develop and maintain data architecture artifacts, including data models (conceptual, logical, and physical).
- Reference Data Architecture: Develop and maintain reference data architecture and frameworks. Create data models, including conceptual, logical, and physical models, to represent data entities, relationships, and business rules.
- Large Data Store Technologies: Utilize knowledge of large data store technologies, such as Data Lakes, Data Warehouses, and Data Hubs, to design and implement storage solutions that meet performance, scalability, and data retention requirements.
- Collaboration and Communication: Collaborate with cross-functional teams, including data engineers, data scientists, developers, and business stakeholders, to define requirements, develop solutions, and ensure successful delivery of data architecture projects.
- Technical Leadership: Provide technical leadership and mentorship to team members and promote best practices in data architecture, data integration, and data management.
- Data Quality Analysis: Perform data profiling and data quality analysis to identify data issues and recommend solutions for data cleansing, enrichment, and standardization.
Required Skills and Qualifications:
- Technical Expertise:
- Proficiency with Azure Data Factory (ADF) Pipelines for designing and managing complex data integration workflows.
- Strong experience with API development and integration design using RESTful APIs, SOAP, and other protocols.
- Expertise in Databricks for data processing, transformation, and analysis.
- Recent experience developing reference data architecture, data modeling (conceptual, logical, and physical), data profiling, and data quality analysis.
- In-depth knowledge of large data store technologies such as Data Lakes, Data Warehouses, and Data Hubs.
- Familiarity with XML, JSON, and Python for data integration and transformation tasks.
- Strong understanding of API concepts and integration architecture.
- Experience and Knowledge:
- Proven experience in data architecture, data management, and data integration in a complex, multi-platform environment.
- Solid understanding of data governance principles, including data quality, data lineage, and metadata management.
- Strong analytical and problem-solving skills with the ability to design solutions to complex data challenges.
- Soft Skills:
- Excellent communication skills, both written and verbal, with the ability to present complex technical concepts to non-technical stakeholders.
- Strong leadership skills and experience in mentoring and guiding technical teams.
- Ability to work effectively in a collaborative, fast-paced environment and manage multiple priorities.
Preferred Qualifications:
- Experience with other integrating SaaS platforms with cloud based (Azure) and on-prim SQL servers a plus.
- Certification in data architecture, Azure, or other relevant technologies.
- Familiarity with DevOps practices, SDLC, Change Control, and subsequent tools.