Position: Data Engineer
Location: Jersey City, NJ (Onsite)
Duration: 12+ months
Job Description:
Must Have Skills: Java, Spark, Data Reconciliation, Python, Drools
Key Responsibilities:
- Data Reconciliation Development: Design, develop, and test Java-based applications to automate data reconciliation processes across various financial data sources, including relational databases, NoSQL databases, batch files, and real-time data streams.
- Implement efficient data transformation, matching algorithms (deterministic and heuristic) with Java and relevant frameworks.
- Develop robust error handling and exception management mechanisms to ensure data integrity and system resilience.
- Data Analysis and Matching: Collaborate with business analysts and data architects to understand data requirements and matching criteria.
- Analyze and interpret data structures, formats, and relationships to implement effective data matching algorithms in Java.
- Rules Engine Integration: Integrate Java applications with rules engines (e.g., Drools) to implement and execute complex data matching rules.
- Develop Java code to interact with the rules engine, manage rule execution, and handle rule-based decision-making.
- Problem Solving and Gap Analysis: Collaborate with cross-functional teams to identify and analyze data gaps and inconsistencies between systems.
- Design and develop Java solutions to address data integration challenges and ensure data quality.
- Contribute to the development of data governance and quality frameworks within the organization.
- Qualifications and Skills:
- Bachelor's degree in Computer Science or a related field.
- 5+ years of hands-on experience in Java development, preferably with exposure to data-intensive applications.
- Strong understanding of data reconciliation principles, techniques, and best practices.
- Proficiency in Java, Spring Data, and related technologies for data access and integration (e.g., Spring Data, Hibernate, JDBC).
- Experience with rules engine integration and development (e.g., Drools).
- Strong analytical and problem-solving skills, with the ability to translate business requirements into technical solutions.
- Excellent communication and collaboration skills to work effectively with business analysts, data architects, and other team members.
- Familiarity with data streaming platforms (e.g., Kafka, Kinesis) and data technologies (e.g., Hadoop, Spark) is a plus.