Overview
The Data Engineer- Ex Capitalone Consultant position is a crucial role within our organization, responsible for designing and implementing scalable and reliable data solutions. This role plays a vital part in driving data-driven decision-making and ensuring the integrity of our data infrastructure.
Key Responsibilities
- Collaborate with cross-functional teams to understand data requirements and implement effective data solutions.
- Design, develop, and maintain scalable data pipelines and ETL processes.
- Optimize data models and databases for performance and efficiency.
- Ensure data quality and integrity through validation, testing, and documentation.
- Implement and maintain data security and privacy measures.
- Conduct performance tuning, troubleshooting, and optimization of data systems.
- Automate manual processes for data ingestion, processing, and analysis.
- Work with large datasets and implement big data technologies as needed.
- Collaborate with data scientists and analysts to support their data needs.
- Stay current with industry trends and technologies in data engineering.
Required Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Data Engineer, preferably with experience at Capital One.
- Proficiency in Python for data processing and automation.
- Strong SQL skills for working with relational databases.
- Experience with data modeling and building data warehouses.
- Expertise in ETL tools and processes.
- Knowledge of big data technologies such as Hadoop, Spark, or Kafka.
- Familiarity with cloud platforms like AWS, Azure, or GCP.
- Excellent problem-solving and analytical abilities.
- Strong communication and collaboration skills to work effectively in a team environment.
Skills: aws,cloud platforms,data modeling,big data technologies,etl,java,sql,python