Reach me at anthonyb@marshalltechnologies.net
Location: Plano, TX (Local Hybrid Onsite - 2/3 days a week: Tuesday, Wednesday, Thursday)
Responsibilities
- Collaborate with the Data Platform Team to maintain and enhance existing data pipelines.
- Build and optimize data pipelines using Python, Spark, and AWS services (specifically AWS Glue and EMR).
- Assist in transitioning from EMR to AWS Glue for managing close to 200 tables.
- Provide support for upstream and downstream data processes.
- Address customer issues via Slack channels.
- Utilize SQL and Snowflake for data queries and database management.
- Optional: Create dashboards using QuickSight.
Must-Have Skills - AWS Data Engineering:
- Proficiency in AWS services, particularly AWS Glue and EMR.
- Programming Languages:
- Strong experience with Python and Spark.
- Data Pipeline Development:
- Ability to build, optimize, and troubleshoot data pipelines.
- Collaboration:
- Work effectively with cross-functional teams.
- Problem-Solving:
- Resolve data-related issues and contribute to continuous improvement.