Who We Are
Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to Customers.
Job Description
Job Title : Big Data Engineer
Job Type : W2
Experience : 5-15 Years
Location : Sunnyvale, California
We are looking for a Senior Data Engineer who will have a strong background in Python development and software design, with extensive experience working with data engineering tools and technologies. This role will be responsible for building and optimizing our data pipelines, integrating new data sources, and ensuring the reliability and efficiency of our data infrastructure.
Responsibilities
- Proven experience in Python development and software design.
- Hands-on experience with Dremio, Presto, or Trino.
- Experience with containerization and orchestration using Docker and Kubernetes.
- Experience with Shell scripting and CLI tools.
- Experience with big data technologies and frameworks.
- Strong knowledge of Snowflake or other data warehouse systems.
- Proficiency in data visualization tools such as Tableau.
- Proficiency with Git and version control systems and Rest APIs.
- Proficiency in data manipulation and analysis using Pandas.
- Advanced SQL skills and experience with MongoDB.
- Familiarity with object stores (S3) and data lake concepts.
- Design, develop, and maintain robust data pipelines using Python and SQL.
- Work with Dremio, Presto, and Trino to optimize data queries and performance.
- Implement and manage data warehouses such as Snowflake, ensuring efficient data storage and retrieval.
- Utilize Tableau for data visualization and reporting.
- Deploy and manage applications using Docker and Kubernetes.
- Leverage MongoDB for efficient data storage and retrieval.
- Implement and manage data in object stores like S3 and understand data lake concepts.
- Use Git for version control and collaboration on the codebase.
- Develop and maintain REST APIs to support data access and integration.
- Utilize Pandas for data manipulation and analysis.
- Write and maintain scripts using Shell and other CLI tools.
Qualification
- Bachelor's degree or equivalent combination of education and experience