MBO Partners is a deep jobs platform that connects and enables independent professionals and microbusiness owners to do business safely and effectively with enterprise organizations. Its unmatched experience and industry leadership enable it to operate on the forefront of the independent economy and consistently advance the next way of working.
Duration: Through 2026
Location: fully remote and maybe travel for quarterly calls in DC Metro. OR TX office
Hours: Work our core hours – 10-6pm ET
Clearance: Not required but will need to process for a Public Trust/ICE Suitability clearance
As a data engineer in our Civil team, you’ll use your extensive technical expertise to design data architecture solutions for mission-critical investigative data. You’ll resolve routine data architecture-related issues in collaboration with business analysts and technology teams by developing Business Process Models and Domain Driven Design.
In this role, you’ll grow your technical expertise, apply best practices and use tools like Spark, Kafka, EKS, and others. With your drive to establish processes and lead technological innovation, you’ll make a lasting impact on the civil aviation industry.
Top Skills:
- Spark Expertise
- Datawarehouse expertise/Data Lake expertise and Data integration
- Databricks
- Java/Scala
Job Description: Sr. Big Data Engineer
Position Overview: We are seeking a highly skilled Sr. Big Data Engineer to join our Civil Team. As a part of our team you will need expertise in data integrations within a large enterprise environment. The ideal candidate will have extensive experience in designing and building scalable data architectures, developing data pipelines, and working with big data technologies. The focal point of this role is helping maintain the Data platform and assist with the integration of new data and applications. This role requires a deep understanding of data integration, data governance, and distributed processing systems. The Sr. Big Data Engineer will play a critical role in ensuring the accessibility, integrity, and availability of data across the organization.
Must Have:
- Sr. Big Data Engineer with expertise in data integrations within a large enterprise environment.
- Minimum of 5 years of experience in building conceptual and logical data models, including defining and maintaining logical and canonical data elements and definitions.
- Extensive experience with Databricks.
- 5+ years of experience in designing information and data architecture products.
- 3+ years of experience in data strategy or data governance.
- 3+ years of experience with systems engineering or enterprise architecture methodologies.
- 3+ years of experience with Spark, MapReduce, or similar distributed processing systems.
- 3+ years of experience with Data Warehouse usage and design.
- 3+ years of experience processing data using streaming and batch methods.
- 3+ years of experience using Python, SQL, Scala, and/or Java.
- Knowledge of data, information, and message exchange structures and standards.
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
- Bachelor's degree in a relevant field.
Nice to Have:
- 3+ years of experience in data analytics.
- Experience designing data flows that leverage a medallion architecture.
- Experience using Unified Modeling Language (UML) or XML.
- Experience with Domain Driven Design (DDD).
- Experience with Kafka and NiFi.
- Experience with Kubernetes and Docker.
- Experience using AWS or other cloud platforms.
- Knowledge of Microservices architecture.
- Knowledge of database schema design.
- Master's degree in a relevant field.
Ideal Resume:
- Extensive experience in data integration within large enterprise environments.
- Strong expertise in Spark and Microservices.
- Big Data Architecture experience.
- Previous experience working with DOD or DHS.
- Experience in consulting roles.
- Background in law enforcement data systems.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes to support data ingestion, transformation, and storage.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.
- Build and optimize data architectures and workflows to ensure data accessibility, integrity, and availability.
- Manage and maintain large, complex datasets, ensuring data quality and consistency.
- Implement data governance practices, including data lineage, metadata management, and data cataloging.
- Monitor and troubleshoot data pipelines and infrastructure to ensure timely and accurate data delivery.
- Utilize cloud platforms (e.g., AWS, Azure, GCP) to manage and deploy data infrastructure and services.
- Develop and maintain data models, schemas, and databases to support business needs.
- Work with big data technologies (e.g., Hadoop, Spark) to process and analyze large volumes of data.
- Implement security best practices to protect sensitive data and ensure compliance with regulatory requirements.
- Automate repetitive data tasks and processes to improve efficiency and reduce manual effort.
Soft Skills:
- Strong problem-solving and analytical skills.
- Ability to work independently and collaboratively within a team.
- Excellent communication and interpersonal skills.
- Attention to detail with a commitment to producing high-quality work.
- Ability to manage multiple tasks and projects simultaneously.
Eligibility Requirements:
- Legal authorization to work in the U.S. is required.
- As a contractor, including remote contractors, you may be required to complete a background check.
- Please Note: Due to the nature of this role, only U.S. citizens are eligible. This project is not open to candidates requiring temporary work visa sponsorship.
- No third parties please.