Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and enhance data pipelines utilizing Python, SQL, and Databricks.

• Create and maintain Databricks notebooks to facilitate data ingestion, transformation, and delivery.

• Architect and manage data tables, schemas, and Delta Lake structures tailored for analytics and data science needs.

• Collaborate closely with data scientists to comprehend data requirements and adapt pipelines, tables, and data structures as necessary.

• Construct advanced SQL queries employing joins, aggregations, window functions, and other sophisticated techniques.

• Integrate and transform data from various structured and semi-structured data sources.

• Ensure data quality through validation, troubleshooting, and resolution of data-related issues.

• Enhance the performance of data pipelines, SQL queries, and data structures.

• Document data sources, transformations, data flows, schemas, and configurations of pipelines.

• Work alongside data scientists, engineers, analysts, and business stakeholders to deliver data solutions that meet mission objectives.

⛳️ Requirements

• Over 4 years of experience in a Data Engineer role or a similar data-centric position.

• Expertise in SQL, including complex joins, aggregations, window functions, and transformations.

• Proficient in Python for data engineering and transformation tasks.

• Practical experience with Databricks, encompassing the development and upkeep of notebooks and data pipelines.

• Background in developing and managing ETL/ELT pipelines and data integration processes.

• Familiarity with Delta Lake technology.

• Knowledge of database design, data modeling, schema development, and data-centric documentation.

• Experience in integrating and transforming data from diverse data source types.

• Proven ability to troubleshoot data challenges and optimize queries and pipelines.

• Strong analytical and problem-solving capabilities.

• Excellent verbal and written communication skills, with the capacity to collaborate effectively with both technical and business stakeholders.

• Bachelor's degree in Computer Science, Mathematics, Data Science, Engineering, or a related field.

• Must possess an active Secret clearance with eligibility for Top Secret.

• U.S. Citizenship is mandatory.

• Must successfully complete the required background screening and drug testing.

• Preferred: Experience in supporting data scientists and advanced analytics or machine learning applications.

• Preferred: Knowledge of Department of Defense, War, or U.S. Air Force data sources and systems.

• Preferred: Experience in logistics, supply chain, maintenance, or operational data.

• Preferred: CompTIA Security+ certification.

• Preferred: Top Secret clearance.

🏝️ Benefits

• Competitive salary aligned with market standards.

• Generous paid time off (PTO) package.

• Comprehensive medical insurance coverage.

• Dental insurance plans.

• Vision insurance options.

• Life insurance benefits.

• 401K retirement plan.

• Short-term disability insurance.

• Long-term disability insurance.

• A fun and engaging organizational culture.

• Training opportunities to stay updated with the latest technologies.