Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, construct, maintain, and troubleshoot dependable data pipelines throughout the software development lifecycle.

• Assist in the modernization of legacy U.S. Army financial systems and their migration to AWS cloud environments.

• Create Python-based workflows to ingest, transform, validate, and deliver both structured and semi-structured data.

• Develop and support cloud-native data solutions utilizing AWS Glue, Lambda, S3, and CloudWatch.

• Integrate data from various systems and sources, including APIs and diverse file formats.

• Handle large datasets while enhancing pipeline efficiency and scalability.

• Engage in technical design discussions related to data integration, storage, performance, and scalability.

• Implement automated testing, logging, monitoring, and error management.

• Produce clean, reusable, and maintainable code adhering to software engineering and DataOps best practices.

• Collaborate with engineers, architects, and stakeholders to convert requirements into functional software.

• Support secure data management, access controls, data governance, and compliance with DoW standards.

• Participate in code reviews and Agile development processes.

• Document data transformations, pipeline procedures, and technical decisions made.

• Enhance workflows, minimize manual processes, and fortify the Army’s data ecosystem.

⛳️ Requirements

• Must be remote within the continental United States, with East Coast working hours and some flexibility.

• Occasional travel may be necessary (3-4 times annually).

• Must be a U.S. Citizen with the capability to obtain a DoW Secret clearance.

• Minimum of 2 years of relevant experience in data engineering, software development, data integration, or a closely related field.

• Proven experience in developing Python-based data transformations, scripts, or workflow automation.

• Practical experience in building, maintaining, or troubleshooting data pipelines and ETL/ELT processes.

• Familiarity with data consumption from APIs and working with X12, JSON, flat files, or fixed-width files.

• Experience with AWS cloud services, specifically AWS Glue, AWS Lambda, Amazon S3, and CloudWatch Logs.

• Acquainted with large datasets and issues related to data processing or performance.

• Understanding of data structures, data modeling concepts, and the organization and transformation of data.

• Experience in writing and maintaining automated tests using Pytest or similar frameworks.

• Knowledge of data validation, error management, logging, and debugging practices.

• Understanding of data security, access controls, and principles of data quality.

• Familiarity with using GitLab for version control and collaborative development.

• Comfortable working via the terminal or command-line interface.

• Experience utilizing Jira and Confluence in an Agile development setting.

• Ability to interpret ambiguous requirements, file specifications, and JSON structures.

• Strong problem-solving abilities and capacity to collaborate with both technical and non-technical stakeholders.

• Must currently hold or be able to obtain an AWS Cloud Practitioner certification (or higher) within 30 days of hire.

• Must currently hold or be able to obtain a CompTIA Security+ certification (or higher) within 30 days of hire.

• Bachelor's Degree in Computer Science or a related engineering field is preferred.

🏝️ Benefits

• Health, dental, and vision insurance plans.

• Flexible Spending Account (FSA).

• 401k plan with employer matching.

• Company-sponsored life insurance, as well as short and long-term disability coverage.

• Flexible paid time off plus all federal holidays off.

• Opportunities for professional development (training, certifications, conferences).

• Paid cloud developer accounts.

• Referral bonus.

• HQ office perks (parking/metro reimbursement, nitro coffee & lunches).

• Annual social events (540 week, hackathon, charity golf tournament, etc.).

• Access to 540’s Washington Capitals & Nationals tickets.