Data Engineer

📋 Description

• Develop and sustain high-volume, scalable data pipelines utilizing Apache Kafka and Apache Spark to address both real-time and batch data processing requirements.

• Architect, create, and enhance data ingestion, transformation, and integration workflows across various enterprise systems.

• Ensure the quality, consistency, and integrity of data across four (4) distinct data sources by implementing validation, cleansing, and reconciliation protocols.

• Create and uphold SQL-based data solutions, encompassing intricate queries, stored procedures, performance optimization, and data modeling.

• Collaborate with data analysts, product owners, and application teams to define data needs and ensure alignment with business objectives.

• Establish monitoring, logging, and alerting systems to guarantee the reliability and observability of data pipelines.

• Assist in data architecture design and contribute to best practices for scalable and secure data engineering solutions.

• Ensure adherence to federal data governance, security, and privacy regulations.

• Engage in Agile ceremonies and support the iterative development and delivery of data capabilities.

• Diagnose and resolve data pipeline challenges, ensuring minimal disruption to downstream systems and reporting.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, Data Science, or a related field (or equivalent experience).

• Over 3 years of experience in data engineering, data integration, or similar technical roles.

• Proficient hands-on experience with Apache Kafka for streaming data pipelines.

• Strong expertise in Apache Spark for large-scale data processing (both batch and/or streaming).

• Advanced SQL development skills, including intricate queries, performance tuning, and data transformation logic.

• Experience in integrating and managing data across various heterogeneous data sources.

• Background in the federal government or other highly regulated environments with security and compliance obligations.

• Solid understanding of data quality management, data validation, and data governance practices.

• Excellent problem-solving and analytical skills.

• Strong communication abilities, capable of conveying technical concepts to non-technical stakeholders.

• Keen attention to detail, particularly in ensuring data accuracy and consistency.

• Ability to operate independently in a fast-paced, mission-driven setting.

• Strong collaboration skills across cross-functional technical and business teams.

• US Citizenship or Permanent Residency is required.

• Must reside in the Continental US.

• Depending on the government agency, specific requirements may include a public trust background check or security clearance.

🏝️ Benefits

• Health care

• Dental

• Vision

• Life insurance

• 401(k)

• Paid time off including PTO, holidays, and any other paid leave mandated by law