Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop and sustain data pipelines that support NLQ, semantic search, and knowledge graph functionalities on the SEC's Enterprise Data Platform.

• Integrate new data sources and expand the semantic layer while ensuring existing live pipelines remain unaffected.

• Ingest, transform, and deliver XBRL and SEC financial filing datasets effectively.

• Collaborate with the Data Architect to support GraphRAG and knowledge graph pipelines.

• Utilize AWS data services, including Athena, DataZone, SageMaker, S3, and Lambda.

• Implement and manage vector databases and tools for semantic search.

• Oversee metadata management, data lineage, and validation of data quality throughout the pipeline ecosystem.

• Monitor the health of pipelines and address data integration challenges across various environments.

• Ensure all tasks comply with federal security and regulatory standards.

• Document pipeline processes and work in conjunction with the Data Architect.

• Carry out other related responsibilities as assigned.

⛳️ Requirements

• Practical experience in constructing enterprise data pipelines within regulated or federal settings.

• Strong proficiency in SQL and experience with PostgreSQL or comparable databases.

• Familiarity with ETL/ELT tools such as AWS Glue or Airflow.

• Proficient in Python for the development and automation of pipelines.

• Experience with AWS data services, including Athena, DataZone, S3, Glue, and Lambda.

• Knowledge of vector databases or tools for semantic search.

• Understanding of FISMA and NIST 800-53 compliance frameworks.

• Applicants must have authorization to work in the United States.

• Certain positions may require U.S. citizenship and the ability to secure and maintain federal background investigations and/or security clearances.

• Bachelor's Degree is required.

🏝️ Benefits

• Fully remote work environment.

• Tech & Education Stipend available.

• Comprehensive benefits package offered.

• Company matching 401(k) plan.

• Flexible PTO and paid holidays.