Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, construct, and sustain scalable data pipelines, ETL/ELT workflows, and data models utilizing Python, Apache Spark (PySpark), Databricks, dbt, SQL (PostgreSQL), and AWS Glue.

• Develop and enhance AWS-native data platforms by leveraging AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), Lambda, Step Functions, Amazon S3, Redshift, RDS, DMS, and CloudWatch.

• Create high-performance ingestion, transformation, and orchestration workflows for both structured and semi-structured data utilizing Apache Iceberg, Parquet, ORC, and Avro.

• Design and optimize analytical data platforms using Amazon Athena, Trino, Hive, OpenSearch, and enterprise data catalog technologies.

• Integrate both enterprise and external data sources across relational and NoSQL platforms such as PostgreSQL, Oracle, Redshift, GraphDB, and additional NoSQL databases.

• Develop AI-enabled data solutions leveraging Amazon Bedrock, RAG pipelines, and vector search technologies including Amazon S3 Vector and OpenSearch vector indexes.

• Create cloud infrastructure using CloudFormation (Infrastructure as Code), GitHub, Harness, and enterprise CI/CD pipelines while utilizing SNS, SQS, and EventBridge for event-driven architectures.

• Enhance the reliability, scalability, performance, and maintainability of enterprise data platforms through monitoring, troubleshooting, automation, and continuous optimization.

• Support critical analytics and reporting solutions within large-scale AWS-based federal data environments, implementing solutions that adhere to FedRAMP and NIST 800-53 security controls.

• Lead modernization efforts in migrating legacy platforms, including IBM DataStage, Hadoop, RunDeck, and shell-based workflows to cloud-native AWS services.

• Mentor junior engineers through technical guidance, architecture discussions, and code reviews while advocating for engineering best practices.

• Collaborate with cross-functional teams in an Agile setting to define requirements, deliver high-quality data solutions, and effectively communicate technical concepts to both technical and non-technical stakeholders.

⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, or a related technical discipline.

• 4+ years of professional experience in data engineering or related fields.

• Strong hands-on experience with Databricks, Apache Spark (PySpark), Python, SQL (PostgreSQL), and dbt for large-scale data engineering, ETL/ELT development, data transformation, and data modeling.

• Proficient in designing, constructing, and maintaining AWS-native data platforms utilizing AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), AWS Lambda, AWS Step Functions, Amazon S3, Amazon Redshift, Amazon RDS, AWS DMS, and Amazon CloudWatch.

• Experience developing scalable data pipelines, workflow orchestration, and data integration solutions across enterprise environments.

• Familiarity with modern data lake technologies including Apache Iceberg and data formats such as Parquet, ORC, and Avro.

• Expertise in designing and optimizing solutions using relational and NoSQL databases including PostgreSQL, Redshift, Oracle, GraphDB, and other NoSQL platforms.

• Ability to build reliable, high-performance data platforms through performance tuning, system optimization, and enterprise-scale ETL/ELT architectures.

• Strong analytical and problem-solving abilities.

• Experience in agile, iterative software development environments.

• Excellent written and verbal communication skills, capable of explaining complex topics to diverse audiences.

🏝️ Benefits

• Remote Work (Hybrid roles will be specified in the job post).

• Competitive Compensation Package.

• Medical, Dental, and Vision coverage.

• Life Insurance, Short/Long Term Disability.

• Employee Assistance Program.

• 401(k) with 4% matching.

• Generous PTO vacation policy.

• Annual Continuing Education allowance.

• Annual Wellness Budget.

• Bonus Incentive Programs (Employee referrals and performance-based rewards).