Data Engineer

This is a fully remote position, open to applicants in District of Columbia, +2 more states.

📋 Description

• Design, construct, and maintain scalable end-to-end data pipelines utilizing Databricks, Spark, and related technologies.

• Develop effective data processing and transformation workflows for analytics and reporting purposes.

• Integrate APIs, databases, and cloud storage to create unified datasets.

• Work collaboratively with data science, analytics, and business unit teams to implement data solutions that align with business objectives.

• Establish validation, monitoring, and observability processes to guarantee data accuracy, completeness, and reliability.

• Contribute to initiatives in data governance, security, and automation.

• Utilize AWS S3, Glue, Lambda, and Redshift to develop and deploy cloud-native data solutions.

⛳️ Requirements

• Proven experience with cloud-based ETL services like AWS Glue, Google Cloud Dataflow, or Azure Data Factory.

• Familiarity with cloud data warehousing technologies such as Amazon Redshift, Google BigQuery, or Snowflake.

• Proficient in Python, SQL, Spark, and PySpark.

• Experience with data platforms including Databricks, Palantir, and Snowflake.

• Understanding of data orchestration and data quality processes.

• Must possess US citizenship.

• Applicants within the US must have work authorization that does not currently or in the future require visa sponsorship.

• Secret clearance or higher is a plus.

• Experience working with federal clients is a bonus.

• Familiarity with COTS and open-source data engineering tools such as ElasticSearch and NiFi is advantageous.

• Experience with Docker/Kubernetes, Hadoop/Spark, NiFi, and the ELK stack is beneficial.

• Familiarity with Agile/Scrum methodologies is a plus.

• Data engineering certification, such as Palantir Foundry Data Engineer, Azure Data Engineer Associate, Google Professional Data Engineer, IBM Certified Data Engineer, or similar, is a bonus.

🏝️ Benefits

• A collaborative and supportive community.

• Hands-on experience opportunities.

• Access to certifications.

• Industry training provided.

• Reasonable accommodations for disabilities or religious observances.

• Commitment to equal employment opportunities and affirmative action employer standards.