Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop, maintain, test, and support moderately complex ETL/ELT pipelines, transformations, and data components utilizing Python, SQL, Spark/PySpark, and associated technologies.

• Take ownership of specific pipeline components from their implementation through testing, deployment, documentation, and production support.

• Diagnose pipeline failures, data quality concerns, and performance issues; determine root causes and apply effective solutions.

• Contribute to the technical design, estimation, and implementation plans at the component level.

• Implement practices related to data modeling, testing, code review, observability, security, source control, CI/CD, and operational assistance.

• Oversee pipeline performance and reliability; identify areas to enhance quality, efficiency, and maintainability.

• Gain a solid understanding of Claims and Consumer data structures, transformations, dependencies, and data quality standards.

• Work collaboratively with engineers, QA, analysts, Product, Support, and data consumers to clarify requirements, address issues, and deliver dependable data changes.

• Evaluate downstream effects of data and schema modifications and assist with vendor-provided data sources and customer inquiries.

• Engage in code reviews, pair programming, sprint activities, technical discussions, documentation, and knowledge sharing.

• Provide support to peers and junior engineers through knowledge exchange and constructive feedback on reviews.

• Utilize approved AI-assisted development tools responsibly to enhance productivity, documentation, test coverage, and code quality.

⛳️ Requirements

• Bachelor's degree in Computer Science, Engineering, Information Systems, or equivalent relevant experience.

• Minimum of 2 years in Data Engineering, Software Engineering, or a related field, building and supporting production data solutions.

• Proficient in Python and SQL, with practical experience in developing and maintaining production data pipelines.

• Familiarity with Apache Spark/PySpark or similar distributed data processing technologies.

• Experience with cloud platforms such as AWS, Azure, or GCP.

• Knowledge of modern data platforms like Databricks, Snowflake, Delta Lake, or equivalent technologies.

• Experience with workflow orchestration technologies, such as Airflow, Argo, or similar platforms.

• Working knowledge of relational databases, data modeling, testing, code review, observability, source control, CI/CD, and production support practices.

• Ability to take ownership of specific technical tasks, troubleshoot problems, collaborate effectively, and communicate progress, risks, and obstacles.

• Preferred: experience with healthcare data, including claims, pharmacy, eligibility, consumer, provider, or reference data.

• Preferred: experience in processing large datasets using lakehouse or medallion architecture patterns.

• Preferred: experience deploying, operating, or supporting Spark workloads in Kubernetes or similar containerized environments.

• Preferred: experience in supporting enterprise data products, analytics platforms, or critical data services in regulated environments.

• Preferred: experience onboarding or supporting vendor-provided datasets and assessing schema or downstream consumer impacts.

• Preferred: experience using AI-assisted engineering tools to enhance development productivity, testing, documentation, and code quality.

🏝️ Benefits

• Eligibility for company bonus or commission plan, depending on the position.

• Medical coverage.

• Dental coverage.

• Vision coverage.

• Unlimited paid time off.

• 401(k) plan with employer contribution.

• Access to industry-leading products.

• Flexible and dynamic work culture.

• Participation in the Definitive Cares community service group.

• Competitive benefits package.

• 401(k) matching.