Data Engineer – AI, Spark, Databricks, Healthcare

This is a fully remote position, open to applicants in United States.

📋 Description

• Develop, maintain, and execute intermediate to advanced Spark scripts for managing data, validating data, and integrating data.

• Develop, maintain, and execute basic to intermediate SQL scripts for data management and validation.

• Enhance query performance to boost the efficiency of daily operations.

• Conduct data analysis and pinpoint any potential issues.

• Collaborate with teams such as Engineering, DBA, Cloud operations, etc., to troubleshoot and resolve environmental or network challenges affecting your work.

• Provide support after hours or on weekends as necessary.

• Create and manage data pipelines as required.

• Validate task outcomes to ensure compliance with all requirements.

• Follow all industry and organizational compliance standards to uphold data integrity.

• Track individual productivity diligently.

• Complete task assignments through the departmental ticketing system within the designated deadlines.

• Meet organizational and personal objectives as outlined in performance evaluations and goal-setting sessions.

• Accomplish all special projects and other assigned responsibilities.

⛳️ Requirements

• Bachelor's degree in Computer Science, Information Technology, or equivalent work experience.

• Over 3 years of practical experience with big data technologies (Spark, S3, Kafka, Ray, Hadoop, etc.).

• More than 2 years of experience with big data/cloud technologies (Databricks, AWS, Azure, Hadoop, Spark, Snowflake, etc.).

• Over 3 years of expertise in cloud platforms (AWS, Azure, GCP, OCI, etc.).

• More than 3 years of experience with RDBMS (Oracle, MS SQL, Vertica, etc.) and proficiency in SQL, PL/SQL, or other data integration/ETL tools.

• Any Databricks/AWS certifications are highly valued.

• Knowledge of data pipeline orchestration tools (e.g., Airflow, Databricks Workflows).

• Over 3 years of experience in data analysis.

• Preferably experienced in the Healthcare sector, including enrollment, medical claims, and/or pharmacy claims.

• Proficient in Microsoft Office Suite applications: PowerPoint, Word, Excel, and Outlook.

• Flexible work schedule is required.

• Experience with project management tools such as JIRA.

• Familiarity with Databricks and/or Snowflake environments is advantageous.

🏝️ Benefits

• Medical, dental, vision, disability, and life insurance coverage.

• 401(k) savings plans.

• Paid family leave.

• 9 paid holidays each year.

• 17-27 days of Paid Time Off (PTO) annually, depending on specific level and length of service with Cotiviti.