Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Design, develop, and maintain a scalable, reliable, and cost-effective data platform architecture and foundational capabilities, primarily utilizing Databricks.

• Construct and enhance data pipelines, integrations, and reusable tools.

• Enhance data processing, developer productivity, and delivery across engineering and analytics teams.

• Monitor and optimize the performance, reliability, and resource utilization of the data platform.

• Identify opportunities to boost efficiency and lower operational costs.

• Collaborate with security and engineering teams to establish security standards, data governance, and meet regulatory and compliance requirements.

• Partner with cross-functional stakeholders to assess platform needs and prioritize foundational improvements.

• Contribute to technical planning and architectural decisions.

• Support continuous improvement initiatives and perform additional duties as assigned.

⛳️ Requirements

• Bachelor's degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience.

• Over 5 years of experience in software engineering, data engineering, or related technical disciplines.

• More than 5 years of hands-on experience with Databricks, focusing on designing, developing, administering, and maintaining scalable cloud-based data platforms.

• Proficient in building data pipelines, integrations, automation, and reusable tools.

• Experience in configuring, monitoring, troubleshooting, and optimizing data platform infrastructure.

• Familiarity with cloud infrastructure and services, preferably AWS; Azure or Google Cloud experience is also valued.

• Proficiency in Scala, Python, SQL, Apache Spark, or similar data engineering technologies.

• Experience with infrastructure-as-code and CI/CD practices.

• Background in supporting production environments and resolving complex technical issues.

• Experience collaborating with cross-functional engineering teams.

• Preferred familiarity with streaming data platforms and event-driven architectures like Apache Kafka or Confluent.

• Strong understanding of modern data platform architecture, distributed data processing, and cloud-based infrastructure.

• Knowledge of Databricks workspace administration, compute configuration, workflows, access management, monitoring, and cost optimization.

• Strong programming and scripting skills, particularly in Python and SQL.

• Understanding of Apache Spark, Delta Lake, and data processing and storage optimization.

• Knowledge of cloud infrastructure, security, networking, and resource management, especially AWS.

• Familiarity with infrastructure-as-code, CI/CD, and automated deployment practices.

• Knowledge of data governance, security best practices, access controls, and principles of platform reliability.

• Must be legally authorized to work in the United States.

• Role must be performed from within the continental U.S., excluding Montana, Alaska, and Hawaii.

• Ability to meet minimal travel requirements.

• Capability to occasionally lift/move up to 25 pounds.

🏝️ Benefits

• Bonus

• Medical insurance

• Dental insurance

• Vision insurance

• 401(k) with company match

• Flexible PTO

• 12 paid holidays

• Paid sick leave

• Paid parental and family leave

• Lifestyle spending account

• Tuition reimbursement

• Cell phone stipend

• Minimal travel, generally 12 days or less per year

• Approximately two team get-togethers per year