Data Engineer, AI

This is a fully remote position, open to applicants in Arizona, +11 more states.

📋 Description

• Design, construct, and enhance scalable ETL/ELT pipelines utilizing Apache Spark and Delta Lake across AWS, Azure, and GCP.

• Develop scalable Lakehouse architectures for both structured and unstructured data.

• Automate workflows for data ingestion, storage, and feature engineering to support analytics and real-time AI operations.

• Optimize Spark jobs for minimal latency, maximum throughput, and cost efficiency in cloud environments.

• Collaborate with ML engineers and data scientists to operationalize AI models in Databricks using MLflow.

• Establish CI/CD pipelines and Infrastructure-as-Code through Terraform and Databricks CLI.

• Uphold enterprise data security, access controls, and compliance standards within Unity Catalog and cloud environments.

• Work alongside client technology leaders, product managers, and engineering teams to translate AI data requirements into actionable business results.

• Create technical documentation and oversee solution transitions to client engineering teams.

⛳️ Requirements

• Minimum of 3 years in cloud-native data engineering.

• At least 2 years of dedicated, practical experience with Databricks, Apache Spark, Delta Lake, and MLflow.

• Strong expertise in Python, SQL, Spark frameworks, CI/CD, and Infrastructure-as-Code (Terraform or Databricks CLI).

• Active Databricks Certified Data Engineer (Associate or Professional) or Databricks ML Professional certification is highly preferred.

• Demonstrated success in fast-paced Agile/DevOps consulting settings.

• Experience in delivering cloud-agnostic architectures.

• Excellent client communication skills.

• U.S. work authorization is mandatory for newly hired direct W-2 employees through E-Verify.

🏝️ Benefits

• Paid Time Off.

• Remote-First Working Environment.

• Comprehensive Health Coverage – Medical, Dental, Vision.

• 401(k) Retirement Plan.

• Annual Learning & Development Stipend.

• Peer Mentorship & Coaching.

• Professional Growth Opportunities.

• Company Outings & Volunteer Opportunities.

• Collaborative, Innovative Culture.