Lead Data Engineer

This is a fully remote position, open to applicants in Minnesota.

📋 Description

• Spearhead the design and implementation of scalable and secure data pipelines that cater to AI/ML workloads.

• Oversee complete data engineering processes including ingestion, transformation, storage, quality assurance, and monitoring.

• Collaborate closely with data scientists and ML engineers on model features, training pipelines, and deployment strategies.

• Promote best practices in data modeling, orchestration, versioning, and performance enhancement.

• Mentor and support junior engineers in their professional development.

• Ensure compliance with data governance, lineage, and regulatory standards across all platforms.

• Facilitate both real-time and batch processing frameworks in production settings.

⛳️ Requirements

• A Bachelor's degree or higher in computer science from an accredited institution.

• A minimum of seven (7) years of experience in data engineering.

• Strong proficiency in Python, SQL, and distributed data systems (such as Spark, Databricks, Synapse).

• Experience in constructing data pipelines that are ready for AI/ML applications.

• Practical experience with cloud platforms, preferably Azure.

• A solid understanding of machine learning concepts.

• Proven expertise in workflow orchestration tools (e.g., Airflow, Data Factory, Synapse Pipelines).

• Familiarity with streaming platforms (e.g., Kafka, Event Hub).

• A background in DevOps, CI/CD, or infrastructure as code methodologies.

🏝️ Benefits

• Medical, Dental & Vision coverage.

• Health Savings Accounts.

• Flexible Spending Accounts for Health Care & Dependent Care.

• Disability benefits.

• Life insurance.

• Voluntary benefits.

• Paid absence policies.

• Retirement benefits.