Senior Lead Data Engineer

This is a fully remote position, open to applicants in Illinois, +2 more states.

📋 Description

• Oversee the design, development, and enhancement of scalable enterprise data platforms that support global procurement analytics and drive business transformation.

• Formulate and assist in implementing a multi-year data engineering strategy emphasizing scalability, reliability, automation, reduction of technical debt, and maintainability.

• Architect, construct, and refine Azure-based data solutions utilizing Databricks, Delta Lake, Azure Data Factory, Azure DevOps, CI/CD pipelines, and automation of infrastructure.

• Integrate and align data from various ERP systems into unified enterprise data models.

• Collaborate with analysts, architects, engineers, and business stakeholders to convert business requirements into reusable data products and engineering solutions.

• Set engineering standards, carry out architecture reviews, enhance documentation, and mentor engineering staff.

• Apply AI-driven methods to speed up development processes, enhance data quality, automate documentation, support testing, and boost analyst productivity.

• Assess and suggest tools, frameworks, patterns, and platform investments.

• Provide support for internal data products, APIs, and user-facing applications aimed at procurement insights.

• Communicate technical recommendations, roadmap priorities, trade-offs, and business impacts to stakeholders and leadership.

⛳️ Requirements

• Bachelor’s degree in Computer Science, Data Engineering, Data Science, Information Systems, Mathematics, Statistics, or a related technical discipline.

• Over 6 years of experience in data engineering, analytics engineering, software engineering, or development of enterprise data platforms.

• Demonstrated expertise in architecting, constructing, and modernizing scalable cloud-based data platforms within an enterprise setting.

• Practical experience with Azure-based data engineering technologies such as Databricks, Delta Lake, Azure Data Factory, Azure DevOps, CI/CD pipelines, and infrastructure automation.

• Strong command of SQL and Python, with experience in using Spark or similar distributed data processing frameworks.

• Experience in designing reliable ETL/ELT pipelines, data warehouse or Lakehouse architectures, data models, and performance-optimized analytical solutions.

• Familiarity with integrating data from multiple ERP or enterprise source systems and reconciling inconsistent master, supplier, purchasing, or transactional data into standardized data models.

• Knowledge of API integrations, REST services, authentication, security, data governance, and production support practices.

• Proven ability to establish engineering standards, conduct architecture reviews, enhance documentation, mentor engineers, and influence technical direction without having direct authority.

• Capacity to collaborate with analysts, engineers, architects, and business stakeholders to translate complex business needs into scalable, maintainable engineering solutions.

🏝️ Benefits

• Flexibility for remote work.

• 10% travel allowance/requirement.