Data Engineer IV

This is a fully remote position, open to applicants in Illinois.

📋 Description

• Design and manage pipelines, data assets, semantic models, search and retrieval layers, and platform patterns that ensure trusted data is accessible for analytics, AI agents, and operational decision-making.

• Create and execute highly efficient, reusable, and scalable data processing systems and pipelines utilizing Kubernetes, Databricks, Snowflake, and other associated cloud services.

• Establish platform patterns for AI-ready data products, encompassing governed data assets, semantic models, vector and search functionalities, retrieval patterns, and integration points for Snowflake, Databricks, and other AI-enabled workflows.

• Assess, implement, and operationalize Snowflake AI initiatives including Cortex Analyst, Cortex Search, Cortex Agents, Cortex Code, Cortex Sense, and other related AI/LLM-enabled data engineering components.

• Assist Databricks requests from data engineering and analytics teams, covering job orchestration, reusable pipeline patterns, performance tuning, production troubleshooting, workspace standards, and best-practice recommendations.

• Utilize and assess AI-assisted engineering tools such as Claude, Claude CLI, GitHub Copilot, and other related tooling.

• Implement solutions for data reliability, accuracy, observability, and operational resilience, employing test-driven development.

• Construct data models and mappings, develop new data assets, and conduct exploratory data analysis on existing products and datasets.

• Train data engineering teams on new data patterns, platform capabilities, AI-enablement strategies, and tools.

• Investigate emerging technologies and multi-agent engineering environments to meet Grainger's requirements.

• Operate within an Agile delivery / Kanban framework to iteratively deliver product increments.

• Collaborate with product and business partners to outline roadmaps, communication strategies, architectural designs, adoption plans, and support models.

• Mentor junior team members and elevate the technical standards for data engineering practices throughout the organization.

⛳️ Requirements

• Over 3 years of experience in batch and streaming data engineering utilizing Spark, Python, Scala, Snowflake, or Databricks for analytics, data engineering, or machine learning workloads.

• More than 3 years of experience orchestrating and implementing production pipelines using Databricks Workflows / Lakeflow Jobs, Apache Airflow, Luigi, or similar orchestration platforms.

• Practical experience with the Databricks platform in a central data platform or data engineering enablement role, including notebooks/repos, Delta Lake, Unity Catalog, Databricks Runtime, job/serverless/cluster compute, workflow monitoring, cost and performance optimization, and production support of user requests.

• Familiarity or strong working knowledge of Snowflake AI and Cortex functionalities such as Cortex Analyst, Cortex Search, Cortex Agents, Cortex Code / Snowflake CoCo, Cortex Sense, AISQL / AI functions, semantic models, and AI-enabled search, retrieval, or agent patterns.

• Comfort in utilizing AI-assisted developer and agent tools like Claude, Claude CLI, GitHub Copilot, or similar tools.

• Understanding of emerging multi-agent system patterns, which include context management, tool utilization, orchestration, guardrails, testing, observability, and secure enterprise adoption.

• At least 5 years of experience preparing structured and unstructured data for data science, analytics, machine learning, or AI-enabled applications.

• Over 4 years of experience with containerization and orchestration technologies such as Docker and Kubernetes.

• Preferred experience with shell scripting in Bash, Unix, or Windows shell.

• Exposure to vector databases, graph databases, relational databases, and NoSQL stores.

• Experience implementing machine learning or AI in data pipelines to discover, classify, enrich, standardize, and clean data.

• Proficiency in executing CI/CD with automated testing using Jenkins, GitHub Actions, GitLab CI/CD, or similar tools.

• Familiarity with AWS or other cloud services, such as AWS Glue, Athena, Lambda, S3, and DynamoDB.

• Proven experience in implementing the data management lifecycle utilizing data quality functions such as standardization, transformation, rationalization, linking, matching, monitoring, and stewardship.

• Please note that this position is not eligible for visa sponsorship.

🏝️ Benefits

• Medical, dental, vision, and life insurance plans available starting on the first day.

• Six complimentary sessions each year with a licensed therapist.

• Eighteen paid time off (PTO) days annually for full-time employees.

• Six company holidays each year.

• A 6% company contribution to a 401(k) Retirement Savings Plan each pay period, with no required employee contribution.

• Employee discounts.

• Tuition reimbursement.

• Student loan refinancing options.

• Complimentary access to financial counseling, education, and tools.

• Maternity support programs.

• Nursing benefits.

• Up to fourteen weeks of paid leave for birth parents.

• Up to four weeks of paid leave for non-birth parents.

• Incentive target of up to 15% based on individual and company performance objectives.