Director, Software Engineering

This is a fully remote position, open to applicants in Virginia.

📋 Description

• Collaborate with the core Databricks team to strategize and implement the transition of current data pipelines to the designated enterprise data platform.

• Establish repeatable engineering methodologies for data ingestion, transformation, testing, deployment, and monitoring across integrated datasets.

• Spearhead the adoption of AI-driven development tools such as Claude, GitHub CoPilot, and Large Language Models (LLMs).

• Create context engineering practices and develop reusable context packs.

• Assist offshore teams in identifying and operationalizing asset-agnostic onboarding strategies for data assets, domains, and source systems.

• Develop reusable ingestion and processing frameworks for both batch and streaming applications.

• Integrate platform pipelines and datasets with the enterprise data mastering system.

• Set up data quality controls, reconciliation procedures, and ensure metadata alignment.

• Promote consistent business definitions, reusable data concepts, and governed consumption patterns.

• Offer senior technical leadership for offshore data platform engineering initiatives.

• Influence architectural choices and participate in design and code evaluations for intricate data platform projects.

• Work in collaboration with global teams focused on platform, architecture, governance, security, data mastering, Databricks-aligned partners, offshore teams, and upstream/downstream stakeholders.

⛳️ Requirements

• Over 10 years of experience in data engineering, data platforms, cloud data architecture, software engineering, or similar engineering fields.

• Proven experience as a senior Individual Contributor, technical lead, principal engineer, staff engineer, distinguished engineer, architect, or similar role in complex data platform settings.

• Strong practical experience in modern data engineering and pipeline development.

• Familiarity with batch and/or streaming data workflows.

• Experience with Databricks-based data platforms and lakehouse-oriented architectures.

• Hands-on experience utilizing AI-assisted engineering tools such as Claude, GitHub CoPilot, LLMs, or equivalent.

• Experience applying LLMs to engineering scenarios such as code generation and optimization.

• External candidates reaching the final interview stage are required to attend at least one in-person interview at the nearest S&P Global office.

• Knowledge of enterprise data cataloging, metadata management, business glossaries, lineage, operational metadata platforms, or semantic data layers.

• Experience with NeoXam DataHub or NeoXam-style mastering workflows.

• Understanding of semantic modeling.

• Strong leadership abilities, including mentoring, empowering team members, and fostering consensus across various groups.

🏝️ Benefits

• Eligibility for an annual incentive plan.

• Comprehensive health care coverage.

• Generous paid time off.

• Resources for continuous learning and career development.

• Competitive salary.

• Retirement planning assistance.

• Continuing education program with company-matched contributions to student loans.

• Financial wellness initiatives.

• Family-friendly benefits and perks.

• Discounts at retail partners.

• Referral incentive awards.