Principal Engineer, Data Platforms – Architecture, Ingestion

This is a fully remote position, open to applicants in New Jersey.

📋 Description

• Define and take ownership of the architecture, strategy, and multi-year roadmap for the Unified Ingestion Platform and Federated Data Lake.

• Design and scale ingestion and processing capabilities to support over 250 TB per day, encompassing logs, metrics, traces, security events, and operational datasets.

• Establish and govern the enterprise Databricks architecture, which includes multi-cloud account structures, workspace strategies, environment isolation, metadata management, data governance, lineage, and secure access models.

• Lead the implementation of Infrastructure-as-Code, Policy-as-Code, automated provisioning, and enterprise-grade CI/CD processes.

• Define practices for security, governance, observability, reliability, and operational excellence that align with OTEL, OCSF, and compliance standards.

• Act as the primary technical escalation point across engineering, security, infrastructure, and platform teams.

• Develop FinOps strategies, cost allocation frameworks, resource optimization practices, and capacity planning models.

• Mentor engineers and architects, promoting the adoption of cloud-native engineering principles and platform best practices.

• Assess emerging technologies and create strategic roadmaps.

• Enable analytics, security analytics, observability intelligence, and AI/ML workloads through governed enterprise data capabilities.

• Collaborate with engineering, security, operations, infrastructure, and business stakeholders to deliver a resilient, scalable, secure, and future-ready data platform.

⛳️ Requirements

• Over 10 years of experience in architecting, building, and operating enterprise-scale cloud data platforms, data lakes, distributed systems, and large-scale data ingestion ecosystems.

• More than 8 years of experience in designing and administering Databricks platforms, including Apache Spark, Delta Lake, Unity Catalog, secure data access patterns, metadata management, lineage, and enterprise governance frameworks.

• At least 8 years of experience implementing Infrastructure-as-Code, platform automation, CI/CD pipelines, and Policy-as-Code controls using Terraform, GitHub Actions, or similar platforms.

• A minimum of 7 years of experience designing and operating high-scale streaming and telemetry ingestion architectures that process over 100 TB per day using Kafka, Event Hub, Kinesis, Pub/Sub, NiFi, Cribl, or comparable platforms.

• At least 5 years of experience serving as a Principal Engineer, Lead Architect, or a similar technical leadership role.

• Proven experience in driving enterprise platform strategy and multi-cloud architecture across Azure, AWS, and GCP.

• Familiarity with engineering standards, security governance, and large-scale transformation initiatives.

• Bachelor’s degree from an accredited university or equivalent work experience (High School diploma + 4 years of relevant experience).

• Preferred qualifications include experience with enterprise-scale data platforms, analytics ecosystems, observability platforms, or cloud-native engineering platforms in large, highly regulated environments.

• Preferred knowledge of Unity Catalog, Delta Sharing, metadata services, lineage frameworks, information stewardship, retention controls, and federated operating models.

• Preferred experience in Splunk rationalization, security analytics modernization, SIEM ecosystems, telemetry consolidation, or monitoring transformation programs.

• Preferred background in secure serverless computing environments, multi-tenant architectures, workload isolation strategies, and Databricks deployments across Azure, AWS, and GCP.

• Preferred knowledge of FinOps disciplines, consumption governance, chargeback methodologies, provisioning automation, and resource optimization.

• Preferred Databricks certifications and familiarity with AI/ML, advanced analytical workloads, OTEL, and OCSF.

🏝️ Benefits

• CVS Health bonus, commission, or short-term incentive program.

• Award target in the company’s equity award program.

• Medical, dental, and vision coverage.

• Paid time off.

• Retirement savings options.

• Wellness programs.

• Additional resources supporting physical, emotional, and financial well-being.

• Initiatives promoting diversity, inclusion, and belonging.

• Support for individuals with military experience and military spouses.