Principal Data Platform Engineer

This is a fully remote position, open to applicants in Arizona, +1 more state.

📋 Description

• Oversee the daily administration, configuration, and operational health of the enterprise data and AI platform portfolio: Power BI, AtScale, Databricks, Microsoft Copilot, and Claude.

• Manage Databricks workspaces, cluster policies, SQL warehouses, Unity Catalog, service principals, and security at the workspace level.

• Handle the Power BI tenant, including tenant settings, capacity management, workspace architecture, data gateways, and deployment pipelines.

• Administer AtScale, focusing on semantic model deployment, catalog and environment management, aggregate strategy, and tuning of the query engine.

• Oversee entitlements, seats, workspaces, projects, and new features within the enterprise AI platform.

• Define and execute processes for upgrades, patches, and releases.

• Ensure platform availability and performance while serving as the primary escalation point for incidents.

• Set up monitoring, alerting, observability, performance baselines, and capacity headroom tracking.

• Diagnose complex issues related to identity, network, compute, and the semantic layer to identify root causes.

• Support data teams and business users through triage, resolution, and follow-up actions.

• Create runbooks, self-service documentation, and enablement materials.

• Manage capacity sizing, compute policies, autoscaling, SKU selection, licensing, and AI expenditures.

• Identify and eliminate inefficiencies within the platform and develop showback/chargeback reporting.

• Forecast platform spending and collaborate with Finance on budget planning and ROI analysis.

• Implement governance protocols, access models, security standards, workspace/catalog standards, and content certification processes.

• Oversee AI data access, connectors, integrations, logging, retention, and auditing practices.

• Collaborate with Security, IT, and Legal teams regarding data residency, DLP, audits, and regulatory compliance.

• Maintain definitive platform standards, architectural decisions, and configuration baselines.

• Configure MCP connections, cross-platform connectivity, SSO, service principals, APIs, and gateways.

• Evaluate, pilot, and operationalize platform capabilities and integrations.

• Manage the promotion of environments across development, testing, and production stages.

• Instrument and report on usage, adoption, AI utilization, and licensing efficiency.

• Provide regular reports on platform health, costs, adoption, and risk to data leadership and executives.

• Maintain technical relationships with Microsoft, Databricks, AtScale, and Anthropic.

• Oversee vendor escalations, roadmap discussions, previews, renewals, licensing input, and service-level accountability.

• Advise the Enterprise Data Engineering leadership on strategic initiatives, architectural decisions, and build-versus-buy evaluations.

• Set technical standards, represent the platform in architecture, security, and AI governance forums, and mentor engineers, analysts, and users.

⛳️ Requirements

• 8+ years of experience in data platform engineering, platform administration, analytics engineering, or a closely related field.

• 5+ years of practical experience in administering and engineering enterprise cloud data platforms, including Databricks and Azure, with responsibilities for production environments, security, performance, reliability, and cost optimization.

• 5+ years of hands-on experience managing enterprise analytics and semantic platforms, including Power BI and similar technologies, supporting hundreds of production assets and over 1,000 users.

• Demonstrated accountability for production data platforms serving a large, distributed user base.

• Proven success in achieving measurable cost savings on cloud data and analytics platforms.

• Experience in establishing governance and technical standards across various tools and stakeholder groups.

• Background in managing enterprise software vendors, including support escalation and input on renewals.

• Proven track record of functioning as the senior technical authority on a platform without direct reports.

• Expert-level administration experience with Databricks, including Unity Catalog, cluster policies, SQL warehouses, identity and workspace configuration, and cost controls.

• Extensive experience in Power BI tenant and capacity administration, including capacity management, gateways, workspace architecture, and deployment pipelines.

• Hands-on experience in administering semantic layer platforms; AtScale is preferred.

• Strong working knowledge of Azure, encompassing Entra ID, networking, storage, private endpoints, and cost management.

• Expert-level SQL skills for troubleshooting, performance tuning, and usage analysis.

• Proficiency in Python and/or PowerShell for platform automation, API-driven administration, and reporting.

• Familiarity with version control, CI/CD, and infrastructure-as-code practices.

• Knowledge of SSO, SAML/OAuth, service principals, role-based access control, and data-level security frameworks.

• Practical experience in configuring and governing enterprise AI tools, connectors, MCP server connections, agent and assistant configurations, and access boundaries.

• Availability during primarily US business hours, with occasional after-hours work for updates, upgrades, and critical incidents.

• Exceptional written and verbal communication skills.

• Proven effectiveness in distributed, cross-time-zone teams.

• Comfort in presenting to executive stakeholders and accountability for platform outcomes.

🏝️ Benefits

• Comprehensive Medical, Dental, and Vision coverage.

• 401K plan with Matching contributions.

• Flexible Time Off policy.

• Corporate Fitness Program.

• Optional benefits such as Legal Plans, Accident and Hospital Indemnity, Pet Insurance, and more.

• Eligibility for annual performance bonuses or commission programs.