Principal Data Platform Developer

This is a fully remote position, open to applicants in Texas.

📋 Description

• Develop and manage Databricks Asset Bundles for data ingestion, transformation, product orchestration, and job execution.

• Adhere to bundle standards and CI/CD methodologies utilizing GitHub Actions and service principals throughout development and production environments.

• Execute Salesforce Lakeflow Connect ingestion processes across various client organizations.

• Address sandbox re-authentication and field-level-security column-loss challenges while ensuring the integrity of PII-tagged silver tables.

• Implement Unity Catalog governance, which includes granting permissions, PII tagging, ABAC masking, and managing table-level access.

• Construct document-processing and retrieval pipelines leveraging foundation-model endpoints for tasks such as classification, detection, and inference.

• Create reporting marts and data products that align with business KPIs and support stakeholder reporting needs.

• Assess and choose LLM serving endpoints through model bake-offs and manage deprecation migrations.

• Troubleshoot and resolve issues with failing pipelines and jobs.

• Maintain documentation of known defects and runbooks for team use.

⛳️ Requirements

• Minimum of 3 years of experience in data engineering, with practical experience on Databricks.

• Proficient in Python, SQL, and PySpark.

• Experience with Delta Lake and Delta Live Tables/Lakeflow.

• Knowledge of Unity Catalog governance practices including grants, tagging, column masking, and row filters.

• Experience in deploying data infrastructure as code via Databricks Asset Bundles, Terraform, or similar tools.

• Familiarity with Git-based CI/CD processes.

• Understanding of Salesforce data models and CRM ingestion methodologies.

• Experience with building or integrating LLM workflows, including prompting, retrieval/RAG, and model assessment, is a significant advantage.

• Working knowledge of Azure services, including identity management, service principals, and storage solutions.

• Bachelor’s degree in computer science, engineering, or a related field, or equivalent relevant experience.

• Excellent communication and documentation abilities.

• Capability to collaborate with both technical and business stakeholders.

• If applicable, fulfill all DOT qualification criteria and adhere to relevant transportation regulations.

• Must possess authorization to work in the country of the position without employer sponsorship or assistance.

• Successful candidates must pass pre-employment screenings.

🏝️ Benefits

• Medical, Dental, and Vision Insurance.

• Company-funded health care concierge service.

• 401(k) plan with company contribution matching.

• Paid vacation, sick leave, personal time, and parental leave.

• Paid time off for volunteering.

• Employee Recognition Program allowing conversion of recognition points into gift cards.

• Employee Assistance Program.

• Access to on-demand training courses.