Senior Data Platform Architect – Palantir Foundry, Databricks Integration

This is a fully remote position, open to applicants in Latin America.

📋 Description

• Design and establish enterprise-level federated architectures that integrate Palantir Foundry and Databricks.

• Set standards for data sharing across Foundry, Databricks, BigQuery, and Cloud Storage environments.

• Define architectural patterns utilizing Delta Lake, Apache Iceberg, Unity Catalog, Foundry Ontology, Virtual Tables, and Catalog Federation.

• Drive the modernization of outdated ETL architectures towards metadata-driven, cloud-native data products.

• Establish enterprise data governance, lineage, metadata, and ownership frameworks across both platforms.

• Develop architectural roadmaps that support AI, machine learning, real-time analytics, and self-service data access.

• Design and implement solutions for the Foundry Databricks Connector, including bulk imports, incremental data synchronization, virtual tables, compute pushdown, and external model integrations.

• Construct scalable ingestion, transformation, and enrichment pipelines using PySpark, Spark SQL, Python, Databricks Workflows, Airflow, and Foundry Code Repositories.

• Architect secure connections among Palantir Foundry, Databricks, Google Cloud Platform, BigQuery, Cloud Storage, and Enterprise APIs.

• Facilitate AI and machine learning workloads across Foundry and Databricks environments.

• Act as the technical authority on enterprise data platform architecture and lead architecture review sessions.

⛳️ Requirements

• Fluent in English.

• Located in Latin America.

• Over 10 years of experience in Data Engineering, Data Architecture, or Platform Engineering.

• More than 5 years of experience with Databricks Lakehouse Architecture.

• Experience in implementing or managing Palantir Foundry data platforms.

• In-depth knowledge of Databricks, PySpark, Spark SQL, Delta Lake, Python, and Data Modeling.

• Strong understanding of Unity Catalog, Apache Iceberg, Lakehouse Architectures, Data Governance, and Metadata Management.

• Practical experience with Google Cloud Platform, including BigQuery, Cloud Storage, Pub/Sub, IAM, and Networking.

• Proven experience in designing highly scalable distributed data systems.

• Excellent skills in architecture, leadership, and stakeholder management.

• Certification in Palantir Foundry or relevant implementation experience is advantageous.

• Experience in Ontology development and operational decision platforms is a plus.

• Knowledge of AI/ML infrastructure and model deployment is beneficial.

• Familiarity with Terraform, Kubernetes, GitHub Actions, and CI/CD Pipelines is a plus.

• Background in survey, consumer intelligence, marketing, or ad-tech data platforms is a plus.

• Experience in supporting data products at the scale of Fortune 500 companies is a plus.

🏝️ Benefits

• Remote work.

• Full-time contract through a payroll platform.