Staff Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Take ownership of the technical design for the data platform that supports Palmetto's products, partners, and AI agents.

• Develop and manage governed, reliable data products that transition from bronze to gold for analysts, engineers, product teams, partners, and AI applications.

• Define semantic context, documentation, and interfaces to ensure AI agents respond using governed definitions with traceability to the source.

• Implement classification, masking, access policies, and change control as code.

• Prepare the platform for SOC 2 and related compliance audits.

• Construct and maintain ingestion and orchestration as code, utilizing CI/CD, observability, alerting, and SLAs.

• Collaborate with finance, operations, product, and engineering teams to understand changing data requirements.

• Design and manage secure, authorized, low-latency data interfaces for products, partners, and agents.

• Review code, mentor colleagues, and provide engineering teams with templates and CI checks.

• Set technical direction, write foundational code, and work in partnership with core platform and product teams.

• Report to the Senior Manager of Data Engineering.

⛳️ Requirements

• Over 8 years of experience in data engineering, including roles at a Staff or lead level managing a platform utilized by multiple teams.

• Extensive expertise in SQL and Python.

• Strong database fundamentals in modern analytical warehouses like Snowflake or BigQuery, covering schema design, query plans, and distributed computing.

• Significant experience with dbt or a similar transformation framework, such as SQLMesh or Dataform.

• Familiarity with ETL orchestration tools like Fivetran or Airflow.

• Proficiency in infrastructure as code and container orchestration across AWS, GCP, or Azure.

• Proven ability to engineer beyond prompting, including executing coding agents in loops validated by specifications and tests.

• Experience in creating design documents that clarify architecture questions across teams.

• Excellent written and verbal communication skills when interacting with non-technical partners.

• Working knowledge of API design, authentication, and authorization mechanisms like OAuth/OIDC and RBAC.

• Understanding of PII handling regulations.

• Employment is contingent upon successful completion of a background check.

• Familiarity with reporting tools such as Looker and Hex is advantageous.

🏝️ Benefits

• Unlimited PTO.

• Medical coverage.

• Dental coverage.

• Vision coverage.

• Paid parental leave.

• Retirement plans.

• Promote-from-within culture.

• Opportunities for talent development and career growth.

• Work that is purpose-driven.

• A culture focused on collaboration, impact, and balance.