Python Engineer, Financial Data Platform – Integrations

This is a fully remote position, open to applicants in New York.

📋 Description

• Successfully deliver at least one full production integration with a financial or credit data provider (Experian, Plaid, or a bank transaction source).

• Establish automated validation and quality checks for the pipelines under your management, ensuring that failures are identified before reaching downstream consumers.

• Develop or significantly enhance a FastAPI service that provides internal access to provider data, complete with tests and documentation for other engineers to utilize.

• Create a well-documented data model and schema mapping for at least one provider domain, collaborating closely with our domain experts and analysts.

• Showcase one LLM-assisted extraction workflow for unstructured input (such as bank statements or credit memos) with robust output validation in place.

• Design, construct, and manage FastAPI services that present financial and credit data to internal stakeholders: focusing on API design, authentication, versioning, pagination, and thorough error handling.

• Oversee the integration layer with external providers (Experian, Plaid, bank transaction sources), including retries, backoff strategies, rate limiting, credential management, and ensuring graceful degradation in cases of provider slowdowns or outages.

• Develop the pipelines that integrate provider data into Cardiff's systems, utilizing serverless ETL on AWS (Glue, Athena, Step Functions, S3 data lakes) to support both batch and near-real-time ingestion.

• Conduct data modeling and schema mapping to maintain semantic consistency across varied provider formats, in collaboration with our domain experts and analysts.

• Create integrations that fulfill Cardiff's security, privacy, and regulatory requirements, with final interpretations and approvals managed by Legal, Compliance, and Security teams.

• Utilize LLM APIs for the extraction and classification of unstructured documents where it is beneficial, ensuring model outputs are validated prior to entering the pipeline.

• Maintain unit tests, integration tests, and documentation for every component you are responsible for.

⛳️ Requirements

• Over 5 years of experience in Python backend engineering focused on data-intensive systems, with an emphasis on recent, demonstrable work.

• Experience in building and operating production backend services within a modern Python framework (FastAPI preferred; Flask or Django acceptable), covering API design, authentication, testing, and asynchronous programming.

• Proven experience with reliable third-party API integration (including retries, idempotency, rate limits, partial failures, and observability), particularly with financial APIs from credit bureaus, banks, or payment processors.

• Significant hands-on experience with Snowflake in a production environment: including loading, modeling, and executing large-scale queries.

• Familiarity with AWS data services for pipeline orchestration (Glue, Athena, S3, Step Functions, Lambda), including efficient data lake layouts and partitioning strategies to optimize querying costs.

• Strong instincts for data modeling, schema alignment, and transformation processes.

• A solid understanding of API authentication, encryption, and the compliance considerations related to regulated financial data.

• Proficiency in SQL and Pandas.

🏝️ Benefits

• Medical, dental, and vision insurance coverage.

• 401(k) retirement plan.

• Flexible PTO and recognized holidays.

• Remote-first work environment, with support for your home office setup.