Senior Data Engineer

This is a fully remote position, open to applicants in United States.

📋 Description

• Create, develop, and manage data infrastructure for a platform focused on clinical trial design.

• Construct ingestion pipelines to handle clinical trial protocols, informed consent form (ICF) documents, summaries of product characteristics (SmPCs), clinical study reports (CSRs), and articles published on PubMed, CTIS, and ClinicalTrials.gov.

• Manage PDF parsing, text extraction, and the normalization of structured data.

• Design and execute relational and knowledge graph data models using Amazon Aurora and GraphDB.

• Develop embedding and vectorization pipelines for retrieval-augmented generation (RAG) workflows within LangGraph agentic frameworks.

• Create and sustain ETL/ELT workflows that convert unstructured clinical content into searchable, linked data.

• Implement validation processes for clinical data quality, including classification of protocol sections, completeness of entity extraction, and integrity of cross-references.

• Develop data-serving APIs using Python and FastAPI for integration with Angular frontends and LangGraph agent layers.

• Establish data lineage tracking and audit trails to ensure regulatory compliance in AI-generated trial design recommendations.

⛳️ Requirements

• Proficient in Python development, particularly with PDF/document parsing libraries such as PyMuPDF, pdfplumber, unstructured.io, or equivalents.

• Advanced knowledge of PostgreSQL-compatible SQL, including Amazon Aurora; skilled in schema design, migrations, query optimization, and indexing strategies for extensive clinical datasets.

• Practical experience with graph databases like Neptune, Neo4j, or similar technologies.

• Proficient in SPARQL or Cypher.

• Background in ontology and knowledge graph modeling for biomedical entities.

• Familiarity with AWS services, such as Aurora PostgreSQL, S3, Lambda, Step Functions, SQS/SNS, and IAM.

• Experience in PDF text extraction, document section classification, and named entity recognition (NER) for clinical or biomedical literature.

• Knowledge of embedding models and vector stores like OpenSearch, pgvector, or Pinecone.

• Proven experience in building data-serving APIs with FastAPI, including asynchronous programming patterns and backend integration.

• Experience preparing data for LangChain/LangGraph applications and designing RAG pipelines, including chunking, retrieval, reranking, and integrating prompt-data.

• Familiar with workflow orchestration tools such as Airflow, Prefect, AWS Step Functions, Temporal, or similar platforms.

• Ability to design multi-stage Directed Acyclic Graphs (DAGs) with dependency management, retry mechanisms, monitoring, and error handling.

• Experience with Terraform or AWS CDK, Docker, and Git, including automated testing and deployment of pipelines on AWS.

• Understanding of clinical trial structures, including protocol sections, objectives, endpoints, eligibility criteria, study design, and statistical considerations.

• Familiarity with clinical data standards or terminologies such as MeSH, MedDRA, SNOMED, ATC codes, or CDISC is highly advantageous.

• Knowledge of regulatory data integrity requirements, including 21 CFR Part 11, EU Annex 11, and ALCOA+ principles.

🏝️ Benefits

• Accrued vacation.

• Medical insurance.

• Dental insurance.

• Vision insurance.

• 401(k) plan with company matching.

• Life insurance.

• Flexible spending accounts.

• Commitment to equal opportunity employment and diversity.