Lead Software Engineer – Search Platform

This is a fully remote position, open to applicants in United States.

📋 Description

• Design a comprehensive full-stack Search Platform encompassing all aspects of indexing and scoring, query comprehension, rewriting, federation, and customizable search experiences.

• Consistently enhance search quality by utilizing evaluation metrics such as precision@K, recall@K, MRR, and relevance testing through real scientific applications.

• Develop advanced hybrid search pipelines that integrate sparse (keyword), structured (metadata), and dense (vector) retrieval. Move beyond standard OpenSearch to create tailored ranking logic, reciprocal rank fusion, and relevance adjustments that pinpoint the precise "needle in the haystack" for drug discovery.

• Set an example by writing code, reviewing designs, and establishing high standards for engineering quality within the Search Platform team. Provide mentorship to engineers and cultivate the team's expertise in search and distributed systems.

• Participate in architectural decision-making, technical strategy, and platform-wide enhancements to expedite scientific insight generation.

• Manage and maintain the Search Platform infrastructure, ensuring high availability, scalability, performance, and observability throughout indexing, embedding generation, and query execution.

• Create and sustain backend services and APIs in Python and TypeScript that empower search functionalities for scientists, data engineers, and AI applications.

• Guarantee security, compliance, and tenant isolation while operating search services within enterprise bio-pharma settings.

• Collaborate with Applied AI Scientists to incorporate embeddings, transformer models, and chemical fingerprints into production search workflows.

• Design and implement scientific entity resolution and knowledge graph pipelines to convert raw text into interconnected knowledge. Develop systems that extract and connect chemical and biological entities (NER/NED) from unstructured documents, allowing the search engine to comprehend relationships between compounds, targets, and assays.

⛳️ Requirements

• Over 10 years of experience in backend or platform engineering, focusing on building distributed, production-grade systems.

• Practical experience with search technologies such as Elasticsearch/OpenSearch, Lucene, or vector databases, encompassing not just deployment, but also custom configuration, relevance tuning, and performance optimization at scale.

• In-depth knowledge of semantic and hybrid retrieval: embeddings, transformer models, vector similarity, ranking logic, relevance tuning, and their integration with traditional keyword search.

• Expert coding capabilities in TypeScript and Python for developing robust APIs and backend services.

• Demonstrated success in building and managing search infrastructure on cloud platforms (AWS preferred), including containerization, CI/CD, observability, and capacity planning.

• Familiarity with processing scientific or unstructured data, such as documents, tables, analytical results, or experimental datasets.

• Outstanding communication and collaboration skills, adept at working alongside scientists, AI researchers, and product teams.

• Exposure to NLP, LLMs, embedding generation, or retrieval-augmented workflows.

• Experience with vector databases/embedding stores (e.g., OpenSearch) for supporting semantic search and RAG.

• Strong problem-solving abilities, comfortable navigating ambiguity, and translating loosely defined scientific workflows and user needs into well-engineered search systems.

🏝️ Benefits

• 100% employer-paid benefits for all eligible employees and their immediate family members.

• Unlimited paid time off (PTO).

• 401K plan.

• Flexible working arrangements, including remote work options.

• Company-paid Life Insurance, LTD/STD.

• A culture of continuous improvement that fosters career growth and coaching opportunities.