AI Developer

This is a fully remote position, open to applicants in Massachusetts.

📋 Description

• Design, prototype, build, test, and deploy AI applications and agents into production.

• Determine the most suitable Copilot Studio, workflow, agent, custom service, vendor, or non-AI methods based on Build-vs-Buy considerations.

• Create retrieval-augmented generation solutions, encompassing document ingestion, chunking, indexing, metadata, retrieval tuning, and grounded responses with source citations.

• Integrate with both internal and external systems utilizing authenticated REST and GraphQL APIs.

• Collaborate on data pipelines for both documents and structured data.

• Implement least-privilege permissions, single sign-on, secrets management, PHI/PII protections, output validation, and suitable logging and retention controls in conjunction with IT Security.

• Develop evaluation harnesses featuring golden test sets, subject-matter expert rubrics, and production sampling methodologies.

• Track accuracy, citation support, completeness, latency, and transaction costs, including results categorized by plan type.

• Set up telemetry for prompts, retrievals, tool calls, and outputs.

• Maintain runbooks, hand-off documentation, cost controls, and operational monitoring processes.

• Progress solutions through the CBG AI Solution Delivery Playbook's decision gates and required documentation.

• Collaborate with the AI Risk Manager, Compliance, Data Protection Officer, Security, and the AI Ethics Committee.

• Share reusable components, templates, and approved prompt patterns.

• Complete AI governance onboarding, facilitate enhancements via the Production Readiness Review, and establish golden-test-set evaluation harnesses.

• Launch approved AI solutions, publish reusable components, and activate monitoring dashboards.

• Achieve approved business KPIs and maintain records for inventory, risk classification, evaluation, and monitoring.

• Ensure all solutions comply with HIPAA, HITRUST CSF, NIST AI Risk Management Framework, and relevant OWASP guidelines.

⛳️ Requirements

• Bachelor's degree in Computer Science, Information Systems, or a related field, or equivalent practical experience.

• Over 3 years of professional software development experience.

• Hands-on experience in generative AI development and evaluation, including RAG, context engineering, tool/function calling, and at least one production or production-pilot solution.

• Proficient in Python and/or C#/.NET development.

• Experience with production APIs, REST integration, OAuth or similar authorization methods, and secure secrets/configuration management.

• Capability to review, test, debug, and maintain AI-generated code.

• Familiarity with SQL, data modeling, and data pipelines.

• Experience managing HIPAA-regulated information, including PHI masking and access controls.

• Background in AI regression testing, monitoring, model version management, release controls, cost optimization, and guarded multi-step agents.

• Ability to document technical decisions, adhere to governance and change control, lead business requirements sessions, and advise on delaying or halting solutions that do not meet evaluation standards.

• Preferred: Experience with Microsoft Copilot Studio, Power Automate, Power Platform, Azure OpenAI, and Azure AI Search.

• Preferred: Proven improvement of retrieval or extraction systems using before-and-after evaluation metrics.

• Preferred: Experience with AI coding agents and Model Context Protocol (MCP).

• Preferred: Familiarity with healthcare, third-party administration, employee benefits, claims, eligibility, plan documents, or EDI.

• A collaborative communicator who can translate business processes into technical requirements and work effectively across business, engineering, security, and compliance teams.

• An analytical, detail-oriented problem solver who values measurable quality, documentation, and continuous improvement.

• Demonstrates sound judgment and accountability in areas of privacy, fairness, human oversight, and responsible AI deployment.

• An adaptable, hands-on builder comfortable choosing between low-code and custom engineering methods.

• Cobalt Benefits Group participates in E-Verify to confirm the employment eligibility of all new hires.

🏝️ Benefits

• Comprehensive medical, dental, and vision insurance.

• Semi-annual employer HSA contributions, covering 50% of the HDHP plan's annual deductible.

• Company-provided Basic Life and AD&D insurance.

• Company-paid Short-Term and Long-Term Disability coverage.

• Flexible Spending Accounts available.

• 401(k) Retirement Plan with up to a 6% employer match.

• Over 10 paid holidays.

• Fully paid half-day Summer Fridays.

• Generous paid vacation and sick leave.

• Annual Paid Volunteer Day.

• Annual Tuition Reimbursement opportunity.

• Annual Health and Wellness Reimbursement.

• Numerous fun company events.