AI Prompt Engineer

This is a fully remote position, open to applicants in South Africa.

📋 Description

• Take ownership of designing, testing, and continuously improving the prompts and system instructions that drive Noodle’s AI agents in enrollment, learning, and student support.

• Write, revise, and maintain system prompts and instruction sets.

• Convert educational, operational, pedagogical, and business requirements into dependable agent behavior, encompassing personas, tone, constraints, fallbacks, and multi-turn logic.

• Develop prompt architectures for multi-step, chained, and RAG-augmented agent workflows.

• Collaborate with engineering teams to configure and deploy agents via Noodle’s AI orchestration platform.

• Create three-way chat experiences linking learners, campus or Noodle support staff, and AI assistants.

• Construct evaluation frameworks to assess accuracy, tone, hallucination rate, task completion, and alignment with learning objectives.

• Utilize Langfuse to oversee production prompt performance, detect regressions, and prioritize enhancements.

• Develop red-teaming and adversarial testing protocols.

• Establish practices for prompt versioning and maintain reusable prompt components.

• Collaborate with Noodle team members and university stakeholders to design, build, and test agents.

• Fine-tune companion agents for enrollment and student success teams in specific university contexts.

• Contribute to the documentation of prompt engineering guidelines and best practices.

• Monitor model capability changes across OpenAI, Anthropic, and other providers, identifying opportunities and risks.

• Undertake additional duties as assigned.

⛳️ Requirements

• A minimum of 2 years of experience in designing and refining prompts for LLM-powered applications in a production setting.

• Profound understanding of prompt patterns: few-shot examples, chain-of-thought, system versus user roles, output formatting constraints, tool-use prompting, and RAG integration.

• Excellent written communication skills.

• Experience in developing and conducting prompt evaluations: defining metrics, crafting test cases, and analyzing results to inform improvement decisions.

• Comfort in collaborating across disciplines, including discussing context windows with engineers and Bloom’s Taxonomy with faculty members.

• Familiarity with LLM observability tools (Langfuse, Weights & Biases, or similar).

• Understanding of LTI (Learning Tools Interoperability) and AI integrations with LMS platforms such as Moodle, Canvas, or Instructure.

• Experience with voice agents or multimodal prompting.

• Ability to read and write basic JavaScript or Python for prompt automation and testing pipelines.

• Knowledge of APIs from OpenAI, Anthropic, or other model providers.

• Capacity to thrive in ambiguous situations and iterate swiftly based on user feedback.

• Proficient in working effectively within a dynamic, rapidly changing, team-oriented environment.

• Must possess legal authorization to live and work in South Africa and maintain residency there throughout employment.

🏝️ Benefits

• Enjoy the flexibility of working from your home office!

• Attractive compensation package.

• We provide the tools you need! A laptop (Mac is our preferred choice) and an EcoFlow inverter.

• 12 weeks of paid parental leave benefits.

• 20 working days of annual leave plus 10 paid national holidays.

• Medical/RA benefits.

• Internet benefits.

• Full access to our employee assistance program (EAP) through Company Wellness.

• Annual education stipend for lifelong learning.