Senior Prompt Engineer

This is a fully remote position, open to applicants in Italy.

📋 Description

• Leverage Automatic Prompt Generation (APG) tools to establish foundational prompts for intricate parent-child template clusters.

• Operate and oversee the Automated Prompt Optimization (APO) tool, evaluate the results, and identify instances where the APO encounters deadlocks or plateaus.

• Manually compose, test, and enhance prompts to navigate complex template frameworks, address anti-patterns, and manage edge cases where tools are deficient or malfunctioning. Resolve edge-case challenges by crafting and improving manual prompts.

• Supervise shadowbot operations to guarantee adequate discrepancies (between human and LLM evaluations) are logged, produced, and monitored.

• Execute prompt variants against predefined gold data to consistently assess autorater quality against the human crowd baseline, calculating accuracy metrics like F1 scores, precision, and recall.

• Prepare technical launch readiness justifications (Launch Certification Documentation) for final approval.

⛳️ Requirements

• Native proficiency in Italian and fluent in English.

• Must reside in Italy.

• Bachelor’s, Master’s, or Doctorate degree in Computer Science, Data Science, Computational Linguistics, Human-Computer Interaction (HCI), Cognitive Science, or a related analytical discipline.

• Minimum of 4 years of experience as a Prompt Engineer. Demonstrated experience in fine-tuning Large Language Models (LLMs) for precise, structured outputs, complex classification tasks, and familiarity with chain-of-thought and few-shot learning techniques.

• Strong expertise in recognizing error patterns, analyzing model performance, and utilizing SQL or other data analytics tools.

• Capability to quickly acquire and master proprietary tools with minimal oversight.

• Exceptional verbal and written communication abilities.

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional growth and development.

• Flexible working hours and remote work options.

• Health and wellness programs.