AI Evaluation Engineer – Planning & Operations

This is a fully remote position, open to applicants in Bangladesh.

📋 Description

• Design and develop **multi-agent benchmark tasks** that encompass:

• Planning, scheduling, and resource allocation.

• Operational decision-making in areas such as logistics, project planning, incident response, and capacity planning.

• Create **constraint-rich problem statements** that feature multiple interacting variables.

• Develop **verification scripts** to assess:

• Feasibility (ensuring all constraints are satisfied).

• Completeness (confirming all requirements are met).

• Optimality (evaluating the efficiency of solutions).

• Define **task decomposition strategies** across specialized sub-agents, such as resource allocation, constraint resolution, and optimization.

• Model realistic operational systems that incorporate **dependencies, timelines, and constraints**.

• Implement validation logic and evaluation pipelines utilizing Python.

• Work within Docker environments to ensure reproducibility and execution.

• Collaborate with internal teams to enhance **task quality, coverage, and evaluation rigor**.

⛳️ Requirements

• A minimum of 5 years of experience in **operations, project management, logistics, or supply chain**.

• Strong capability to **formalize constraints, dependencies, and scheduling logic**.

• Proficiency in **Python** for developing validation and verification scripts.

• Familiarity with **optimization techniques** such as linear programming, constraint satisfaction, and scheduling algorithms.

• Excellent **structured problem-solving and decomposition abilities**.

• Experience with **AI benchmarks or evaluation frameworks** (e.g., SWE-bench or similar).

• Practical experience with **Docker** (including Dockerfiles, image builds, and debugging).

• **Nice to Have**:

• Background in **operations research or domains that heavily involve optimization**.

• Experience with **simulation or modeling tools**.

• Familiarity with **AI planning systems or automated reasoning**.

• Project management experience or relevant certifications (PMP, Agile, etc.).

🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Opportunities for professional development and continuous learning.

• Flexible working hours and remote work options.

• Comprehensive health and wellness benefits.

• Collaborative and innovative work environment.