
Senior Software Engineer – Akamai Inference Cloud
Posted 2 days ago

Posted 2 days ago
This is a fully remote position, open to applicants in Poland.
• Design and sustain prompt processing and tokenization pipelines to ensure efficient model execution.
• Implement request routing, scheduling, and batching strategies to enhance throughput and reduce latency for concurrent inference workloads.
• Integrate new model architectures and serving backends into the existing runtime framework.
• Produce well-tested and thoroughly documented code.
• Engage in code reviews.
• Assist in operational readiness by monitoring, debugging, and performing performance analysis of runtime components.
• Collaborate with seasoned engineers to convert user requests into optimized model inputs.
• Relevant experience along with a Bachelor's degree or its equivalent in Computer Science or a related discipline.
• Proficiency in Python and at least one systems programming language, such as C++, Go, or Rust.
• Understanding of natural language processing concepts, including tokenization, encoding, and text processing pipelines.
• Familiarity with AI inference, model serving, or LLM deployment.
• Knowledge of inference frameworks such as TensorRT, vLLM, TorchServe, and Triton.
• Experience in building high-throughput, low-latency data processing systems or services.
• Familiarity with Linux systems, containerized environments, and profiling or debugging tools.
• Strong desire to learn and advance within the AI inference and model serving domain.
• Benefits that support health, well-being, finances, and life beyond work.
• FlexBase flexible work options: at home, in an office, or a combination of both.
Zen Educate
Weekday (YC W21)
Toast
Milliman
Get handpicked remote jobs straight to your inbox weekly.