Principal Software Engineer – Akamai Inference Cloud

Do you thrive on solving complex technical challenges in AI infrastructure?

Are you ready to architect the future of AI at the edge?

The Akamai Inference Cloud team is part of Akamai’s Cloud Technology Group. We build AI platforms for efficient, compliant, and high-performing applications. These platforms support customers in running inference models and empower developers to create advanced AI solutions effectively. #AIC

The Principal Software Engineer will lead and design a globally distributed AI inference platform with OpenAI-compatible endpoints. Responsibilities include making technical decisions, setting architectural patterns, and managing large-scale inference workloads. This position requires broad experience, extensive knowledge of AI/ML systems, and a history of successfully delivering production AI platforms.

As a Principal Software Engineer, you will be responsible for:

• Architecting and designing the core infrastructure of Akamai Inference Cloud, ensuring it is performant, compliant, economical, and explainable.
• Providing thought leadership on AI inference optimization, model serving architectures, and establishing technical standards for the platform.
• Making critical technical decisions on frameworks, tools, and approaches for AI workload deployment and global traffic orchestration.
• Mentoring engineers and collaborating with cross-functional teams to define technical requirements and elevate team capabilities.
• Contributing hands-on to critical path development, code reviews, and architectural implementations.

To be successful in this role you will:

• Have broad software engineering experience with deep expertise in AI inference optimization, model serving, and LLM deployment at scale.
  
• Demonstrate mastery of inference frameworks (TensorRT, vLLM, TorchServe, Triton), containerization strategies, and hardware-specific optimizations for AI workloads.
• Show expert-level proficiency with cloud-native technologies, Kubernetes, distributed systems, and architecting services at global scale with strict SLOs.
• Possess strong opinions and proven experience building platforms for AI developers (RAG, agents, fine-tuning workflows).
• Demonstrate excellent technical leadership with ability to mentor engineers, influence architectural decisions, and communicate effectively across teams.
• Have deep knowledge of AI safety, responsible AI practices, and explainability frameworks.

Our ability to shape digital life today relies on developing exceptional people like you. The kind that can turn impossible into possible. We’re doing everything we can to make Akamai a great place to work. A place where you can learn, grow and have a meaningful impact.

With our company moving so fast, it’s important that you’re able to build new skills, explore new roles, and try out different opportunities. There are so many different ways to build your career at Akamai, and we want to support you as much as possible. We have all kinds of development opportunities available, from programs such as GROW and Mentoring, to internal events like the APEX Expo and tools such as Linkedin Learning, all to help you expand your knowledge and experience here.

Not sure if this job is the right match for you or want to learn more about the job before you apply? Schedule a 15-minute exploratory call with the Recruiter and they would be happy to share more details.