[Remote] Senior Engineer II, AI Inference Engine Systems

Note: The job is a remote job and is open to candidates in USA. DigitalOcean is expanding its AI Infrastructure layer to support the next generation of AI-driven applications. They are seeking a Senior Engineer II to join their AI Inference Engine Systems team, responsible for designing, developing, and delivering high-scale data plane services that power their Inference as a Service offering.

Responsibilities

• Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models
• Architect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standards
• Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing
• Work cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs
• Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement
• Maintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform health

Skills

• Strong experience with microservices, messaging systems, databases, and infrastructure as code
• Hands-on experience hosting large language or multimodal models using inference engines like vLLM, SGLang, or Modular
• Familiarity with distributed inference serving frameworks such as llm-d, NVIDIA Dynamo, or Ray Serve
• Understanding of GPU-level optimization and experience with interconnect technologies like NVlink, XGMI, or RoCE
• Knowledge of common LLM architectures and optimization techniques (e.g., continuous batching, quantization)
• Expert-level proficiency in GoLang or Python and familiarity with gRPC
• Proven experience shipping customer-facing software products and running critical services in a high-scale environment similar to DigitalOcean
• Experience integrating and building with open-source software

Benefits

• We provide employees with reimbursement for relevant conferences, training, and education.
• All employees have access to LinkedIn Learning's 10,000+ courses to support their continued growth and development.
• Employee Assistance Program
• Local Employee Meetups
• Flexible time off policy
• You may qualify for a bonus in addition to base salary; bonus amounts are determined based on company and individual performance.
• Equity compensation to eligible employees, including equity grants upon hire and the option to participate in our Employee Stock Purchase Program.

Company Overview

Company H1B Sponsorship