Teams at CAST AI

Optimize large-scale ML inference systems for throughput, latency, cold-start time, and cost per token. Tune SGLang, vLLM, and TensorRT-LLM across GPUs and distributed clusters; improve batching, scheduling, KV caching, quantization, weight loading, JIT caching, and multi-node inference. Profile and instrument systems to validate performance gains, eliminate storage and networking bottlenecks, and establish technical direction through reproducible benchmarks and experiments.