AI Platform
AI Platform
Built a FastAPI backend serving multiple LLM models with streaming responses — handling 10K concurrent inference requests with sub-100ms latency.
AI Platform
Built a FastAPI backend serving multiple LLM models with streaming responses — handling 10K concurrent inference requests with sub-100ms latency.
10K
concurrent requests
<100ms
P95 latency
Technology
View in portfolio