LLM Serving
-
The Swarm's Bill Arrives at the Deadline
MAServeBench held the generated work fixed and changed only the order in which agents were allowed to run. Median deadline attainment went from 100% to 4%. October's measurements put the cost of a swarm at the deadline, and the token meter is the smaller bill.