What's the Prompt and Response length in the Paper? #70
Description
The paper has mentioned 50K tps for generation phase and 100K throughput for prefill phase, my question is what's the tested prompt length and repsonse length in avg and P99?
{{ message }}