linear growth of latency

#57

by yn-mark - opened Jul 26, 2023

Jul 26, 2023

i want to increase the batch_size to get larger throughput,but i got linear growth of latency.if i run some incorrect settings?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment