zeta: Smaller reject batches (#43942) (cherry-pick to preview) (#43950)

zed-zippy[bot] and Agus Zubiaga created 1 month ago

Cherry-pick of #43942 to preview

----
We were allowing the client to build up to
`MAX_EDIT_PREDICTION_REJECTIONS_PER_REQUEST`. We'll now attempt to flush
the rejections when we reach half max.

Release Notes:

- N/A

Co-authored-by: Agus Zubiaga <agus@zed.dev>

Change summary

crates/zeta/src/zeta.rs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Detailed changes

crates/zeta/src/zeta.rs 🔗

@@ -941,7 +941,7 @@ impl Zeta {
         });
 
         let reached_request_limit =
-            self.rejected_predictions.len() >= MAX_EDIT_PREDICTION_REJECTIONS_PER_REQUEST;
+            self.rejected_predictions.len() >= MAX_EDIT_PREDICTION_REJECTIONS_PER_REQUEST / 2;
         let reject_tx = self.reject_predictions_tx.clone();
         self.reject_predictions_debounce_task = Some(cx.spawn(async move |_this, cx| {
             const DISCARD_COMPLETIONS_DEBOUNCE: Duration = Duration::from_secs(15);