collab: Temporarily bypass LLM rate limiting for staff (#16089)

Marshall Bowers created 1 year ago

This PR makes it so staff members will be exempt from rate limiting by
the LLM service.

This is just a temporary measure until we can tweak the rate-limiting
heuristics.

Staff members are still subject to upstream LLM provider rate limits.

Release Notes:

- N/A

Change summary

crates/collab/src/llm.rs | 5 +++++
1 file changed, 5 insertions(+)

Detailed changes

crates/collab/src/llm.rs

@@ -400,6 +400,11 @@ async fn check_usage_limit(
     ];
 
     for (usage, limit, resource) in checks {
+        // Temporarily bypass rate-limiting for staff members.
+        if claims.is_staff {
+            continue;
+        }
+
         if usage > limit {
             return Err(Error::http(
                 StatusCode::TOO_MANY_REQUESTS,