8000
Skip to content

[BFCL] OpenAICompletionsHandler: stale keep-alive connections cause spurious 400 errors #1339

Description

@jhughes2112

Describe the issue
When running BFCL against a self-hosted inference server (llama-server, vllm, etc.), the generate phase produces intermittent 400 responses — typically in bursts of 3–5 — then recovers. The errors are not caused by invalid request bodies, exhaustively verified it was not a server configuration issue. Turn concurrency down to 1 on your server and it happens all the time, if the model is local and fast.

ID datapoint
Logging request.content in an httpx send hook shows a valid, fully-serialized JSON body. Server-side verbose logging for the same request (llama-server) shows an empty body received. So the connection likely was cut before delivery.

What is the issue
OpenAICompletionsHandler creates one OpenAI client that persists for the entire benchmark run. The underlying httpx connection pool keeps connections alive between requests. Between test cases, the inference server may close an idle connection. httpx then reuses that dead connection for the next POST: the server receives only the TCP headers before the RST, sees an empty body, and returns 400. The httpx client does not retry non-idempotent POST requests by design, so the error surfaces as a BFCL failure. This throws off your error rate.

Proposed Changes
Pass a custom http_client with max_keepalive_connections=0 to the OpenAI() constructor. This forces a new TCP connection per request. The overhead is negligible relative to inference latency (LLM inference >> TCP handshake). httpx is already a transitive dependency via the openai package, so no new dependency is iintroduced.

The fix belongs in _build_client_kwargs in openai_completion.py. Here's the patch:

File: bfcl_eval/model_handler/api_inference/openai_completion.py

   import json
   import os
   import time
   from typing import Any

  +import httpx
   from bfcl_eval.constants.type_mappings import GORILLA_TO_OPENAPI

       def _build_client_kwargs(self):
           kwargs = {}

           if api_key := os.getenv("OPENAI_API_KEY"):
               kwargs["api_key"] = api_key

           if base_url := os.getenv("OPENAI_BASE_URL"):
               kwargs["base_url"] = base_url

           if headers_env := os.getenv("OPENAI_DEFAULT_HEADERS"):
               kwargs["default_headers"] = json.loads(headers_env)

  +        kwargs["http_client"] = httpx.Client(limits=httpx.Limits(max_keepalive_connections=0, max_connections=100))

           return kwargs

Additional Context:
Disable keep-alive to prevent stale connections from causing 400s. Between benchmark requests, the server may close an idle connection. httpx will attempt to reuse it, sending the POST body on a dead socket. The server receives only headers (empty body) and returns 400. Disabling keep-alive forces a fresh connection per request, which is negligible overhead compared to inference latency.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    0