status indicates the current state of batch processing.

Batches transition from `In_progress through optional `Canceling to `Ended.

request_counts tracks the status of individual requests in a batch.

The sum of all counts equals the total number of requests in the batch.

t represents a batch job with its current state and statistics.

Batches have a 24-hour processing window before expiration. Results remain available for download after completion.

request defines a single message request within a batch.

The params field should contain the same JSON structure as a regular Messages.create call, including model, messages, and other parameters.

individual_response contains the result of a single batch request.

Each response maps back to its original request via custom_id. Failed requests include structured error information.

submit client ~requests () submits a new batch for processing.

• parameter requests

  List of message requests to process (max 10,000).

• raises Api_error

  if the batch is too large or malformed.

Example: Submits a batch of similar requests.

  let make_request i =
    {
      custom_id = Printf.sprintf "req_%d" i;
      params =
        `Assoc
          [
            ("model", `String "claude-3-5-haiku-20241022");
            ("max_tokens", `Int 100);
            ( "messages",
              `List
                [
                  `Assoc
                    [
                      ("role", `String "user");
                      ( "content",
                        `String (Printf.sprintf "Summarize article %d" i)
                      );
                    ];
                ] );
          ];
    }
  in
  let requests = List.init 100 make_request in
  match Batches.submit client ~requests () with
  | Ok batch -> Printf.printf "Submitted batch %s\n" batch.id
  | Error e -> Printf.eprintf "Error: %s\n" (string_of_error e)

get client ~batch_id () retrieves the current status of a batch.

Use this to poll for batch completion or check processing statistics.

Example: Polls until batch completes.

  let rec wait_for_completion client batch_id =
    match Batches.get client ~batch_id () with
    | Ok batch when batch.processing_status = `Ended ->
        Printf.printf "Batch completed: %d succeeded, %d failed\n"
          batch.request_counts.succeeded batch.request_counts.errored
    | Ok _ ->
        Eio.Time.sleep env#clock 5.0;
        wait_for_completion client batch_id
    | Error e -> Printf.eprintf "Error: %s\n" (string_of_error e)

list client ?limit ?after_id () retrieves a paginated list of batches.

Lists batches in reverse chronological order (newest first).

• parameter limit

  Maximum batches per page (default: 20).

• parameter after_id

  Pagination cursor for subsequent pages.

cancel client ~batch_id () cancels an in-progress batch.

Only batches with status `In_progress can be canceled. Already-processed requests are not affected.

• raises Api_error

  if the batch has already ended or been canceled.

results client ~batch_id () streams individual results from a completed batch.

Results are streamed as newline-delimited JSON. The batch must have status `Ended to retrieve results.

• raises Api_error

  if the batch hasn't completed yet.

Example: Processes batch results.

  match Batches.results client ~batch_id () with
  | Ok stream ->
      Eio.Stream.iter
        (function
          | { custom_id; result = `Succeeded response } ->
              Printf.printf "Request %s succeeded\n" custom_id
          | { custom_id; result = `Errored error } ->
              Printf.printf "Request %s failed: %s\n" custom_id
                error.message)
        stream
  | Error e -> Printf.eprintf "Error: %s\n" (string_of_error e)