package anthropic

  1. Overview
  2. Docs

Module Anthropic.BatchesSource

Batches API for asynchronous bulk message processing.

This module enables processing multiple message requests asynchronously, reducing costs and improving throughput for large-scale operations. Batches process in the background and results can be retrieved when complete.

Sourcetype status = [
  1. | `In_progress
  2. | `Canceling
  3. | `Ended
]

status indicates the current state of batch processing.

Batches transition from `In_progress through optional `Canceling to `Ended.

Sourcetype request_counts = {
  1. canceled : int;
    (*

    Requests canceled before processing.

    *)
  2. errored : int;
    (*

    Requests that failed during processing.

    *)
  3. expired : int;
    (*

    Requests that expired before processing.

    *)
  4. processing : int;
    (*

    Requests currently being processed.

    *)
  5. succeeded : int;
    (*

    Successfully completed requests.

    *)
}

request_counts tracks the status of individual requests in a batch.

The sum of all counts equals the total number of requests in the batch.

Sourcetype t = {
  1. id : string;
    (*

    Unique batch identifier.

    *)
  2. type_ : string;
    (*

    Batch type (always "messages_batch").

    *)
  3. processing_status : status;
    (*

    Current processing state.

    *)
  4. request_counts : request_counts;
    (*

    Request statistics.

    *)
  5. created_at : string;
    (*

    ISO 8601 creation timestamp.

    *)
  6. expires_at : string;
    (*

    ISO 8601 expiration timestamp.

    *)
  7. ended_at : string option;
    (*

    ISO 8601 completion timestamp.

    *)
  8. cancel_initiated_at : string option;
    (*

    ISO 8601 cancellation timestamp.

    *)
  9. archived_at : string option;
    (*

    ISO 8601 archival timestamp.

    *)
  10. results_url : string option;
    (*

    URL for downloading results.

    *)
}

t represents a batch job with its current state and statistics.

Batches have a 24-hour processing window before expiration. Results remain available for download after completion.

Sourcetype request = {
  1. custom_id : string;
    (*

    User-defined identifier for tracking.

    *)
  2. params : Yojson.Safe.t;
    (*

    Message API parameters as JSON.

    *)
}

request defines a single message request within a batch.

The params field should contain the same JSON structure as a regular Messages.create call, including model, messages, and other parameters.

Sourcetype individual_response = {
  1. custom_id : string;
    (*

    The custom ID from the original request.

    *)
  2. result : [ `Succeeded of Messages.response | `Errored of api_error ];
    (*

    Processing result.

    *)
}

individual_response contains the result of a single batch request.

Each response maps back to its original request via custom_id. Failed requests include structured error information.

Sourceval submit : client -> requests:request list -> unit -> (t, error) result

submit client ~requests () submits a new batch for processing.

  • parameter requests

    List of message requests to process (max 10,000).

  • raises Api_error

    if the batch is too large or malformed.

Example: Submits a batch of similar requests.

  let make_request i =
    {
      custom_id = Printf.sprintf "req_%d" i;
      params =
        `Assoc
          [
            ("model", `String "claude-3-5-haiku-20241022");
            ("max_tokens", `Int 100);
            ( "messages",
              `List
                [
                  `Assoc
                    [
                      ("role", `String "user");
                      ( "content",
                        `String (Printf.sprintf "Summarize article %d" i)
                      );
                    ];
                ] );
          ];
    }
  in
  let requests = List.init 100 make_request in
  match Batches.submit client ~requests () with
  | Ok batch -> Printf.printf "Submitted batch %s\n" batch.id
  | Error e -> Printf.eprintf "Error: %s\n" (string_of_error e)
Sourceval get : client -> batch_id:string -> unit -> (t, error) result

get client ~batch_id () retrieves the current status of a batch.

Use this to poll for batch completion or check processing statistics.

Example: Polls until batch completes.

  let rec wait_for_completion client batch_id =
    match Batches.get client ~batch_id () with
    | Ok batch when batch.processing_status = `Ended ->
        Printf.printf "Batch completed: %d succeeded, %d failed\n"
          batch.request_counts.succeeded batch.request_counts.errored
    | Ok _ ->
        Eio.Time.sleep env#clock 5.0;
        wait_for_completion client batch_id
    | Error e -> Printf.eprintf "Error: %s\n" (string_of_error e)
Sourceval list : client -> ?limit:int -> ?after_id:string -> unit -> (t page, error) result

list client ?limit ?after_id () retrieves a paginated list of batches.

Lists batches in reverse chronological order (newest first).

  • parameter limit

    Maximum batches per page (default: 20).

  • parameter after_id

    Pagination cursor for subsequent pages.

Sourceval cancel : client -> batch_id:string -> unit -> (t, error) result

cancel client ~batch_id () cancels an in-progress batch.

Only batches with status `In_progress can be canceled. Already-processed requests are not affected.

  • raises Api_error

    if the batch has already ended or been canceled.

Sourceval results : client -> batch_id:string -> unit -> (individual_response Eio.Stream.t, error) result

results client ~batch_id () streams individual results from a completed batch.

Results are streamed as newline-delimited JSON. The batch must have status `Ended to retrieve results.

  • raises Api_error

    if the batch hasn't completed yet.

Example: Processes batch results.

  match Batches.results client ~batch_id () with
  | Ok stream ->
      Eio.Stream.iter
        (function
          | { custom_id; result = `Succeeded response } ->
              Printf.printf "Request %s succeeded\n" custom_id
          | { custom_id; result = `Errored error } ->
              Printf.printf "Request %s failed: %s\n" custom_id
                error.message)
        stream
  | Error e -> Printf.eprintf "Error: %s\n" (string_of_error e)