class Memo::Providers::HTTPPool

• Memo::Providers::HTTPPool
• Reference
• Object

Overview

Posts requests to an embedding API over reused connections, retrying transient failures.

HTTP::Client handles one request at a time, so idle clients are kept in a small pool: concurrent requests each take one (opening a new connection only when none is idle) and return it afterwards. A reused connection skips the TCP and TLS handshakes.

Retries rate limits (429, except an exhausted quota), server errors (500, 502, 503, 504) and network errors, up to #max_retries times. Waits as the Retry-After header says, or backs off exponentially (0.5 s, 1 s, 2 s, ... with jitter).

Defined in:

memo/providers/http_pool.cr

Constant Summary

MAX_IDLE = 8

MAX_RETRY_WAIT = 20.seconds

RETRY_STATUSES = {429, 500, 502, 503, 504}

Constructors

• .new(base_url : String, max_retries : Int32 = 3)

Instance Method Summary

• #close : Nil

  Close idle connections.

• #max_retries : Int32
• #post(path : String, headers : HTTP::Headers, body : String) : HTTP::Client::Response

  POST to path (e.g.

Constructor Detail

Instance Method Detail