module Memo::Search

Overview

Semantic search operations using USearch HNSW index

Extended Modules

Defined in:

memo/search.cr

Constant Summary

EXACT_SCAN_MAX = 2048

Prefiltered searches compare the query with each match directly when there are at most this many (else they use USearch's filtered search).

MAX_CANDIDATES = 256

Unfiltered candidates fetched per filter check, at most (or 25 per result for large limits). A filter that would need more is narrow enough that applying it before searching is cheaper: at 20K vectors of 1536 dimensions, a search for 256 neighbors takes ~4 ms, while comparing the query directly with 2,000 vectors takes ~4 ms.

Instance Method Summary

Instance Method Detail

def mark_as_read(db : DB::Database, chunk_ids : Array(Int64)) #

Mark chunks as read (increment read_count)


[View source]
def semantic(db : DB::Database, embedding : Array(Float64), service_id : Int64, usearch_index : USearch::Index, limit : Int32 = 10, min_score : Float64 = 0.7, filters : Filters | Nil = nil, detail : Symbol = :reference, sql_where : String | Nil = nil, like : Array(String) | Nil = nil, match : String | Nil = nil, include_text : Bool = true, sql_where_args : Array(DB::Any) = [] of DB::Any, track_matches : Bool = true) : Array(Result) #

Semantic search using USearch HNSW index

Returns results ranked by cosine similarity.

IMPORTANT: Must provide service_id to ensure embeddings are from same vector space.

When metadata filters are present (source_type, like, match, sql_where), see search_filtered for how they're applied. When no filters, uses direct USearch search.


[View source]