Module `Tfidf.Generic`Source

A functor for building a Tfidf search index over different types of document.

Parameters

module Uid : Uid

Sourcetype t

Sourcetype key = Uid.t

Sourcemodule Witness : sig ... end

Sourcemodule Uid : sig ... end

Sourcetype 'v uid = 'v Uid.witness

A value of type 'v uid can be used to uniquely identify documents of type 'a.

Sourcetype binding =

| KV : ('v uid * 'v) -> binding
(*
A binding is returned when searching in a heterogeneous search index.
*)

Sourcetype doc = binding

Documents are bindings.

Sourceval index : t -> uid:'doc uid -> token:string -> doc -> unit

index t doc uid indexes a given document doc in t with a unique identifier uid.

Sourceval add_document : t -> 'doc uid -> key -> 'doc -> unit

Adds a new document to the indexer

Sourceval apply : 'v uid -> default:'a -> ('v -> 'a) -> doc -> 'a

apply uid ~default fn doc runs the function fn on doc if uid identifies the types as being the same, otherwise it returns default.

Sourceval apply_exn : 'v uid -> ('v -> 'a) -> doc -> 'a

Like apply except without a default return value so it may raise Invalid_argument _.

Sourceval add_index : t -> 'doc uid -> ('doc -> string) -> unit

Adds a new index and re-indexes everything.

Sourceval add_indexes : t -> 'doc uid -> ('doc -> string) list -> unit

Same as add_index but allows you to add multiple indexes at a time before re-indexing occurs.

Sourceval search : t -> string -> binding list

search t k searches the index t using k returning the possible bindings.

val empty : 
  ?santiser:(string -> string) ->
  ?strategy:(string -> string list) ->
  ?tokeniser:(string -> string list) ->
  unit ->
  t

Create a new empty search index.

parameter sanitiser
Run on each token to normalise them, by default this is String.lowercase_ascii

parameter strategy
The indexing strategy, by default this is a prefixing strategy such that abc is indexed with a, ab and abc

Sourceval pp : Format.formatter -> t -> unit

Dumps the search index.