package matrix
Module Matrix_text.SegmenterSource
The display units of a stream that arrives in pieces.
A segmenter carries what cutting a stream into units takes from one scalar to the next, the state of the boundary rules and the width of the unit in progress, so that the units do not depend on where the stream was cut.
The type for the segmentation of one stream. A segmenter is mutable and must not be used by two domains in parallel.
create m is a segmenter for the units of method m, before the first scalar.
reset s ends the unit in progress, so the next scalar starts a unit, as at the start of a text (UAX #29 GB1).
may_continue m u v is false when v starts a unit of method m after any text that ends with u, and true when v continues the unit that ends with u in some text. It reads the two scalars only. A unit may continue by a rule that looks further back than u, as two regional indicators do after an even number of them and not after an odd number, and then only add, which has seen that text, tells. It allocates nothing.
ascii_run s str ~pos ~len is the number k of bytes from pos that are complete units of one printable ASCII character (U+0020 to U+007E) each. Each of them starts a unit and is followed within the slice by another printable ASCII byte, which therefore starts the next unit. k is 0 when no such run starts at pos, in particular when the unit in progress absorbs the byte at pos, as a Prepend character does (UAX #29 GB9b). Afterwards s is as if the k scalars had been added with add, and the byte at pos + k is not added. A writer therefore places k cells at once and goes on with add, and the units do not depend on how the stream was cut. It allocates nothing.
Raises Invalid_argument if pos and len do not designate a substring of str.