package sedlex

3.8.1 (2026-10-06)

Bug fixes

• End of input counts as one more character read, as it did before 3.8: a rule matching s, eof beats an earlier rule matching s. 3.8 let the earlier rule win, so a lexer with a rule matching the empty string before its eof rule looped forever at end of input (#223)

3.8 (2026-10-01)

New features

• Named capture groups: (r as x) binds x in the action to a Sedlexing.submatch, extracted with Sedlexing.Utf8.of_submatch, Latin1.of_submatch, Utf16.of_submatch or lexeme_of_submatch. Both branches of an or-pattern must bind the same names, and as is not allowed under Star, Plus, Opt, Rep, Compl, Sub or Intersect. Submatches are chosen leftmost-greedy within the longest match, using a tagged DFA; a boundary at a fixed offset from the start or end of the lexeme costs nothing at runtime. A submatch is only valid inside its action (#177, #178, #204, #208, #209, #212)
• Nested let .. in definitions of [%sedlex.regexp?] (#176)
• Unicode 18.0.0 (#205)
• Compl, Sub and Intersect accept Rep (c, 1 .. 1) (#201)
• Clearer errors for a bare Star, Plus, Opt, Utf8, Latin1 or Ascii and for unknown operators (#196)

Bug fixes

• Rule priority with eof: a rule matching s, eof no longer beats an earlier rule matching s (#211)
• End of input is read once: Plus eof and Star ('a' | eof) no longer loop forever at end of input, and eof, eof no longer matches (#211)
• Empty character sets (Chars "", Compl any, Sub or Intersect of coinciding classes) are a compile error instead of crashing the ppx or producing a rule that never matches (#201)
• Rule sets where every rule is nullable, such as a lone "", no longer generate invalid code (#201)
• UTF-8 decoding: the surrogate range rejected by Utf8.Helper.check_three ended at U+DF00 instead of U+DFFF (#184)
• Sedlexing.sub_lexeme and its Latin1, Utf8 and Utf16 versions raise Invalid_argument when the range is not within the lexeme, instead of returning code points of the rest of the buffer or, for a negative length, an empty string (#219)

Performance

• Faster lexeme, sub_lexeme and of_submatch in Sedlexing.Latin1, Sedlexing.Utf8 and Sedlexing.Utf16 (#219)
• The memory cells of as bindings are no longer cleared at the start of every token (#220)

Other changes

• Require OCaml 4.11, the oldest compiler still in opam-repository

3.7 (2025-10-06)

• Update to unicode 17.0.0

3.6 (2025-01-05)

• Fixed one of the ranges implementing Implement Corrigendum #1: UTF-8 Shortest Form for 4-bytes long characters (#171)

3.5 (2025-05-29)

• Implement Corrigendum #1: UTF-8 Shortest Form
• Add utf8 support for string literal (#127)

3.4 (2025-03-28)

• Make the library compatibility with ppxlib.0.36 (#166)

3.3 (2024-10-29)

• Add support for unicode 16.0.0 (#157)
• Add API for retrieving start and stop positions separately (#155)

3.2 (2023-06-28):

• Restore compatibility with OCaml 4.08
• Use Sedlexing.{Utf8,Utf16}.from_gen to initialize UTF8 (resp. UTF16) lexing buffers from string.
• Delay raising Malformed until actually reading the malformed part of the imput. (#140)
• Count lines in all cases (#130). Previously, certain functions for initiating the lexical buffer would disable lines counting.
• Check and fix invariants from Cset. The codebase was not respecting invariants documented in the Cset module which could break code relying on it. The code generated by sedlex.ppx could be affected.
• Do not rely on comments from unicode UCD files
• Add API to track position in bytes. Should be opt-in and backward compatible. (#146)

3.1:

• Fix directly nested sedlex matches (@smuenzel, PR #117, fixes: #12)
• Use explicit stdlib in generated code (@hhugo, PR #122, fixes: #115)
• Preserve location of lexbuf (@hhugo, PR #118, fixes: #19)
• Don't use gen to consume channels (@hhugo, PR #124, fixes: #45)
• New expect_test testsuite (@hhugo, PR #124)
• Properly recognize malformed truncated input (@hhugo, PR #124)
• Raise Malformed instead of Invalid_arg (@hhugo, PR #126, fixes: #91)
• Updated unicode support to 15.0.0

3.0:

• Dropped Stream api which was removed in 4.14.0 ahead of the 5.0 release.

2.6:

• Adapted to ppxlib 0.26, thanks to @pitag-ha

2.5:

• Fix exponential compilation time, thanks to @mnxn for reporting in #97 and @fangyi-zhou for fixing in #106
• Update unicode support for 14.0.0.

2.4

• Update dune support to 2.8, add auto-generated opam files.
• Optimize generated code, thanks to @bobzhang
• Update unicode version to 13.0.0

2.3

• Switch to ppxlib

2.2

• Support for OCaml 4.08

2.1

• GPR#78: Auto-generate unicode data

2.0

• GPR#70: Switch to dune, opam v2
• GPR#60: Breaking change: switch from int codepoints to Uchar.t codepoints
• GPR#59: Track lexing position

1.99.4

• GPR#47: Switch to ocaml-migrate-parsetree (contributed by Adrien Guatto)
• GPR#42: Added 'Rep' (repeat operator) (contributed by jpathy)

1.99.3

• Update to work with 4.03 (4.02 still supported)

1.99.2

• First official release of sedlex

1.99.1

• Support for new Ast_mapper registration API, follow OCaml trunk after the inclusion of the extension_point branch

1.99

• First version of sedlex. The history below refers to ulex, the ancestor or sedlex implemented with Camlp4.

1.1

• Generate (more) globally unique identifiers to avoid conflicts when open'ing another module processed by ulex (issue reported by Gerd Stolpmann)

1.0

• Update to the new Camlp4 and to ocamlbuild (release for OCaml 3.10 only), by Nicolas Pouillard.

0.8

• Really make it work with OCaml 3.09.
• Support for Utf-16.

0.7 released May 24 2005

• Bug fixes
• Update to OCaml 3.09 (currently CVS). Still works with OCaml 3.08.
• MIT-like license (used to LGPL)

0.5 release Jul. 8 2004

• Document how to use a custom implementation for lex buffers
• Update to OCaml 3.08

0.4 released Jan. 10 2004

• Bug fix (accept 1114111 as valid Unicode code point)
• Add the rollback function-

0.3 released Oct. 8 2003

• Bug fix
• Add a new predefined class for ISO identifiers

0.2 released Sep. 22 2003

• Changed the names of predefined regexp
• Fix max_code = 0x10ffff
• Lexers that changes encoding on the fly
• Documentation of the interface Ulexing

0.1 released Sep. 20 2003

• Initial release