package index
Build not yet checked sectionYPositions = computeSectionYPositions($el), 10)"
x-init="setTimeout(() => sectionYPositions = computeSectionYPositions($el), 10)"
>
On This Page
A platform-agnostic multi-level index for OCaml
Install
dune-project
Dependency
Authors
Maintainers
Sources
index-1.7.0.tbz
sha256=7874a6b68888ca3af1cc252262a1bbad46388b78ecfa9e20109d874d28c02cb2
sha512=c4e9baf0cadd1fe6d8b675e841bb8e1e5401d987b69e94efd62185d5f669d37da56bd94e65e455a8f82de0468cb8a48d314668dde9eabe53ec0e7263d4dd33bb
doc/CHANGES.html
1.7.0 (2026-11-13)
Added
- Add
Stats.mergeandStats.merge_in, to combine the statistics recorded by several domains (#404, @lyrm) - Expose the Unix platform as
Index_unix.Private.Platform, so thatIndex.Makecan be applied to it directly (#396, @art-w) - Document the file format of
Index_unix.Private.Raw(#380, @tomjridge, @balat)
Changed
- Breaking: OCaml 5.0 or later is now required (#404, @lyrm)
- Breaking: statistics are now recorded per domain, in a
Domain.DLSkey, instead of in a global value.Stats.getandStats.reset_statsonly see the calling domain (#404, @lyrm) - Mirage support:
indexno longer depends onunix, andindex.unixis now an optional library (#396, @art-w) - Breaking:
Platform.SandChecks.Platform_argsnow require aProgressand aFmt_ttymodule, andPrivate.Logs.setupandPrivate.Logs.setup_termtake an extra(module Platform.FMT_TTY)argument (#396, @art-w) - Support cmdliner 2.0.0. Cmdliner 1.1.0 or later is now required (#382, @MisterDA, #402, @art-w)
Fixed
- Lock files are now opened with the
O_CLOEXECflag (#394, @vect0r-vicall) - Read-only instances could return stale values after
sync.syncnow removes from their LRU the keys of the entries it loads, from the log or from the log used during merges. It also holds the lock thatfindtakes, so that a concurrentfindcannot cache an outdated value in the middle of async(#398, #399, @art-w, #406, @balat) - Merges could lose entries, and
itercould give wrong keys, when values are encoded on zero bytes and the key decoder returns its input string without copying it, asRepr.decode_bindoes (#387, @Ngoguey42) - Concurrent
finds on distinct instances of the sameIndex.Makeapplication, from several threads or domains, could miss bindings: they shared the buffer in which they prefetch entries of the data file. Each instance now has its own buffer, created withPrivate.Io_array.prefetch_bufferand passed toPrivate.Io_array.v(#406, @balat) Index_unix.Private.Raw: reading a truncated header now raisesNot_written, as for a missing header, instead of failing an assertion. Writing a version that is not 8 bytes long raisesInvalid_argument, and a write that makes no progress raisesFailureinstead of being dropped (#406, @balat)- Add the missing
logs.threadeddependency of the Unix tests (#400, @anmonteiro)
1.6.2 (2023-06-06)
Changed
- Update for compatibility with
mtime.2.0.0, which is now the lower bounds (#392, @patricoferris)
1.6.1 (2022-06-08)
Added
- Support all version of cmdliner (#386)
Fixed
- Make
index-bench'sbenchexecutable private, avoiding executable name collisions in opam-monorepo projects. (#389, @NathanReb)
1.6.0 (2022-02-12)
Added
- Added a
Raw.Header_prefixfunction, for use by libraries that share the file format used byindex.unix. (#378)
1.5.0 (2021-11-09)
Changed
- The benchmarks now use
tezos-base58instead oftezos-context-hash(#367) - Add an LRU to cache the result of
Index.findoperations. The default LRU capacity is 30_000 entries. (#366)
1.4.2 (2021-10-15)
Fixed
- Fix stats recording in
Raw.unsafe_write(#351)
Changed
- Changed the implementation of the write-ahead log to significantly reduce its memory usage (at the cost of some additional disk IO). (#355)
1.4.1 (2021-07-16)
Fixed
- Proper cleaning of merge file descriptors when aborting a merge (#326)
- Recover from crash of the merge thread. When this happen, the main thread can continue to run and will need to recover from the crash before doing a new merge. This fixes a critical issue which might cause data loss (#339)
- Make sure that no entries can disappear for read-only instances during log_async recovery (#338)
1.4.0 (2021-06-16)
Fixed
- Fixed a crash-consistency bug due to a potential flush of an incomplete entry to disk. Entries are now flushed as complete strings. (#301)
- Attempt to recover from
log_asyncinvariant violations during an explicit sync operation, rather than failing immediately. (#329)
Changed
- Release overly defensive warnings occuring when pre-fetching the disk. (#322)
- Optimised the in-memory representation of index handles and intermediate buffers, resulting in a significant reduction in memory use. (#273, #279)
- Benches are now executed 3 times and a new option
nb-exechas been added (#292) Index.Makenow requires an implementation of a monotonic time source. (#321)- The
Index.Makefunctor now takes a singlePlatformargument containing all system dependencies (i.e.IO,Clock,SemaphoreandThread). ThePlatformmodule holds the necessary types for these modules. (#321, #330)
Added
- Added benchmarks that replay a trace of index operations. (#300)
- Log reporter for the benches
1.3.1 (2021-04-29)
Fixed
- Reduce allocations during merge (#274, #277)
- Protect concurrent syncs with a lock (#309)
- Fixed a performance issue for
Index.syncwhen there is a blocking merge in progress: thelog_asyncfile was not cached properly and fully reloaded from disk every time. (#310) - Release the merge lock if a merge raises an exception (#312)
- Added fsync after
Index.clearto signal more quickly to read-only instances than something has changed in the file (#308)
Changed
- Specialise
IO.vto create read-only or read-write instances. (#291) clearremoves the files on disks and opens new ones containing only the header. (#288, #307, #317)
1.3.0 (2021-01-05)
Added
- Added
flush_callbackparameter to the creation of a store, to register a callback before a flush. This callback can be temporarily disabled by~no_callback:()toflush. (#189, #216) - Added
Stats.merge_durationsto list the duration of the last 10 merges. (#193) - Added
is_mergingto detect if a merge is running. (#192) - New
IO.Header.{get,set}functions to read and write the file headers atomically (#175, #204, @icristescu, @CraigFe, @samoht) - Added a
throttleconfiguration option to select the strategy to use when the cache are full and an async merge is already in progress. The current behavior is the (default)`Block_writesstrategy. The new`Overcommit_memorydoes not block but continue to fill the cache instead. (#209, @samoht) - Add
IO.existsobligation for IO implementations, to be used for lazy creation of IO instances. (#233, @CraigFe) Index.closenow takes an~immediately:()argument. When passed, this causescloseto terminate any ongoing asynchronous merge operation, rather than waiting for it to finish. (#185, #234)- Added
Index.Checks.cli, which provides offline integrity checking of Index stores. (#236) Index.replacenow takes a~overcommitargument to postpone a merge. (#253)Index.mergeis now part of the public API. (#253)Index.try_mergeis now part of the public API.try_merge' is a no-op if the number of entries in the write-ahead log is smaller thanlog_size, otherwise it'smerge'. (#253 @samoht)
Changed
synchas to be called by the read-only instance to synchronise with the files on disk. (#175)- Caching of
Indexinstances is now explicit:Index.Makerequires a cache implementation, andIndex.vmay be passed a cache to be used for instance sharing. The default behaviour is not to share instances. (#188)
Fixed
- Added values after a clear are found by read-only instances. (#168)
- Fix a race between
mergeandsync(#203, @samoht, @CraigFe) - Fix a potential loss of data if a crash occurs at the end of a merge (#232)
- Fix
Index.iterto only iterate once over elements persisted on the disk (#260, @samoht, @icristescu)
1.2.1 (2020-06-24)
Added
- Added
Index_unix.Syscalls, a module exposing various Unix bindings for interacting with file-systems. (#176)
Fixed
- Fail when
Index_unix.IOfile version number is not as expected. (#178) - Fixed creation of an index when an empty
datafile exists. (#173)
1.2.0 (2020-02-25)
Added
- Added
filter, removing bindings depending on a predicate (#165)
Changed
- Parameterise
Index.Makeover arbitrary mutex and thread implementations (and remove the obligation forIOto provide this functionality). (#160, #161)
1.1.0 (2019-12-21)
Changed
- Improve the cooperativeness of the
mergeoperation, allowing concurrent read operations to share CPU resources with ongoing merges. (#152) - Improve speed of read operations for read-only instances. (#141)
Removed
- Remove
force_mergefromIndex.S, due to difficulties with guaranteeing sensible semantics to this function under MRSW access patterns. (#147, #150)
1.0.1 (2019-11-29)
Added
- Provide a better CLI interface for the benchmarks (#130, #133)
Fixed
- Fix a segmentation fault when using musl <= 1.1.20 by not allocating 64k-byte buffers on the thread stack (#132)
- Do not call
pwritewithlen=0(#131) - Clear
log.memonclose(#135) - Load
log_asyncon startup (#136)
1.0.0 (2019-11-14)
First stable release.
sectionYPositions = computeSectionYPositions($el), 10)"
x-init="setTimeout(() => sectionYPositions = computeSectionYPositions($el), 10)"
>
On This Page