package wire
Binary wire format DSL with EverParse 3D output
Install
dune-project
Dependency
Authors
Maintainers
Sources
wire-1.1.0.tbz
sha256=b1310eabd9a945b38b3b4dc62f91986fd9d54ee2a006a65c8dcf8c36a61563ba
sha512=60a73a9f045079223b812863aea92ac7d3bffdc17b878ff9940301a93fb2f0f86f9c1a17999fdcd9f6647e13d0f53aef958e8d59c1ca42f84645e8809b39f1ce
doc/CHANGES.html
1.1.0
Added
Wire.Expr.land64is an int64 bitwise AND for masking a full-widthuint64field inside a constraint, and~self_int64/Field.int64now accept amap-decoded uint64 field. A sign-magnitude offset bound such as a bsdiff seek (land64 self mask <= max) is now expressible and projects to a 3D refinement EverParse verifies (#227, @samoht)
Changed
- The standalone
c/archive (3dmode) now builds and installs in every dune context through that context's own toolchain, so a cross build produces and installs a target-native verified parser instead of skipping the C. The build uses%{ocaml-config:c_compiler}, theocaml-configpartial linker, and the binutils that compiler resolves (-print-prog-name), so a crossccfinds the target'sobjcopy/ar.EverParseEndianness.hgains a freestanding__BYTE_ORDER__/__builtin_bswap*branch so it compiles for an OS-less target (a unikernel defines neither__linux__nor__APPLE__). Only C regeneration and theagreedifferential test stay host-gated (#231, @samoht) uint32/uint32benow decode toOptint.tanduint63/uint63betoOptint.Int63.trather than a nativeint, so a value with bit 31 (or the high half) set is no longer silently truncated on a target whoseintis narrower than 63 bits (js_of_ocaml, wasm_of_ocaml); a TCP sequence number now round-trips there. Read such a field withOptint.to_int/Optint.to_int32; auint32used as a size parameter must becomeint32be(#226, @samoht)- Decode errors are redesigned.
parse_erroris now a{ at; field; kind }record instead of a flat variant:atis the failing field's byte offset,fieldthe root-to-leaf path of field names to it, andkinda closederror_kind(Unexpected_eof,Invalid_enum,Invalid_tag,Missing_terminator,Non_zero_padding,Value_out_of_range,Constraint_failed), so a failure deep in a nested struct is both locatable and matchable.Constraint_failed of stringbecomesConstraint_failed { which; value }, naming the predicate that failed and carrying the offending field's value.Validation_erroris removed (Codec.validateand the_exndecoders raise the singleParse_error), and the type gainsequal_parse_error/compare_parse_error,pp_error_kind, and publicparse_error/eofconstructors. Match one.kind, reade.at, replaceValidation_errorwithParse_error, and wrapWire.parse_errorin your own variant for domain errors (#219, @samoht) - Decoding a record codec with 17 to 32 fields no longer allocates a short-lived closure on each decode: the closure-free decode ceiling that 1.0.0 raised to 16 fields now extends to 32. Real protocol headers cross the old limit routinely, such as an 18-field TCP header with its flag bits broken out, or telemetry and IPv4 headers in the high twenties, which now allocate only the record they return. A codec wider than 32 fields keeps the recursive fallback (#214, @samoht)
Fixed
- A standalone
Wire_3darchive no longer exposes its raw<Base>Validate*validators to C callers. Those entrypoints skip aStartPosition <= InputLengthbound check, so a direct caller passingStartPosition > InputLengthunderflowed the read span and went out of bounds. The install now ships only the checked<Base>Wrapper.hheader (not<Base>.h), and the archive build localizes every symbol except the<Base>Check<Codec>wrappers, so the raw validators are neither declared nor linkable; the wrapper validates from position 0 and is the safe public API (#228, #229, @samoht) - Codec values are now safe to share across domains: their validation scratch and parameter backing are per-domain, so encoding, decoding, or validating one codec concurrently no longer corrupts the result (#223, #224, @samoht)
- A generated FFI parser (
parse buf off) now raisesInvalid_argumentwhenoffis negative or past the end ofbuf, instead of reading out of bounds. The C stub computed its length asBytes.length buf - offin unsigned arithmetic, so an out-of-rangeoff(which can carry a length or offset field parsed from untrusted input) underflowed the length into a huge span and pushed the read pointer past the buffer before the validator ran. An in-rangeoffis unaffected (#222, @samoht) - A non-zero byte in an
all_zerospadding field now reports the same error whether the field is decoded through a struct (Codec.decode/Codec.validate) or directly (Wire.of_string). The struct path used to report a stringlyConstraint_failedthat dropped the byte offset while the direct path reported the typedAll_zeros_failedwith the offset; both now reportAll_zeros_failedcarrying the offset (#220, @samoht) - A
casetypewhose discriminant matches no case now fails with the typedInvalid_tagcarrying the tag value, the same class as an out-of-range lookup index, instead of a stringlyConstraint_failed(#220, @samoht) - A non-integer field referenced where an integer is required (a schema mistake, such as a float used as a length) now raises
Invalid_argumentrather than a parse error, keeping schema errors distinct from malformed input (#220, @samoht)
1.0.0
Added
Field.actionreturns a field's action (the?actionpassed toField.v), completing the field accessor set alongsideField.constraint_andField.doc.Action.ppandParam.pppretty-print an action block and a parameter, matching the existingField.pp(#189, @samoht)Wire.enum_open name cases baseis an open enumeration: it names the known values for documentation but accepts any value. UnlikeWire.enum/Wire.variants, it does not reject an unlisted value (no decodeInvalid_enum, and the field projects as its base scalar with no membership refinement), which is what an open value set (a protocol field that may carry unknown or future codes) needs. The known codes are still emitted as a 3D enum declaration, so they stay documented in the generated.3d(#166, @samoht)- The doc pipeline's differential
agree.cis now derived from the codecs alone: it computes each validator's name and parameter types from the Wire definitions instead of reading the EverParse-generated<Name>Wrapper.h. The self-check therefore regenerates as pure OCaml, with no EverParse needed and no chance of drifting from the codec, and its build rule is split out from the committed C.Wire.Everparse.Raw.input_param_c_typesexposes the per-parameter C types this relies on (#167, @samoht) Wire.Expr.if_then_else cond t ebuilds a conditional value expression (the 3D? :ternary), so a size or constraint can depend on another field, e.g. a 16-bit length where 0 means 65536:if_then_else Expr.(len = int 0) (int 65536) len. The underlying constructor was previously reachable only through the wrapped internal module (#164, @samoht)- The doc pipeline's differential harness now covers parameterized codecs: the corpus oracle binds each codec's
Param.inputvalues and the generatedagree.cpasses the same values to the EverParse validator, so a length-bound or otherwise parameter-dependent frame (e.g. a CCSDS TC / AOS / TM / USLP layout) is checked end to end rather than skipped.Wire.Paramgainsbind_by_name, which binds an input parameter by name without its typed handle (#163, @samoht) Wire.Field.vtakes an optional?self_int64, andWire.Field.int64/Wire.Expr.int64build full-width 64-bit field constraints. This lets schemas constrain domains such as signed-magnitudeuint64values without truncating the field through OCaml's nativeint(@samoht)Wire.Field.vtakes an optional?doc(read back withWire.Field.doc): a free-text note, such as an RFC section, that the documentation projection renders as a/* ... */comment above the field in the generated 3D. A protocol spec can now cite the standard each individual field comes from, not just the struct as a whole, and EverParse accepts the comment (#157, @samoht)- The standalone projection pipeline (
Wire_3d.main ~mode:Standalone) now auto-generates a differential self-check:dune runtestfuzzes inputs, records whether the OCaml codec accepts each, and replays them through the EverParse-generated C validator, failing on any input the two decide differently. This catches a doc projection that drifts from the codec (a wrong bit order, a constraint that means something else over the wire type), which nothing checked before since the doc validator carries no FFI. The build also produces an installedlib.a archive of the validator. NewWire_3d.generate_corpusandWire_3d.generate_agree` expose the two halves (@samoht) Wire.Codec.vtakes an optional?doc(read back withWire.Codec.doc): a free-text note, such as an RFC citation, that the documentation projection renders as a/*++ ... --*/comment on the codec's 3D typedef. The generated spec then documents which standard each protocol struct comes from, and EverParse accepts the comment (#155, @samoht)Wire_3d's documentation helpers (generate_doc,generate_dune_doc, andmain ~mode:Standalone) take an optional?namethat sets the generated.3d /.c file base independently of the opam~package, so a package likeocaml-tcpcan emit aTcp.3d` spec while still installing under its own name (#154, @samoht)Wire.Everparse.project ~mode:StandaloneandWire.Everparse.writeproject a codec, or a whole family of codecs, to a clean.3dwith no FFI scaffolding: enums render as named 3D enum types, types shared across codecs are emitted once, and a protocol family lands in one readable.3d`. The result doubles as a protocol specification and as input to EverParse, which compiles it to a standalone verified C parser with no FFI (#151, @samoht) Wire.Codec.renamereturns a codec with a new name, leaving its wire encoding and field constraints unchanged, so a generically built codec can be given a unique, meaningful name before code generation (@samoht)Wire.nested/Wire.nested_at_mostnow accept a composite inner (aWire.array, or another nested region), and aWire.casetypefield's case body may be such a region; both round-trip and generate a verified EverParse validator (#109, @samoht)- An embedded sub-codec (
Wire.codec cused as a field orField.repeat/arrayelement) that takesParam.inputparameters now works: the outer codec exposes the sub-codec's input params as its own, soCodec.env/Param.bindreach them and the values are threaded into the sub-codec on encode, decode, and projection (#108, @samoht) - A
Wire.casetypeused as aField.repeatelement may now have a bitfield case body, alongside the scalar, byte-span, NUL-terminated, and sub-codec bodies already allowed (#105, @samoht) Field.optionalandField.optional_ornow accept a variable-size inner, so an optional field can be a length-prefixed string or a whole sub-message, not just a fixed-width value, and generate a verified EverParse validator (#88, #133, @samoht)Wire.zerotermandWire.zeroterm_at_most ~sizefor NUL-terminated strings: the bytes up to a terminator, optionally bounded to a fixed-size region (#77, @samoht)Wire.casetypenow accepts any tag type ('k typ, not justint), so a string-discriminated union (abyte_array-tagged casetype, as in many SSH messages) is expressible (#49, @samoht)- A
Wire.casetypecase body can be an embedded sub-codec, including one that ends inall_bytes(#50, @samoht) Field.repeatnow supports variable-size elements, e.g. a list of length-prefixed sub-messages (#51, @samoht)Wire.Codec.size_of_value: the encoded byte length of a value (#58, @samoht)Wire.casetypeandWire.nested ~sizecan be used asCodecfields (#47, @samoht)- Add
Field.optional/Field.optional_or/Field.repeat/Field.repeat_seq(#46, @samoht) - Add
Wire.rest_bytesfor trailing "rest of buffer" fields, plus directall_bytes/all_zerossupport asCodecfields (#44, @samoht) - Add signed integers
int8/int16(be)/int32(be)/int64(be)(#42, @samoht) - Add IEEE 754 floats
float32(be)/float64(be)andWire.is_finite/Wire.is_nanpredicates (#42, @samoht) - Add
Wire.byte_array_where ~size ~per_bytefor byte spans with a per-byte refinement (#40, @samoht) - Add
Codec.validator_of_struct/validate_struct/struct_size_of/struct_min_size(#37, @samoht) - Add
Codec.slice_offset/Codec.slice_length(#37, @samoht) - Add
Wire.codectype alias for'r Codec.tandWire.pp_value(#39, @samoht)
Changed
- Encoding a variable-length byte field (
byte_slice,byte_array,zeroterm,all_bytes, ...) no longer allocates: on a flambda-off switch the writer rebuilt two short-lived closures per field on every encode. Encoding a record made of scalars and var-bytes fields is now allocation-free (#213, @samoht) - Decoding a record codec with more than 8 fields no longer allocates a short-lived closure on each decode: the constructor is now applied in one saturated call for codecs of up to 16 fields. On a flambda-off switch this partial application was the dominant cost of decoding a wide header (CLCW at 13 fields, CFDP at 14), which now allocates only the record it returns (#212, @samoht)
- The EverParse projection API is consolidated to two entry points. The two projections, previously
Wire.Everparse.schemaandWire.Everparse.doc, are nowWire.Everparse.project ?mode, where`Standalone(the default) emits a clean.3dthat EverParse compiles to a standalone verified C parser (the production output, which also reads as a spec) and`Ffiemits the OCaml-callable bridge withWireCtxextern callbacks. Writing is likewise oneWire.Everparse.write ?mode(replacingwrite_3dandwrite_doc). The struct-level entry pointsstruct_of_codecandschema_of_structmove underWire.Everparse.Raw(asstruct_of_codecandproject_struct), andWire_3d.main's mode is now`Standalonerather than`Doc(#210, @samoht) - A long
?docnote on a field or codec (an RFC citation, say) now wraps across several comment lines in the generated.3dinstead of rendering as one line past 80 columns, so the generated spec stays readable (#191, @samoht) - A package generated by the
Wire_3dpipeline now treats its EverParse C (<Name>.c, the wrappers,EverParse.h) as committed source. A plaindune buildordune testuses the committed C and never invokes3d.exe, and fails loudly if the C was never committed; setBUILD_EVERPARSE=1to regenerate it (the rule then promotes the result back into the tree). Previously the rule regenerated whenever the C was missing, which silently required EverParse on an ordinary build and let committed C drift unnoticed. The pure-OCaml.3dandagree.care unaffected and still regenerate on demand (#168, @samoht) Wire_3d.mainnow takes packed codecs (Wire_3d.pack codec) and a mandatory~mode:[Ffi |Standalone], so everygen.mlstates what it emits.`Ffikeeps the per-codec FFI parsers;`Standaloneemits one FFI-free<Package>.3dspecification and a single standalone<Package>.cparser for the whole package, through the newWire_3d.generate_docandWire_3d.generate_dune_doc. Migrate agen.mlby replacing[schema c; ...]with~mode:Ffi [pack c; ...], or~mode:Standalonefor the single-file output (#152, @samoht)- Reading or writing a
uint32oruint63field now stays in the nativeintinstead of round-tripping through a boxedInt32orInt64. The boxing surfaced as per-field allocation in tight decode and encode loops; field access is now allocation-free regardless of how the compiler optimises the surrounding code. Pure speedup, no API change (#150, @samoht) Wire.Codec.decodeno longer allocates a fresh validation buffer on every call: each codec reuses a single buffer across decodes, so decoding the same codec in a loop allocates a constant amount instead of growing with the number of decodes. Pure speedup, no API change (#149, @samoht)- Decoding a
Field.repeatover aWire.casetype(the DHCP / TCP option-list shape) no longer allocates a closure and a boxed length per element, so decode allocation no longer grows with the number of elements. Pure speedup, no API change (#148, @samoht) - The build rules
wire.3dgenerates (thedune.incand the wrapper in its setup example) now use a3dalias instead of the genericgen, sodune build @3dregenerates the.3dfiles and EverParse C parsers. Update anydune build @geninvocation accordingly (#146, @samoht) Wire.of_readernow rewinds on failure: every byte consumed by a failed decode is pushed back, restoring the reader to its position before the call, so the caller can retry with another description or after more input arrives (#145, @samoht)Wire.of_readernow consumes only the bytes of the decoded value and leaves the rest on the reader, so several values can be decoded back-to-back from the same reader. Previously the first call drained the whole reader. Types that extend to the end of input (all_bytes,all_zeros) still consume the whole stream (#144, @samoht)- The
Wire.Everparse.plug_fieldrecord fields lose theirpf_prefix (pf_nameis nowname,pf_idxisidx, and so on). Update custom plug generators accordingly (#144, @samoht) - Codecs that share a synthesised type (an
enum, or a refined-byte or element-wrapper struct) can now be linked into one binary, so full protocol stacks built from per-codec parsers (Ethernet, IPv4, TCP, ...) link cleanly. TheWire_stubs-generated FFI now compiles each codec's C as its own unit and links them; use the newWire_stubs.build_codec_archiveto build the link archive (#135, @samoht) Field.repeatandWire.arrayover aWire.casetypenow raiseInvalid_argumentat construction when a case body has no per-element projection (a nested region, array, or optional), instead of building a codec that fails later at decode (#105, @samoht)Wire.default(a casetype's default branch) no longer takes a fixed~tag; instead it threads the matched discriminator throughinjectandproject, so an arbitrary unclaimed tag round-trips.injectis now'k -> 'w -> 'a(it receives the matched tag along with the body) andprojectis now'a -> ('k * 'w) option(it returns the tag to write back), so a default branch can recover and re-emit the tag it caught (e.g. a DHCP or TCP option code). Migrate by taking the tag ininject(fun _tag body -> ...to ignore it) and pairing it inproject(#100, @samoht)- Decoding a struct with variable-size fields (
byte_slice,byte_array, or arepeatsized by a cross-field expression) no longer allocates on each field access. Pure speedup, no API change (#81, @samoht) - Remove
Wire.optional/Wire.optional_or/Wire.repeat/Wire.repeat_seqfrom the typ-level surface; use the matchingField.*combinators instead (#46, @samoht) - Rename
Wire.decode_*/Wire.encode_*toof_string/of_bytes/of_reader/to_string/to_bytes/to_writer; add_exntwins that raise on parse error (#39, @samoht) - Fold
Codec.decode_withintoCodec.decodevia?env; split intoCodec.decode(result) andCodec.decode_exn(raises) (#39, @samoht) - Speed up
Codec.bitfield~5% (#37, @samoht) - Drop the
pe_prefix onTypes.param_envfields: they are nowcodec_id,slots,bound. Callers usingParam.bind/Param.getare unaffected; code that pattern-matches the record or reads fields directly needs to update (#63, @samoht) Codec.raw_encodenow returns the offset after the written bytes instead ofunit.Codec.encodeis unaffected (#65, @samoht)
Documentation
- Add odoc cross-reference links and the doc comments that were missing across the public interfaces, and expose a
ppprinter onUInt32,UInt63,Param, andWire.Diff(#73, @samoht) - Type-check
README.mdand every public.mliundermdx(#39, @samoht)
Fixed
- The generated C
<Name>Check<Codec>wrapper now validates the whole buffer: it returnsFALSEunless the validator consumed every input byte, so a valid record followed by trailing bytes is rejected instead of accepted as a valid prefix. The differential corpus oracle applies the same whole-buffer rule, so the OCaml and C verdicts still agree. The raw<Name>Validate<Codec>entry point keeps its prefix semantics and returns the consumed position.wirenow depends onreat runtime, previously a test-only dependency (#215, @samoht) Wire.Everparse.projectnow rejects a codec that cannot project to 3D when the schema is built, not later when it is rendered. A constraint with no 3D projection (afield_pos, or a subtraction or multiplication over a field) used to build a schema that only raised once passed toto_3d, so the projection was not a reliable projectability check (#209, @samoht)Codec.validatenow enforces every checkCodec.decodedoes, for any codec. It used to skip a field's own decode-side checks (enum and variant membership, a lookup index bound, a refined or NUL-terminated span, an embedded codec or array element's constraints) whenever the struct had no top-level constraint orwhere, so validating untrusted input accepted values decode rejects and a following zero-copyCodec.gettrusted them. A codec with nothing to check (a header of plain scalars and byte spans) still validates without allocating, so validating before a batch ofgetcalls on a hot path stays cheap (#207, #211, @samoht)- A field constraint that subtracts or multiplies a field (such as
Expr.(a * b = int 0)orExpr.(a - b >= int 5)) is now rejected at projection. Such an expression under- or overflows the field's narrow width, which EverParse refuses to verify, leaving the codec with no validator; unlike addition (widened to avoid overflow), neither has a sound projection. A constantSub/Muland additive field arithmetic are unaffected (#206, @samoht) Codec.validatenow enforces anall_zerospadding field, rejecting a non-zero byte exactly asCodec.decodedoes. The zero check lived only in the decode reader, so validating a frame with tampered padding succeeded and a following zero-copy read accepted it (#205, @samoht)Codec.validatenow runs decode's structural bounds check for every codec, including one with no field constraints orwhere. It used to be a no-op for a constraint-free codec, so validating a truncated buffer succeeded and a following zero-copyCodec.getread out of bounds; the documented safety gate now rejects a short buffer (#204, @samoht)- A signed field equality constraint (an
int8field whose~self_constraintisExpr.(s = int k)) now projects to the two's-complement byte the unsigned validator reads, and folds to a constant when the target is outside the signed range. The raw constant was compared against the unsigned projection, so the generated C andCodec.decodedisagreed on the same byte (#203, @samoht) - A
byte_slicewhose resolved size is negative, for example aSubon a length field that underflows on crafted input, now fails with aCodecparse error instead of escapingCodec.decodewith a rawInvalid_argument. The slice read skipped the bounds check the other byte spans run and crashed (#202, @samoht) - A
Wire.casetypewhose tag is auint ~sizevalue, or an enum over a big-endian base, is now rejected at construction. Neither projects to a 3D type the dispatch can name, so the codec built without a verified validator. A fixed-width integer, bitfield, or little-endian / 1-byte enum tag is unaffected (#201, @samoht) - A non-trivial
Wire.whereused as aWire.casetypecase body is now rejected at construction. Such a refinement projects tocase k: T { cond } v;, which is not valid 3D (a case body takes no refinement), unlike a top-level fieldwhere, so the codec had no verified validator (#200, @samoht) Wire.bits ~widthis now validated against its base word: a width above the base size (e.g.bits ~width:9 U8) or below 1 is rejected at construction. Such a field had no faithful wire meaning, and the OCaml shift and the EverParse-generated validator read different values from the same bytes (#199, @samoht)Field.repeatover a zero-width element (Wire.empty) is now rejected at construction, likeWire.arrayalready was. A byte-budget list of a 0-width element does not extract through EverParse, so the codec had no verified C validator; the error now fires when the schema is built rather than producing an unverifiable spec (#198, @samoht)- A closed
enum(orvariants) used as anarrayelement, arepeatelement, or anoptionalinner now enforces its value set in the EverParse-generated C validator, matchingCodec.decode. Previously only a byte-wide enum array element carried the membership check; a wider or big-endian array element, a repeat element of any width, and an optional inner accepted codes outside the named set that the OCaml decoder rejects withInvalid_enum, so the two disagreed on the same input (#197, @samoht) - The documentation pipeline's differential
agree.cnow predicts each EverParse validator-wrapper symbol withWire_3d.pascal_case, a transcription of EverParse's ownpascal_casemangling, computed aspascal_case (module ^ "_check_" ^ codec). The old code gluedeverparse_name-normalized parts, which kept a capital after a digit (TPM2BgaveTpmCheckTpm2Bwhere EverParse emitsTpmCheckTpm2b), soagree.creferenced a symbol that did not exist and failed to compile under-Werror. Regenerate a package'sagree.cto pick this up (@samoht) - The
dune.incrules generated by theWire_3dpipeline now invoke the generator through%{exe:gen.exe}and use plain dune actions instead of shell(system ...)commands. The old rules ran./gen.exe, which relied on dune placing agen.exesymlink in the action's working directory; under sandboxing that is not guaranteed, so the build could fail withgen.exe: not found. The corpus, the differentialagreedriver, and the validator archive are now built as ordinary targets and run directly. Regenerate a package'sdune.inc(dune build @<pkg>/c/3d) to pick this up (@samoht) - A codec whose name has a lower-case segment after an underscore (such as
Grpc_message) now generates C identifiers that match the ones EverParse emits.Wire_3d.everparse_namecapitalized only the first segment, yieldingGrpcmessagewhere EverParse producesGrpcMessage, so the generated FFI stubs and the documentation differential harness referenced a name that did not exist and failed to link. Every underscore-separated segment is now capitalized (@samoht) - A field constraint that adds narrow unsigned fields (such as
a + b <= 10over twouint8fields) now projects to a.3dEverParse verifies. The sum was emitted at the field's own width, which EverParse refuses to verify because it can overflow, whileWire.Codec.decodecomputes the same sum in OCaml's wide native int. The addition's operands are now widened to 64-bit so the generated C validator and the decoder agree (#188, @samoht) - An
arrayof an openenum(Wire.enum_open) now validates identically in the OCaml decoder and the EverParse-generated C validator: both accept any element value. The C validator used to reject element values outside the named codes (it constrained each element to the enum's named set), whileWire.Codec.decodeaccepted them, so the two disagreed on which buffers are valid (#187, @samoht) - An
enum/variantsover a big-endian base (e.g.enum ... uint16be) now projects to a.3dEverParse accepts. It was emitted as aUINT16BE enumdeclaration, which EverParse rejects (it types the integer constants as the native width: "Expected UINT16BE, got UINT16"). A big-endian-based enum now projects as its base scalar with a membership refinement (closed) or bare base (open), with no enum declaration (#185, @samoht) Wire.array/Wire.array_seqnow reject a zero-width element (empty/ unit) at construction. Such an array carries no bytes and projected to a zero-size 3D array EverParse rejects; it is a degenerate shape and is refused up front (#184, @samoht)Wire.Codec.vnow rejects, at construction, a non-last field that is aWire.casetypewith a case body ending in a greedy field (all_bytes/all_zeros). If that case is selected the greedy tail consumes the rest of the buffer, starving the following field, so the record failed to round-trip while construction silently accepted it. The greedy-must-be-last check now looks through casetype case bodies, as it already does through an embedded sub-codec (#183, @samoht)Wire.Codec.validateon a buffer too short to hold the fields a check reads now fails cleanly instead of raisingInvalid_argument. Awhereor field constraint may read a field whose offset depends on a length read from the buffer;decodebounds-checks the buffer first, butvalidateran the check kernel directly, so a short buffer read out of bounds and crashed (#181, @samoht)- A zero-length
Wire.byte_slicenow decodes to an empty slice instead of raisingInvalid_argument. The slice constructor rejects a zero length, so abyte_slicewhose size resolved to 0 crashed the decoder rather than yielding the empty slice (#180, @samoht) Wire.Codec.vnow rejects, at construction, a non-last field whose type is an embedded sub-codec ending in a greedy field (all_bytes/all_zeros). Such a tail consumes the rest of the buffer with no boundary, so it silently swallowed the following field's bytes and the record failed to decode. The greedy-must-be- last check already rejected a bare greedy field that is not last; it now looks through an embedded sub-codec too (#179, @samoht)Wire.of_string(and the other typ-level entry points) now return a cleanErroron a truncated input to a variable-size codec, instead of raisingInvalid_argument. Computing the codec's span reads its length and gate fields up front; on a buffer too short to hold them, that read ran off the end and escaped as an out-of-bounds exception rather than a parse error.Codec.decodealready guarded this; the typ-level path now does too (#178, @samoht)Wire.of_string(and the other typ-level entry points) now accept an unlisted code in aWire.enum_openfield, matchingWire.Codec.decode. The typ-level decoder kept the closed-enum membership check regardless of theenum_openflag, so it raisedInvalid_enumon a value the codec accepts, an open enum that behaved as closed on that path (#177, @samoht)Wire.Codec.decodenow rejects a parametric codec whose env is missing or leaves an input param unbound, raisingInvalid_argument(naming the param) the wayCodec.encodealready does. Decoding without binding a parameter used to resolve a parameter-driven field size to 0 and silently truncate the field; the binding precondition is now enforced up front on both sides (#176, @samoht)Wire.Param.bind_by_namenow drives a parameter-dependent field size on decode, not onlywhereclauses and constraints. A field whose size comes from a parameter (abyte_array,byte_slice, oruint_varsized byParam.expr) read as zero bytes when its parameter was bound by name, silently truncating the field and misaligning everything after it; only the typedParam.bindworked. Both binders now resolve parametric sizes identically (#175, @samoht)- A signed integer field's ordering constraint (e.g.
int8 xwithx < 100) now projects soundly. A signed field becomes an unsignedUINT*in 3D, so the refinement was emitted as an unsigned comparison and the verified C validator disagreed with the OCaml decoder on bytes whose sign bit is set (byte 200 is the signed value -56: accepted by OCaml, rejected by C). The ordering is now rewritten to its two's-complement unsigned form. A float field ordering constraint, which has no faithful unsigned projection (IEEE bit patterns do not order as unsigned), is rejected when the codec is projected (#174, @samoht) - A
Wire.whereplaced on a container element (an array or repeat element, or an optional inner) is now rejected at codec construction with a clear error. EverParse cannot express a refinement on an array or optional element that references an outer field, so such awhereproduced a codec whose generated.3ddid not compile while OCaml decode silently ignored the constraint. Awhereis supported as a top-level field refinement; move the constraint onto the field itself or a codec~where(#173, @samoht) - The doc pipeline's differential self-check (
agree.c) now links for a codec whose name has interior consecutive capitals. EverParse normalizes such a name in the validator symbol (SpaceOSFrame -> SpaceOsframe), but the harness built the symbol from the raw name, so the generated check called an undeclared function. The name now goes through the same normalization, which also collapses a consecutive-capital run anywhere in a name, not only at the start (#171, @samoht) - The doc pipeline's differential self-check (
agree.c) no longer false-reports a mismatch for a codec with a large payload. The generated reader held each corpus line in a buffer one char short of two hex digits per input byte, so an 8 KB payload (16384 hex chars) truncated the line and misparsed the verdict. The buffer is now sized to the input width (#172, @samoht) Wire.Codec.decodeandWire.Codec.validatenow enforce a constraint written asWire.where cond ton a field, and any field~action. Such awherewas projected into the generated.3d(so the EverParse C validator rejected violating input) but was silently dropped on the OCaml side, so OCaml accepted what the verified C rejects; andCodec.validateskipped field actions thatdecoderan, so the two disagreed. Decode and validate now share a single validation path and enforce identical semantics (#169, @samoht)- A
Wire.casetypethat switches on aWire.enumtag now projects to a 3D schema EverParse accepts: each case label is emitted as the enum constant name (case InteriorIndex:) instead of the raw integer (case 2:), which EverParse rejected as not a member of the enumerated type (#162, @samoht) Wire.Expr.( = )andWire.Expr.( <> )are explicitly re-exported from the expression language, so equality in a localExpr.(...)open buildsEq/Neconstraints rather than depending on the surrounding equality binding (@samoht)- A codec whose name contains a capital
V(e.g.VeritySuperblock) now generates its C parser. EverParse names the validator<Name>Validate<Name>, and the name reader stopped at the firstV, so C generation failed for any such name (#143, @samoht) - A
Wire.enumfield now enforces its membership in the EverParse-generated C validator, rejecting values outside the named cases exactly asCodec.decodedoes (raisingInvalid_enum), including for an enum nested inside a sub-codec or record. Previously the verified C accepted out-of-range values the OCaml decoder rejects (#131, @samoht) - Decoding a
Wire.enumthrough theCodecAPI now rejects a value that is not one of the named cases (raisingInvalid_enum), on a scalar field and on every array or repeat element, matching the EverParse validator and theWire.of_stringpath. TheCodecdecoder previously accepted any value (#130, @samoht) - A
Wire.lookupfield now enforces its index bound in the EverParse-generated C validator, on a scalar field and on everyWire.array/Field.repeatelement, so the validator rejects out-of-range indices exactly as the OCaml decoder does. Previously the verified C accepted indices the decoder rejects (#126, @samoht) - A
Wire.nested/nested_at_mostfield, and aWire.byte_array_wherespan inside one, now generate a verified EverParse validator. Previously any codec with such a field failed schema generation (#99, #132, @samoht) Wire.rest_bytesnow generates a verified EverParse validator. Previously any codec with arest_bytesfield failed schema generation regardless of width (#117, @samoht)- A codec with a
Wire.uint63/uint63befield now generates a verified EverParse validator (projecting to the 8-byteUINT64). Previously schema generation failed and the codec had no verified C parser at all (#125, @samoht) - A codec mixing signed-integer or float fields of different widths (e.g. a
float32then afloat64, or anint8then anint32) now generates a verified EverParse validator. Previously such a codec had no verified C parser (#127, @samoht) - A
Wire.enumused as aWire.array/Field.repeatelement (or inside an optional or sized region) now generates a verified EverParse validator. Previously such a codec had no verified C parser (#128, @samoht) - A statically-absent
Field.optional/Field.optional_or(~present:false) now generates a verified EverParse validator. Previously the codec had no validator at all (@samoht) - An
Action.on_successending in a conditionalAction.return_bool(anAction.if_with areturnbranch), and anAction.on_actwhose body ends inAction.return_bool, now generate a verified EverParse validator (@samoht) - Projecting an expression (a
~where/ field constraint /~self_constraint) that uses a construct with no projectable form (a negative integer literal orfield_pos) now raises a clearInvalid_argumentinstead of emitting C that EverParse rejects with a cryptic error. Every other operator (shifts, bitwise, casts, mod, div, comparisons,sizeof,sizeof_this) projects (@samoht) - The generated dune rule now compiles the EverParse C under strict C11 (
-std=c11 -D_DEFAULT_SOURCE) instead of-std=c99, so the verified validators build on Linux glibc (the BSD endian helpers the C uses need_DEFAULT_SOURCE) (@samoht) - Decoding no longer raises
Invalid_argumenton adversarial input where aField.repeatbyte budget, or a variable field's cross-field size, exceeds the buffer; an oversized length now fails with a cleanParse_error(#117, @samoht) - A cross-field length / offset /
presentexpression that reads an integer beyond the native int range (auint64/int64length overmax_int), or reads a non-integer field, now fails the parse instead of silently reading 0. The old behaviour masked malformed input (#82, @samoht) - A
byte_array/byte_slice(or any field) whose~sizereads aField.optional_orfield no longer resolves that size to 0, decoding the span as empty (silent truncation) and raising a length mismatch onCodec.encode; it now reads the present-or-default value (#101, @samoht) - A greedy field (
all_bytes/all_zeros) reads the rest of the buffer, so it is now rejected withInvalid_argumentanywhere it is not the final field: a non-last field of a codec, aField.repeat/Wire.arrayelement (or a sub-codec ending in one), or aWire.casetypecase body. It remains valid as the last field, the supported way to consume the rest (#107, #110, #111, @samoht) - An embedded sub-codec's
whereclause and field constraints are now enforced when the codec is decoded as a field or element. They were silently dropped on the embedded path, so a value the sub-codec would reject standalone was accepted when embedded (#108, @samoht) Wire.array/array_seqnow reject a non-fixed-width element (aWire.nestedregion, aWire.byte_array_whererefined span, or a nestedWire.array) at construction withInvalid_argument. An element must be a scalar, a fixed byte span, or a fixed-size sub-codec, matchingField.repeat(#107, @samoht)Wire.array/array_seq/Field.repeat/repeat_seqover a sub-codec built only from byte-span fields (byte_array,byte_slice, a varint) now raiseInvalid_argumentat construction. A sub-codec with at least one fixed-size field is accepted as before (#115, @samoht)Wire.array/array_seqover a float, a signed integer, auint63, or aWire.where/Wire.mapwrapping a fixed byte span no longer raise during 3D projection; all fixed-width scalars and wrapped byte spans now project (#116, @samoht)Field.repeat/repeat_seqnow reject an element type that has no clean per-element projection (a sub-bytebitsfield, a refined or at-most byte span,all_zeros, or a nestedarray/nested) at construction with a clearInvalid_argument. Supported elements are unchanged: fixed-width scalars and byte spans,zeroterm, sub-codecs, and casetypes (#97, @samoht)- A bitfield (
Wire.bits/Wire.bit) is now rejected withInvalid_argumentat construction as an element ofField.repeat/Wire.array/Wire.nestedor as aField.optionalinner: a bitfield only exists packed inside a record, with no standalone wire form (#90, #98, #107, @samoht) - A
Wire.casetypewhose case body is a NUL-terminated string (zerotermorzeroterm_at_most) now encodes, decodes, and sizes correctly as aField.repeatelement, so a list of such tag-dispatched options round-trips (#103, @samoht) Field.repeatover aWire.casetypeelement now encodes and decodes instead of raising. This covers DHCP-style options whose cases mix bare single-byte tags with length-prefixed bodies (#75, @samoht)Field.repeat/Wire.arrayover a fixedbyte_array/byte_sliceelement (a list of n-byte chunks, e.g. fixed-size addresses) now encode, decode, and generate a verified EverParse validator; decoding previously raisedFailure(#89, #92, @samoht)Field.repeatover azerotermelement (a list of NUL-terminated strings within a byte budget) now encodes, decodes, and generates a verified EverParse validator; it previously raisedFailurewhen decoding (#93, @samoht)Wire.arrayover a fixed-size sub-record (aWire.codecelement, e.g. an array of{ x; y }points) now decodes instead of raisingFailure(#96, @samoht)- A
Field.optional_orwith a dynamic gate now generates an EverParse C validator that accepts the bytesCodec.encodeproduces; the two previously disagreed on the field's layout when the gate was false (#88, @samoht) - A codec that embeds a variable-size sub-codec (
Wire.codec, e.g. a length-prefixed string) as a field is now accepted by EverParse; it previously failed schema generation (#87, @samoht) - A variable-size sub-codec or
Field.repeatmay now follow a variable-size field (#38, @samoht) Codec.encodeno longer requires an?envfor a codec whose only parameters are decode-side outputs (a field with anAction.assigninto aParam.output). Output params are never read when encoding, so demanding an env raisedInvalid_argumentspuriously, and an output-param sub-codec embedded as a field could not be encoded at all (#95, @samoht)Codec.size_of_valuenow sizes aField.repeatwith a dynamic budget, aWire.casetypefield, and a packed bitfield (wrapped byWire.bitor an enum / map) correctly. The first two were under-counted (soCodec.encodeoverran the buffer) and the bitfield over-counted (soencoderaised a spuriousInvalid_argument) (#72, #78, #79, @samoht)Codec.encodenow raisesInvalid_argumentwhen the writer emits fewer bytes thansize_of_valuepromised, instead of shipping a value with uninitialised trailing bytes (#62, @samoht)Codec.encodeinto a too-small buffer now fails with a precise byte count instead of writing past the end (#61, @samoht)Field.optional/Field.optional_orwith a dynamic gate now encode from the value (the gate selects the decoded value or the default on decode);optionalraisesInvalid_argumentrather than writing a phantom byte or overrunning the buffer when the gate and value disagree (#58, @samoht)Field.optional/Field.optional_orpredicates that use bitwise / shift / mod operators are no longer silently treated as always-true, andField.refon anoptionalfield now reads the decoded value instead of 0 (#48, @samoht)- Decoding an
all_zerosfield that contains a non-zero byte now returns aConstraint_failederror instead of raising (@samoht) Wire.to_stringon aWire.nested ~size:nfield now zero-pads tonbytes when the inner writes fewer, so it agrees withWire.of_string(@samoht)Wire.to_stringon acodecfield whose inner ends inall_bytes/rest_bytes/all_zerosno longer appends a 4 KB scratch tail; the size is computed from the value (#54, @samoht)Codec.encode/Codec.raw_encodeaccept?env:Param.env, likeCodec.decode. Encoding a parametric codec with a missing param binding now raisesInvalid_argumentnaming the param instead of writing zero-sized regions (#53, @samoht)- Fix C stub generation for schema names with two or more leading capitals (e.g.
IPv4,EP_Header) (#36, @samoht)
0.9.0
Initial release.