package MlFront_Exec

  1. Overview
  2. Docs
package MlFront_Exec
Legend:
Page
Library
Module
Module type
Parameter
Class
Class type
Source

Module BuildInstance.ValueStoreSource

Saving and fetching values into the value store

Sourceval safe_local_values_file : values_file_local:string -> values_file_sha256:String.t -> [> `Validated of BuildCore.Io.file_object ] option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t

The "local" file is used to display errors, and is pretty printed to the screen with line and markers ... and for convenience for the user in an IDE that pretty print OBSERVER_RESULT should have a clickable file location. The value store is an OK fallback, but it won't let the user edit the original file.

But we can't trust the "local" file. Since the trace may come from a remote source, either accidentally or maliciously an entry could be '/etc/passwd'. So only let the build system use the "local" file if the file matches the SHA256 checksum.

Sourceval get_value_file : valuestore:MlFront_Core.FilePath.t -> value_id:string -> unit -> MlFront_Core.FilePath.t option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval put_value_file : valuestore:MlFront_Core.FilePath.t -> value_id:string -> string -> (bool * (string * int64)) MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval get_constant0 : valuestore_get: ('a -> MlFront_Core.FilePath.t option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t) -> value_id:'a -> unit -> string option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval get_constant : valuestore:MlFront_Core.FilePath.t -> value_id:string -> unit -> string option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval add_generic_ast0 : valuestore_put:('a -> string -> 'b) -> build_seckey:[< `SecretKey of string ] -> value_id:'a -> marshalled_ast_bytes:(unit -> Bytes.t) -> unit -> 'b

Add an AST value to the value store. If already present it is not added.

Sourceval add_generic_ast_exn : valuestore:MlFront_Core.FilePath.t -> build_seckey:[< `SecretKey of string ] -> value_id:string -> marshalled_ast_bytes:(unit -> Bytes.t) -> unit -> (bool * (string * int64)) MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval add_constant_exn : valuestore:MlFront_Core.FilePath.t -> value_id:string -> string -> unit MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t

Add a constant value to the value store. If already present it is not added.

The constant location will be "'c' || base32(value_id)".

Sourceval add_values_file_exn : valuestore:MlFront_Core.FilePath.t -> values_file_sha256:string -> BuildCore.Io.file_object -> bool MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t

Add values file to the value store. If already present it is not added.

The location will be "'v' || base32(sha256_hex(values_file))".

Sourceval add_debug_source_file_exn : valuestore:MlFront_Core.FilePath.t -> source_file_sha256:string -> BuildCore.Io.file_object -> bool MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t

Add debug source file to the value store. If already present it is not added.

The location will be "'d' || base32(sha256_hex(source_file))".

The return value is true if and only if the source file was newly added.

Sourceval get_validated_ast_bytes_exn : on_fail:exn -> input_bytes:(bytes -> int -> int -> unit option) -> input_byte:(unit -> int option) -> build_pubkey:[< `PublicKey of string ] -> unit -> bytes * Digestif.SHA256.t
Sourceval get_generic_ast0 : build_pubkey:[< `PublicKey of string ] -> valuestore_get: ('b -> MlFront_Core.FilePath.t option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t) -> value_id:'b -> unmarshal:(bytes -> 'a) -> unit -> ('a * [ `Sha256 of string ]) option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval get_generic_ast : valuestore:MlFront_Core.FilePath.t -> build_pubkey:[< `PublicKey of string ] -> value_id:string -> unmarshal:(bytes -> 'a) -> unit -> ('a * [ `Sha256 of string ]) option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval make_value_available : ?buildlogtrace:unit -> valuestore:MlFront_Core.FilePath.t -> value_id:string -> value_sha256:string -> integrity:[< `Checksum | `Existence | `None ] -> unit -> MlFront_Core.FilePath.t option BuildCore.Alacarte_6_4_test.CSuspending.t

make_value_available ~valuestore ~value_id ~integrity () gets the value out of the possibly remote value store and gives it a local path.

The integrity check can be `Checksum (check SHA-256), `Existence (check file exists) or `None (no check, assume the file is correct). If the file is not present or the check fails, the function will return None.

The `Checksum integrity check uses the constructive trace store as the source of truth for the SHA-256 checksum of values. If the file is not in the constructive trace store, or if the file is not in the constructive trace store, the `Checksum integrity check will remove the corrupt value from the value store.

Sourceval prepare_value_for_upload : valuestore:MlFront_Core.FilePath.t -> value_id:string -> unit -> MlFront_Core.FilePath.t MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t

prepare_value_for_upload ~valuestore ~value_id () a place on the local filesystem that must be used to place an object file before upload_value.

For implementations without cloud storage, this may be optimized to be the local object storage.

Sourceval upload_value : valuestore:'a -> value_id:'b -> value_sha256:'c -> MlFront_Core.FilePath.t -> unit MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval download_value : on_error: (string -> ([> `Failed | `Success of 'b ] as 'a) BuildCore.Alacarte_6_4_test.CSuspending.t) -> valuestore:MlFront_Core.FilePath.t -> download: (MlFront_Core.FilePath.t -> [ `Downloaded of [ `Sha256 of string * int64 ] | `Failed ] BuildCore.Alacarte_6_4_test.CSuspending.t) -> value_id:string -> ([> `Sha256 of string * int64 ] -> 'b) -> 'a BuildCore.Alacarte_6_4_test.CSuspending.t

download_value ~on_error ~valuestore ~download ~value_id success_value.

The object location will be "'o' || base32(value_id)".

If download return `Failed the failure must already be pending on the monad.

Sourceval add_generic_value : on_error: (string -> ((MlFront_Exec__BuildCore.Alacarte_6_4_test.StateSuspending.state * MlFront_Thunk__BuildWriters.Standard.MutableWriterAsync(MlFront_Thunk.Promises.PromiseMinimal)(MlFront_Exec__BuildCore.Alacarte_xtraces_test.EngineBackend).journal) -> ('a * MlFront_Thunk__BuildConstraints.StateMessage.t list * (MlFront_Thunk__BuildWriters.Standard.MutableWriterAsync(MlFront_Thunk.Promises.PromiseMinimal)(MlFront_Exec__BuildCore.Alacarte_xtraces_test.EngineBackend).journal_entry_id * MlFront_Thunk__BuildWriters.Standard.journal_entry_value) MlFront_Thunk.UniqueInsertionList.t) MlFront_Thunk.Promises.PromiseMinimal.t) MlFront_Thunk.Promises.PromiseMinimal.t) -> valuestore:MlFront_Core.FilePath.t -> src_file_or_dir:BuildCore.Io.file_object -> value_id:string -> staging_dir:MlFront_Core.FilePath.t -> (value_id:string -> value_sha256:string -> 'a) -> 'a BuildCore.Alacarte_6_4_test.CSuspending.t
Sourceval add_object : on_error: (string -> ((MlFront_Exec__BuildCore.Alacarte_6_4_test.StateSuspending.state * MlFront_Thunk__BuildWriters.Standard.MutableWriterAsync(MlFront_Thunk.Promises.PromiseMinimal)(MlFront_Exec__BuildCore.Alacarte_xtraces_test.EngineBackend).journal) -> (BuildCore.Alacarte_3_2_apparatus.V.t * MlFront_Thunk__BuildConstraints.StateMessage.t list * (MlFront_Thunk__BuildWriters.Standard.MutableWriterAsync(MlFront_Thunk.Promises.PromiseMinimal)(MlFront_Exec__BuildCore.Alacarte_xtraces_test.EngineBackend).journal_entry_id * MlFront_Thunk__BuildWriters.Standard.journal_entry_value) MlFront_Thunk.UniqueInsertionList.t) MlFront_Thunk.Promises.PromiseMinimal.t) MlFront_Thunk.Promises.PromiseMinimal.t) -> valuestore:MlFront_Core.FilePath.t -> src_file_or_dir:BuildCore.Io.file_object -> value_id:string -> staging_dir:MlFront_Core.FilePath.t -> object_id:MlFront_Thunk.ThunkCommand.module_version -> object_range:Fmlib_parse.Position.range -> unit -> BuildCore.Alacarte_3_2_apparatus.V.t BuildCore.Alacarte_6_4_test.CSuspending.t
Sourceval add_bundle : on_error: (string -> ((MlFront_Exec__BuildCore.Alacarte_6_4_test.StateSuspending.state * MlFront_Thunk__BuildWriters.Standard.MutableWriterAsync(MlFront_Thunk.Promises.PromiseMinimal)(MlFront_Exec__BuildCore.Alacarte_xtraces_test.EngineBackend).journal) -> (BuildCore.Alacarte_3_2_apparatus.V.t * MlFront_Thunk__BuildConstraints.StateMessage.t list * (MlFront_Thunk__BuildWriters.Standard.MutableWriterAsync(MlFront_Thunk.Promises.PromiseMinimal)(MlFront_Exec__BuildCore.Alacarte_xtraces_test.EngineBackend).journal_entry_id * MlFront_Thunk__BuildWriters.Standard.journal_entry_value) MlFront_Thunk.UniqueInsertionList.t) MlFront_Thunk.Promises.PromiseMinimal.t) MlFront_Thunk.Promises.PromiseMinimal.t) -> valuestore:MlFront_Core.FilePath.t -> src_file_or_dir:BuildCore.Io.file_object -> value_id:string -> staging_dir:MlFront_Core.FilePath.t -> bundle_id:MlFront_Thunk.ThunkCommand.module_version -> bundle_range:Fmlib_parse.Position.range -> bundle_values_canonical_id:string -> bundle_values_file_sha256:string -> bundle_values_file_local: [ `Validated of MlFront_Thunk__ThunkIo.Make(MlFront_Thunk.Promises.PromiseMinimal).file_object ] option -> unit -> BuildCore.Alacarte_3_2_apparatus.V.t BuildCore.Alacarte_6_4_test.CSuspending.t
Sourceval evict_value : on_error: (string -> unit MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t) -> valuestore:MlFront_Core.FilePath.t -> value_id:string -> unit -> unit MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval hydrate_values : ?buildlogtrace:unit -> valuestore:MlFront_Core.FilePath.t -> build_pubkey:[< `PublicKey of string ] -> value_id:string -> value_sha256:'a -> values_file_sha256:String.t -> values_file_local:string option -> MlFront_Thunk.ThunkAst.t -> BuildCore.Alacarte_3_2_apparatus.V.t option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval hydrate_traces : ?buildlogtrace:unit -> valuestore:MlFront_Core.FilePath.t -> build_pubkey:[< `PublicKey of string ] -> trace list -> trace list MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t

hydrate_traces ?buildlogtrace ~valuestore dehydrated_traces restores dehydrated_traces traces from their dehydrated, persistent form using the value store valuestore.

Sourceval dehydrate_key : valuestore:MlFront_Core.FilePath.t -> debug:(?suffix:string -> string -> string -> unit) -> BuildCore.Alacarte_3_2_apparatus.K.t -> BuildCore.Alacarte_3_2_apparatus.K.t MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval dehydrate_value_values : valuestore:MlFront_Core.FilePath.t -> build_seckey:[< `SecretKey of string ] -> debug:(?suffix:string -> string -> string -> unit) -> BuildCore.Alacarte_3_2_apparatus.K.t -> BuildCore.Alacarte_3_2_apparatus.V.values BuildCore.Alacarte_3_2_apparatus.V.persistent -> BuildCore.Alacarte_3_2_apparatus.V.t option MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t
Sourceval dehydrate_traces : ?buildlogtrace:unit -> valuestore:MlFront_Core.FilePath.t -> build_seckey:[< `SecretKey of string ] -> trace list -> trace list MlFront_Exec.BuildCore.Alacarte_xpromise_apparatus.Promise.t

dehydrate_traces ?buildlogtrace ~valuestore traces returns traces that can and should be persisted to a constructive trace log.

Any potentially large items are saved in the valuestore.