package ocaml-emcc
Build not yet checked
Compile OCaml programs with C stubs to WebAssembly with emscripten
Install
dune-project
Dependency
Authors
Maintainers
Sources
1.0.3.tar.gz
sha256=d68b5338757a5333521fd288dfddca815201fc0cf2ee42ee2bf39e460ea9362e
sha512=42f3d09e4a98059851ef9ac81f390d97120243d87ff6986954f409f3acce01b35c54b6691d13770f55ec88399475cf0ec269ae681ba07d4a4d19c7b3b11a999c
doc/CHANGES.html
1.0.3
- Windows: the unit tests pass under Cygwin and MSYS2, as opam-repository's CI runs them: without a host
arthey skip the archives, and no longer compare with driver.sh's checksums, which they computed differently there.
1.0.2
- js_of_ocaml: a callback called with more arguments than its function takes (
forEachpasses the index and the array too) gives the function its own, and its result the others only if it is a function, as js_of_ocaml. It failed ("function signature mismatch" compiled,Stack_overflowinterpreted). - js_of_ocaml:
String(v)andv.toString()of an OCaml string given to JavaScript are the string, as with js_of_ocaml: Brr'sConsole.strprinted[object Object].
1.0.1
- Windows: the unit tests discard the errors of the commands they run through
NUL, not/dev/null, which does not exist there (they failed).
1.0.0
- Compiled mode is the default:
buildandlinkcompile the bytecode to C with c_of_ocaml (inPATH, or$OCAML_EMCC_C_OF_OCAML). The interpreter stays available with--interpreted(orinterpreted: yesinocaml-emcc.conf), less tested, and is used with OCaml before 4.14;--compiledis still accepted.tests/run.sh --interpretedruns the tests interpreted. - Compiled, c_of_ocaml checks its options (
c_of_ocaml --check) before anything is built: an option it lacks (a newer c_of_ocaml has it) or a combination it refuses (--cps-stackwith--effects disabled) fails at once, with its message. - The dune patched for the (ctypes) stanza is built again once dune is upgraded (the copy of the previous one, read-only, made it fail: bls12-381, lilv).
--jspi(jspi: yesinocaml-emcc.conf) links with JSPI (emcc's-sJSPI):mainis asynchronous, and the compiled code recurses deeper than the stack of a browser (6 000 to 15 000 nested calls) on new stacks of the engine (c_of_ocaml's segmented stack), in Chrome and Edge 137 or later, or node 24 and later (node 24's--experimental-wasm-jspiis set at startup). Without JSPI, the program says so instead of failing to instantiate. Under--wasi, the compiled code has no new stacks (it imported emscripten's JavaScript for them, which a WASI host lacks). It also replaces-sASYNCIFYfor C that suspends (emscripten_sleep, raylib's loop), keeping the tail calls, for a smaller program which may suspend under the code of plugins.--cps-stack(cps-stack: yes), compiled: the recursive functions get a version in continuation-passing style (c_of_ocaml's--cps-stack), in which a recursion deeper than the stack of a browser goes on, on the heap, in every browser and with a synchronousmain, for a program about 1.5 times bigger. With--dynlink, the plugins have it too (plugin --mainfollows the program, orplugin --cps-stack), and all have double translation. Not with--effectsother thandouble-translation. It works with-sASYNCIFY(no wasm tail calls), C suspending under a deep recursion, with a c_of_ocaml whose returns to the continuations bounce on the trampoline.- wasm32: the minor heap is 8M words (32 MB) by default instead of 1M, used as the program allocates: programs allocating much spend less time in minor collections (compiled, binary_trees 1.40 s, now 0.86 s; hamming 0.27 s, now 0.18 s; boyer_no_exc 0.51 s, now 0.38 s), for up to 28 MB more memory.
OCAMLRUNPARAM=s=...still sets it. Existing runtimes:ocaml-emcc setup --force. - OCaml 5.1 and later: the minor heaps and the pools of the major heap take their memory from malloc instead of emscripten's mmap, which cleared all of it and kept a list of the mappings: a program with
--threadsstarts some 5 ms faster with 28 MB less memory (the minor heaps of its 8 possible domains), and a larger minor heap (OCAMLRUNPARAM=s=) costs no memory until it is used. Existing runtimes:ocaml-emcc setup --force. - Compiled for wasm32, under node 24 and later:
mainruns on a large stack of its own (JSPI'sWebAssembly.promising, enabled at startup under node 24) instead of in a worker thread, 15 to 35 ms less at startup. Programs with js_of_ocaml's primitives or asyncify, whose callbacks enter the program again from node's event loop, and wasm64 ones keep the worker. --node-fs: reading a directory (Sys.readdir,Unix.readdir, readdir(3)) takes the types of its entries from node's readdir, without emscripten's lstat of each entry: ocamlc compiling 160 files spends 6 % less time (3.9 s, now 3.7 s). FIFOs and sockets have their own type instead of a regular file's, andd_inois not the inode of the entry.- Compiled, under node: the main thread compiles the module while the worker with the large stack starts (some 15 ms less at startup for a 6 MB program, ocamlc).
- Compiled, under node: a program linked with
--cps-stackor--jspiruns on the main thread, without the worker and its large stack (20 to 40 ms less at startup), unless$OCAML_EMCC_STACK_MBis set. Out of stack on the main thread is an uncaughtStack_overflowtoo. - Compiled with
-sASYNCIFY: asyncify saves the stack in 1 MB (-sASYNCIFY_STACK_SIZE=1048576, unless given) rather than 4 KB, which suspending under some hundred frames exceeded. Only-sASYNCIFYitself turns off the tail calls, not its other settings (-sASYNCIFY_IMPORTS). --no-tail-calls(tail-calls: no), compiled: without wasm tail calls (emcc's-mtail-call), for the browsers that lack them (Safari before 18.2, Firefox before 121, Chrome before 112), usually with--cps-stack. Long loops of tail calls (through closures, continuations) still run in constant stack, c_of_ocaml returning the next tail call past a few hundred nested ones to the last call which is not one.-sASYNCIFYimplies it. Its plugins follow (plugin --main, orplugin --no-tail-calls).- Compiled mode compiles the parts of the C in parallel, the biggest first,
$OCAML_EMCC_JOBSat a time (default: the number of processors). - The dune commands of several ocaml-emcc running at the same time take turns (a lock in the temporary directory) instead of failing ("Another Dune instance is currently running"): the links of a project can run in parallel.
- Compiled mode leaves to c_of_ocaml what is about the code, and keeps here what is about emscripten; it needs a c_of_ocaml with a manifest (
--manifest) andcamlc_main. The primitives the code calls and the effects it was compiled with come from c_of_ocaml's manifest instead of its C (the marker thatplugin --mainreads followed--effects, not c_of_ocaml's choice). Backtraces follow the debug events of the bytecode on c_of_ocaml's side, which warns of integer literals beyond 31 bits itself (linkonly does for the interpreter). The program starts frommain.c, and JSPI's new stacks for the segmented stack come fromshare/runtime/jspi.c, linked with--jspionly. The C functions the code of plugins calls are linked in the program by c_of_ocaml (asinandacoswere missing from thoselink --dynlinkexported). - Effect handlers with
--dynlink: a program whose own code performs effects is compiled with them (double translation) without--effects, instead of raisingEffect.Unhandled(c_of_ocaml 3456533a), andpluginrefuses a plugin performing effects for a program without them, or without--main, unless--effects disabled. --dynlinkis implied when a compiled program (not a standalone.wasm) linksEmcc_dynlink.- The C of an executable's own
(foreign_stubs ...)is linked (its primitives raisedFailure "... not implemented"). - A program with plugins exports the usual C functions of libc (
snprintf,strchr,qsort...) even when it does not call them, as the C of its plugins may (which C the program links depends on its primitives). - Interpreted,
--dynlinkloads the C stubs of bytecode plugins (OCaml'sDynlink, of a.cmanaming adllX.so), whichplugin --interpreted LIB.cma -o DIRbuilds as wasm side modules (--mainnaming an interpreted program implies it). A hostdllX.sothatDynlinkfinds first gives way to the wasm one of$OCAML_EMCC_STUBLIBSor the current directory.linknotes that Dynlink's plugins with C need--dynlink. - Plugins of a program linked with
--threadsload:plugin --mainlinks them with-pthread, for the program's shared memory, as doesplugin --threads(they failed with "could not load dynamic lib"). The callback ofEmcc_dynlink.load_urlruns aftermainin such a program (mainrunning in a pthread, the program exited first). With--jspi, the threads other than the main one load the plugins synchronously (a thread started after a plugin is loaded failed:SuspendError, emscripten's loads suspending outside of JSPI). A plugin's entry is read as data, its address in the plugin's part of the table: looked up as a function, it got a slot in the table of one thread only, and on wasm64 the next loads aborted ("table index is out of bounds"). - Domains, interpreted: the interpreter's state for its re-entries (see the
ocamlrecipe) is per thread. Shared, two domains entering the interpreter at the same time could take the other's, a rare crash (memory access out of bounds, a few runs in a thousand oftests/domains). (Sysroots built before note the change:ocaml-emcc install --force ocaml.) --threads: the memory of a thread that ends (its stack, its TLS) is freed once the thread no longer uses it. With--wasi, a detached thread (OCaml's threads and domains) freed it itself while still running on it; with emscripten's glue, the main thread could free it while the thread was still asking it to. A thread whose memory is above 2 GB (wasm32) is given back too: its address, read as negative, was that of no thread ("Cannot read properties of undefined (reading 'pthread_ptr')", then memory corruption, in programs starting and ending many domains).- js_of_ocaml with
--threads: a thread other than the main one calling JavaScript raisesFailure(the program crashed: JavaScript values belong to the main thread), the GC of another thread releases JavaScript values on the main thread, and oncemainreturned, the other threads go on between the callbacks (they waited forever,mainkept the runtime). --wasi: files can be mapped (Unix.map_file,mmap), as with emscripten's glue: a copy, written back bymsyncandmunmapwhen shared and writable (it failed with ENOSYS).--wasi, compiled: c_of_ocaml's small functions padding the module, which only V8's inlining heuristic needs, are left out (--disable small-functions, when c_of_ocaml has it): a smaller module.- js_of_ocaml, compiled: the property and method names of the program (its literals) are decoded once, which halves the cost of a property access (120 to 65 ns) or a method call (280 to 140 ns).
- The sqlite3 and openblas recipes are thread-safe (
SQLITE_THREADSAFE=1, sqlite's default, and openblas'sUSE_LOCKING): with--threads, threads using them at the same time no longer share their state unprotected. (Sysroots built before note the change:ocaml-emcc install --force sqlite3 openblas.) --node-fs, compiled:Sys.chdirchanges the current directory, which relative paths then follow (node's worker thread running the program cannot change the process's: chdir did nothing, Rocq found none of its libraries).- x86's SIMD and floating-point flags (
-msse2,-mfpmath=sse...), which a configure run on the host may add (rocq-runtime on wasm64), are dropped from C compilations unless they ask for wasm SIMD (-msimd128). build: the libraries compiled against a package rebuilt for the target are rebuilt however many rounds it takes (at most 5 before: netsnmp's async libraries, over core, failed with "inconsistent assumptions").- Packages building C with their own build (make, CMake, configure, run by a dune rule) get emcc in the wasm context under the host's names:
cc,gcc,g++,c++,clang,clang++,arandranlib. They were the host's, which refused emcc's flags (qbf, re2, cudd, hdr_histogram, cucumber, bls12-381). CMake looks for their libraries in the sysroot first (hdr_histogram's zlib; the wrappers skip directories named like the tool, emscripten's cmake/). In the host context, those compilers drop emscripten's flags, which a build script may take fromCFLAGS(bls12-381's build_blst.sh). - As with gcc, C++ stubs may use
register(clang refuses it in C++17: gbddml), and C functions declared without a prototype are no error with-Werror(cucumber's gherkin-c). - A
dll<x>.sothat a package's own build makes with emcc (stem's snowball) is replaced by a placeholder when ocamlc links an executable of the package for the target ("Unrecognized magic"). - Stub libraries a package enables for some architectures only (curve448: amd64, arm64...) are compiled from the C files of its native build: the wasm context's architecture is
none, so dune skipped them. ocamlc -configof the wasm32 toolchain saysword_size: 32andint_size: 31, the program's, and packages choosing their OCaml code by them in their dune files are built for wasm32 when they are rebuilt for the target.- Recipes:
HOST_TOOLS=1(withPATCH_OCAML=1) builds the executables that a package's build runs for the host, in a host context, when its OCaml libraries are rebuilt for the target (rocq-runtime's configure.exe, with C stubs). - The C files behind the members of a native stub archive built by CMake (
x.c.o, out of the source tree) are found. - Stubs without dune or a recipe: the C files reaching the OCaml headers through several local headers are found (orsetto), and those including them without their directory (
<mlvalues.h>, as before OCaml 3.08: labltk's). - ctypes-foreign: the functions emscripten only has as failing stubs (
getpwnam...) are found bydlsym(opasswd stopped at startup with "undefined symbol"), and emscripten's libraries are looked for in$EM_CACHEwhen it is set. The rest of the users database, which it lacks, finds no one and writes nothing (putpwent, the shadow passwords'getspnam...,lckpwdf). - The stubs of packages whose dune package is named otherwise (ocaml-systemd:
-p systemd) are built. - The OCaml libraries rebuilt for the target are built again once dune is downgraded: an older dune cannot read the newer one's
dune-packagefiles ("Version 3.24 of the dune language is not supported": re2, async, shell...). - The OCaml libraries rebuilt for the target whose C alone failed to build (re2's vendored C++ library, janestreet_cpuid's x86 stubs) are kept, their stubs being built apart (with an empty archive where their
dune-packagenames one); a library that needed rebuilding and failed is reported as such, not as one the target does not need. - Added recipes: bls12-381 and bls12-381-signature (a 32-bit size_t), codex (its 32-bit branches), eigen (Eigen 3.4.1's fix for recent clang), gdbm (dbm), hdr_histogram (musl's <endian.h>), janestreet_cpuid (its CPUID primitives fail, as in JavaScript; its flags shifted in 63 bits), libcurl (curl), libpcre (pcre), mariadb (mysql, mariadb), mm (mm.oss without OSS), ocaml_intrinsics (CRC-32C in software), optiml-transport (no GCC suffix on a floating constant), owl (no x86 cpuid; ndarray dimensions as intnat), readline, rocq-runtime (no perf counter; its OCaml rebuilt for 31-bit integers on wasm32), shared-memory-ring (memory barriers for wasm), shell (no disk quotas: ENOSYS), tuntap (no devices: ENOSYS), unionFind (identifiers for 31-bit integers: frama-c on wasm32), unisim_archisec (floating-point exceptions defined, never raised), unix-errno (Linux's errno codes), xapi-stdext-unix (no block devices: ENOTSUP; TCP options at IPPROTO_TCP), zmq (libzmq).
- Stub archives: one a package's rule makes with
cc -c -o lib<x>.a(an object, not an archive) is put in an archive, and of two copies of an archive (the rule's own and dune's) one is kept: radamsa's functions were defined twice. - Stubs compiled from the C files of a package: the headers of all the opam packages it depends on, not only directly, are found (jane-street-headers's
ocaml_utils.hfor core's), and those included with their package's directory (<posix-time/posix-time.h>: posix-mqueue). - When
dune build -xfails for a package's stubs, the archives it built are kept, and only the others are compiled from the C files of the native build, those that do not compile left out: libbinaryen (whosedllbinaryen.sofailed) no longer builds binaryen natively for an hour, and core's other stubs build without linux_ext's (sys/prctl.h). - The C compiler of the sysroot drops the options of GNU ld that wasm-ld does not know (
-Wl,--no-undefined,-Wl,-z,defs,-Wl,--as-needed,-Wl,--eh-frame-hdr): libbinaryen'sdllbinaryen.solinks. - conf-clang (the clang binary, a tool of the build: libbpf compiles its BPF programs with it) no longer builds the clang recipe (clang's libraries, an hour): libbpf fails at once, on its missing headers.
- The libc's libraries (
-lm,-lpthread...) a package asks for build no recipe: frei0r's (whose link flag is-lm) was built for blake3, efl... - ctypes-foreign: the OpenGL functions named in the program are found by
dlsym, emscripten's WebGL ones, with its OpenGL 1.x emulation (-sLEGACY_GL_EMULATION) forglBeginand the like; those emscripten lacks (display lists, feedback mode) abort when called (gl-legacy stopped at startup onglBegin). cargo buildin a dune rule of the wasm32 context builds the Rust static library forwasm32-unknown-emscripten(its standard library from rustup), with emcc, its build scripts with the host'scc, and copies it where the rule expects it (blake3's, which then stops on its vendored crates: ocaml-sys'sc_charis unsigned on wasm32, Rust's signed). Without that standard library (Debian's cargo), it says so.- The C compiler of the sysroot drops
-g, which dune gives every stub: the link strips the debugging information, and clang needed more than 3 GB for some generated C++ with it (unisim_archisec's decoders: 0.9 GB without). - POSIX message queues (
mq_open...), which emscripten's libc only declares, fail withENOSYSinstead of being undefined at link (posix-mqueue, interpreted). - A switch without ocamlfind: building the runtime failed (
Unix_error ENOENTrunningocamlfind printconf) instead of using the default findlib configuration of the switch. --owi(owi: yesinocaml-emcc.conf, for--wasi) links a module for owi's symbolic execution: WASI inside the module (no files, stdout dropped, stderr to owi'sprint_char), atomic instructions made plain ones (one thread:--threadsrefused), andocaml-emcc.owi(Emcc_owi) for symbolic values, assumptions and checks. owi has to runtry_tableandthrow, which its release does not yet. A runtime built before its recipe changed is a warning: an older one can take owi seconds to start.
0.5.1
- macOS: the symbol table of archives made by its
ar(__.SYMDEF SORTED, a BSD long name) is no longer taken for an object (the tests failed). - OCaml 4.08 to 4.11: the commands whose output ocaml-emcc reads (
opam var prefix,nm,dune describe...) are found inPATH(they were not, so the tests failed). - The stubs a recipe built before
-pthread -fPIC(zarith, lwt, mirage-crypto...) are built again when a program links them, as 0.5.0 said: with OCaml 5 they crashed at startup (memory access out of bounds) or failed to link with--threads.
0.5.0
- Threads: with
--threads(orthreads: yesinocaml-emcc.conf),Thread.createstarts a real thread and, with OCaml 5.3 or later,Domain.spawna domain running in parallel, in node, browsers (cross-origin isolated pages) and WASI hosts with wasi-threads (wasmer, wasmtime up to 46), interpreted or compiled (with c_of_ocaml's support). lwt then runs its jobs in its threads, andLwt_preemptiveworks, as does Async'sIn_thread.run(core_unix'sprctlnames threads). The C code of the runtime, recipes and stubs is now compiled with-pthread(programs without--threadsstay single-threaded) and-fPIC(for plugins, at no measured cost): existing runtimes and recipes are built again. The mpfr, libssl, libevent, pixman and fontconfig recipes are thread-safe, liblo has its server thread. - Sockets with WASI:
--wasix(orwasix: yesinocaml-emcc.conffor--wasi) makes a module for wasmer (wasmer run --net) with WASIX's sockets:Unix.socket,connect,bind,listen,accept, options, addresses, UDP,getaddrinfoandgethostbyname, and lwt's sockets, with or without threads. With--wasialone, the program accepts connections on the sockets its host preopens (wasmtime 46:-S preview2=n -S tcplisten=ADDR), the descriptors after the standard streams. - The local time zone has its file,
/etc/localtime, also under its name in/usr/share/zoneinfo, which core'sTimezone.localreads: under node the host's (and that of$TZ), in browsers one made from the browser's time zone (its offsets from 1970 to 2037). It was missing, outside--node-fs. --wasi: an absolute symbolic link is followed through the preopened directories (/etc/localtime), which WASI hosts refuse to do.--wasi: lwt works (Lwt_unix,Lwt_io):pollwith no timeout failed (emscripten's new probe,ENOSYS), as didfdatasync(imported from JavaScript) and, with node's WASI, waiting for a file.- js_of_ocaml's JavaScript primitives: programs using the libraries js_of_ocaml (
Js,Dom_html, its ppx, js_of_ocaml-lwt, Brr...) run in browsers and node, interpreted or compiled, on both targets, with the conventions of wasm_of_ocaml for JavaScript values. Callbacks keep running after the main program, JavaScript exceptions areJs_error.Exn(orFailure),Js.exportgives functions to the page, andSys_jsredirects channels to functions and mounts files given on demand. Their stubs, which only abort, are no longer linked. --js-file FILE(andjs-files:inocaml-emcc.conf): primitives of the program implemented in JavaScript, called with JavaScript values.- Plugins, in compiled mode:
ocaml-emcc plugin LIB.cma -o LIB.wasmcompiles a library to a wasm side module, which a program linked with--dynlinkloads with the new libraryocaml-emcc.dynlink(Emcc_dynlink.loadfile, orload_urlto fetch it in a web page), checking their interfaces asDynlinkdoes. A plugin links the C stubs of its library. Effect handlers with--effects double-translation, which plugins built with--mainfollow. - Added recipes: async_ssl, async_unix, bls12-381-unix, bytesrw, cairo, class_group_vdf, core_unix, cpu, eio_posix, faad, ffmpeg, freetype, frei0r, gd, glfw3, hacl_func, lacaml, lame, libev, libevent, libfontconfig, libjpeg, liblo, liblzo2, libogg, libopus, libpng, libssl, mad, memcpy, miou, mirage-block-unix, ncurses, nlopt, num, oniguruma, openblas, osdp, pixman, poll, rowex, scrypt, secp256k1, setcore, shine. zlib installs its pkg-config file and comes from its GitHub releases (zlib.net fails at times), gmp installs its C++ interface (gmpxx).
- Jane Street's libraries on wasm32: jst-config, the libraries reading its configuration (time_now, core, core_unix) and those compiled against an interface that differs there (the async stack) are rebuilt for the target, as ctypes users are. Their 64-bit code treated an
Int63as anint, where Base has anint64on wasm32 (a crash incaml_int64_divin async). The configuration is the host's but for the word size and endianness. core and async run on wasm32, interpreted or compiled, in node and with--wasi. Without threads, Async's thread pool runs its jobs synchronously (async_unix recipe), as lwt does. - dune-configurator's C tests in the wasm context read the runtime's
caml/config.h, not the host's (its-Iof the host's standard library is replaced). - OCaml 5.1 and later: the runtime is built with zstd when the switch's compiler is, and can read compressed marshalled data: the
.cmiand.cmtfiles (ocamlc.byte, merlin, odoc, toplevels failed with "Corrupted compiled interface") and compiler-libs'Compression. Existing runtimes:ocaml-emcc setup --force. - wasm32: the minor heap is 1M words (4 MB) by default instead of 256k (1 MB, half a 64-bit native program's): programs allocating much spent their time in minor collections (binary_trees compiled: 1.26 s, now 1.01 s; a list pipeline: 0.80 s, now 0.44 s), for 3 ms and 3 MB more at startup.
OCAMLRUNPARAM=s=...still sets it; wasm64 is unchanged. Existing runtimes:ocaml-emcc setup --force. --wasi: pipes are buffers in memory (Async's and lwt's wake-up pipes),fchdirworks,sendmsgandrecvmsgare no longer imported from JavaScript, and a standard stream the host reports as a socket is a FIFO (node's piped standard streams).- Recipes:
PATCH_OCAML=1rebuilds the OCaml libraries of the package of anopam:recipe for the target with its patches. These patches are also applied to the builds for the target of ctypes users and Jane Street's libraries (core_unix's C stubs), whose stamps record them. - A primitive that no linked library defines (a stub written for one OS only: epoll, Windows...) becomes a stub raising
Failure "NAME not implemented"when called, as with js_of_ocaml:linkwarns, and a program that never calls it runs.--no-missing-primitivesmakes it a link error, as before. - Compiled mode: the generated C is split in parts (
c_of_ocaml --split), compiled separately, and a part left unchanged is not compiled again. - The heap is sized with
-sINITIAL_HEAP(emscripten 3.1.55 and later) unless the user's flags size the memory: withMAXIMUM_MEMORY, the initial memory was fixed at 16 MB, which the static data alone can exceed (72 MB of embedded bytecode for core with-linkall). - The C compiler wrapper drops
-march,-mtune,-mcpu,-fopenmp(emscripten's OpenMP needs threads: the code runs serially) and the host's-I/usr/include...;-Ofastbecomes-O3 -ffast-math. ocamlc -configin the wasm context reportsarchitecture: none: stubs choosing their C code with%{architecture}take the portable one (num's bng, without amd64 assembly).- A package that does not lay C types out is rebuilt for the target only if an interface it was compiled against differs there: otherwise its build was inconsistent with the packages depending on it (frama-c's plugins). One already built is only built again if one of its dependencies was rebuilt, and one depending on no root (ctypes, jst-config) is checked without fetching its sources.
- The stubs of a dune package are built with what its opam build selects:
--releaseinstead of-p PKGwhen it builds the whole tree (frama-c's plugins, which are other packages). - Stubs found without a recipe nor dune: a C file including the OCaml headers through a local header counts (b0's
b0_stubs.h), those of the library are preferred when the package has several (bytesrw), and without the native archive the files that do not compile are left out. - Recipes:
meson_setupconfigures a meson project for the target (cross file, static libraries, the sysroot's pkg-config files). - OCaml 5: the runtime has a single domain (
Domain.spawnalready failed). It reserved at startup the minor heaps of 16 domains in wasm32 and 128 in wasm64, which emscripten'smmapallocates and clears: a hello world now starts with 16 MB of wasm memory instead of 29 MB (wasm32) and 317 MB (wasm64), and ~10 ms (wasm32) and ~130 ms (wasm64) sooner.OCAMLRUNPARAM=d=Nstill sets it (5.3 and later). Existing runtimes:ocaml-emcc setup --force. - The stubs built with dune or from a package's native build, and the packages rebuilt for ctypes, record the version of the toolchain's wrappers in their stamps: a fix of the wrappers builds them again. Those built before the pkg-config fix below had the host's C flags (ctypes-foreign's wasm64 stubs: the host's
ffi_cif, 8 bytes too small, and calls failing with "Unexpected type 0"). - The stubs whose C signature differs from their OCaml arity are found by reading the wasm objects of the libraries (
linkingsection, function types), no longer by parsing wasm-ld's warnings, whose wording could change with emsdk.linknow links once instead of twice when stubs need adapters. - What the recipe driver decided moved from bash to OCaml: which recipes to build (stamps,
DEPENDS), their sources (opam, tarballs, git, system libraries), the C stubs of opam packages (dune, native build, blind), the builds for the target of ctypes users, the patched dune and the toolchain's wrappers.share/driver.sh(1084 lines, now 339) reads recipes and runs theirbuildfunction; the recipe format and helpers are unchanged. Unit tests (dune test) cover the logic that moved. - dune-configurator runs
pkg-configfrom a temporary directory, where the sysroot's wrapper took it for the host context: the wasm stubs got the host's C flags (gslread its headers from/usr/include). The wrapper now goes byINSIDE_DUNE. - Compiled mode (experimental): with
--compiled,buildandlinkcompile the bytecode to C with c_of_ocaml instead of interpreting it. Programs run 2 to 5 times faster in node, and their.wasmis 2 to 8 times smaller. It works with OCaml 4.14 and 5, on wasm32 and wasm64, effect handlers included. The stubs get the same signature adapters as with the interpreter.tests/run.sh --compiledruns the tests in this mode. - The bytecode interpreter runs about twice as fast in node and browsers (fib: 52.7 s before, 23.9 s after, 15.8 s for the native
ocamlrun). V8 optimizes a wasm function for its next calls only, so the interpreter, called once for the whole program, stayed in the baseline tier: it now returns to its caller every 2^20 polls and is called again. Itssetjmp, which wrapped each of its calls in an exception handler, moved to that caller (patches/4.x/andpatches/5.x/). - New recipe:
raylib(raylib-ocaml). The raylib it vendors is built for raylib'sPLATFORM_WEB(emscripten's GLFW, WebGL), linked with-sASYNCIFYso that the usualwhile not (window_should_close ())loop runs in a browser (-- -sJSPIreplaces it). Test:raylib. raylib-ocaml's master after 2.3.0 builds it for the web itself: the recipe's patch for it only applies to the releases before, and now builds from fresh sources too (it ran host helpers it did not depend on). - The placeholder
dll<name>.soof a wasm stub archive defines its C functions (aborting if called): a (ctypes) generator linked against a library of its package with stubs could not start ("unknown C primitive"). - The shadow build of a package installing many
.cmifiles (raylib) was reported as failed:find | grep -q, underpipefail, failed whenfindwas killed by SIGPIPE. The same in the check for wasm archives. -cclib "-l X"is read as-lX.
0.4.0
- Recipe patches are placed by their text, not their line numbers (
share/tools/patch.awk): the old text of each hunk must occur once in the file,@@lines need no numbers, and the context only has to tell the place apart. A hunk matching no place, or several, stops the build (GNU patch applied some of ours with fuzz, at the place it guessed). Theocamlpatches lost their spare context, andpatches/5.1/andpatches/5.2+/becamepatches/5.1+/. - New recipes for packages that the default paths cannot build:
mlcuddidl(CUDD configured for the target, stubs generated by camlidl from its.idlfiles),hacl-star-raw(HACL*'s portable C with cmake, ctypes stubs printed by generators run on the host, the vectorized and Vale variants trapping) andmccs(GLPK silenced withglp_term_out: musl writes what was written to a closedstdoutonce fd 1 is back). Anocaml-luarecipe unpacks and patches the Lua it bundles, as its opam file does before dune runs (it only built from a kept build directory). Tests:cudd,hacl,mccs, andposix(posix-time2, -socket, -stat). - ctypes bindings are rebuilt for wasm64 too: emscripten's musl does not lay the libc's structs out as glibc (
struct stat'sst_sizewas glibc'sst_mtime). When a package rebuilt for the target is rebuilt again (another version), the packages depending on it are too; the builds of versions no longer installed, or made against other versions of their dependencies, are dropped first (ctypes 0.24's next to ctypes-foreign 0.22: "inconsistent assumptions"); a package depending on one that cannot be rebuilt (ctypes 0.20, built by make) keeps the host's build, andlinkwarns. A package that dune builds no library of (hacl-star-raw, built by make next to hacl-star) keeps the host's build. - A recipe built from an opam package (
SOURCE=opam:) is built again for another version of it, as dune-built stub libraries already were (recipes built before:install --force). - opam commands of the driver act on the switch being built for, not on opam's current one; the sources of a package come from its build directory or
opam source, which applies thepatches:of its opam file (not from the archive in.opam-switch/sources, which has none). - wasm64:
getaddrinfois resolved in C (compat.c): emscripten's JavaScript one throws on memory64 (BigInt(null)), so didUnix.getaddrinfo. - Stubs compiled blindly get the include directories of the package's opam dependencies (ctypes before 0.21:
ocaml_integers.h). The native build behind the archives thatdune -xbuilds with host objects is made when_build/defaultlacks them (dune -xleaves only what the target needs there). A recipe asked for a library it did not build is built again. mlgmpidl: its stubs regenerated with camlidl getMPFR_VERSION_MAJOR(apron's test failed without a kept build directory).zarith: the defines of its configure, and an emptyz_features.hbefore 1.12.- Tests:
pcre2andlz4dropped,clibskeeps checkseum and bigstringaf (the others needed nothing other tests do not cover).tests/versions.shruns tests against several versions of their package.
0.3.0
- OCaml 5.0 to 5.3: the
ocamlrecipe builds their runtime. They commit the minor heap withmmap(MAP_FIXED), which emscripten lacks ("Failed to create main domain"):patches/5.0/andpatches/5.1/commit it withmprotectas 5.2 and later do (patches/5.4+/becomespatches/5.2+/). 5.0 callspthread_getaffinity_np, which its configure only compiles (patches/5.0/), and names the otherlibs stub archives without thebytsuffix. - OCaml 5:
-pthreadis kept out of the runtime by givingPTHREAD_CFLAGS=-D_REENTRANTto configure, instead of editingMakefile.config(5.0 put it inOC_CFLAGS, which the edit missed: "no domain lock held"). - The C compiler of the wasm context drops
CPATH,C_INCLUDE_PATH,CPLUS_INCLUDE_PATHandLIBRARY_PATH: their host headers came before those of the sysroot (ctypes-foreign's stubs were compiled with the host'sffi.h: "Unexpected type 0" at the first call). - ctypes on wasm32, OCaml 5.2 and later: a library compiled against the host's ctypes (ctypes-foreign, once zstd had rebuilt ctypes) is rebuilt for wasm32 again. ocamlc quotes the files of "make inconsistent assumptions", which hid them.
- A stub library of the sysroot built from an opam package (with dune, or blindly) for another version than the installed one, after an
opam upgrade, is built again: the old one was linked (digestif 1.4's blake3 primitives undefined). - A library of the compiler missing from the runtime of the sysroot (
libthreads.a, built since) has the runtime built again, instead of failing ("libthreads.a (wasm) not found"). - ctypes on wasm32: ctypes and the packages of the program depending on it are rebuilt for wasm32 (
driver.sh shadow) into the sysroot's findlib directory, which the toolchain searches first, andbuildbuilds the bytecode against them: struct layouts,sizeofand arrays now have the sizes of wasm32, for the structs laid out at run time and for the type descriptions generated when the package is built (yaml now works on wasm32).linkanddoctorwarn when a bytecode was compiled against the host's builds, or uses a package that could not be rebuilt (packages without dune). - dune's
(ctypes)stanza (a project's, or a package's): dune links its generators natively only (ocaml/dune#13650), which the wasm context cannot. A project or package using it is built by a dune patched to link them as custom bytecode withoutocamlopt(share/tools/dune/ctypes-linkage.patch), built once per version from the sources of the switch's dune: its structs have the layouts of the target. Thebuildof a project with the stanza used to fail, and the stubs of such packages came from their native build. - The C compiler of the wasm context makes node scripts of the executables it links (
-sEXECUTABLE), for build rules to run, with the node of emscripten's configuration (memory64 for wasm64). - The
ocamlcwrapper of the toolchains no longer writes placeholderdllX.sofiles in the host's switch, next to its installed libraries. On OCaml 4.x, those ofunixandstrinlib/ocaml(inld.conf) hid the real ones: the host'socaml unix.cmaandocamlc unix.cmafailed (The external function unix_waitpid is not available). A switch wherebuildran before keeps them: delete the emptydll*.sooflib/outside thestublibsdirectories. --wasi(link,build):OUT.wasmis a WASI module, importing only WASI functions, for wasmtime, node's WASI and other WASI hosts (wasm32). emscripten's standalone libc fails to open orstata file and imports the rest of the file system from JavaScript:share/runtime/wasi.cimplements these system calls with WASI (preopened directories, a current directory), sleeping,selectandgetrusagewithpoll_oneoffand the WASI clocks,dup/dup2of an opened file by opening it again, file locks and owners as no-ops. Linked with-sPURE_WASI, the module is translated to the standard exception handling instructions (exnref) that WASI hosts implement.tests/run.sh --wasiruns the suite this way (wasmtime or node's WASI), and CI does on 4.14 and 5.5.- A standalone
OUT.wasm(with or without--wasi) has its bytecode as C arrays started bycaml_startup_code, instead of a file of emscripten's JavaScript file system, which it does not have.linklists the imports that are not WASI. - The functions of
share/runtime/*.c(compat.c,dlsym.c,wasi.c) that the program's libraries define as well are left out: the program's are linked. ocamlrecipe: on wasm32, the integer literals beyond 31 bits are wrapped bycaml_startup_codeas well; the debug events of a program started bycaml_startup_codecan come from memory (backtraces of standalone modules have their locations). A runtime built before needsocaml-emcc setup --force(linksays so).- Recipes change the sources they build with patches (
patches/next torecipe.sh) instead ofsedandperl: ocaml, libffi, liblmdb, lmdb, lwt, mirage-crypto, cryptokit, secp256k1-internal. Theocamlones apply to the sources of every version from 4.08 to 5.5 (on 5.0 to 5.3, the wasm32 build used to stop on the DATA literals). A recipe's checksum covers its patches: the "recipe changed" note shows when they change.
0.2.1
- Installed packages configured against a host library: in the wasm context of
dune -x,llvm-configis theclangrecipe's (aspkg-configalready was the sysroot's), and-lclang-cpp,-lclangX,-lLLVM-<major>build theclangrecipe, which links the LLVM libraries with-lclang-cpp. A package binding clang's C++ API, installed with opam against the host's LLVM, builds without configuration; its stubs used to be compiled against the host's LLVM headers. Recipes declare such names withLIB_PATTERNS=. Aclangrecipe built before needsocaml-emcc install --force clang. - The
mpfrrecipe installsmpfr.pc:pkg-config mpfrfailed in the wasm context, and packages asking it fell back to compiling their C files without their own flags. tests/depgraph: an installed package (mpfr throughpkg-config, zarith) whose whole dependency graph is built from the program.
0.2.0
Tried on FFI-heavy packages (mirage-crypto, lwt, cryptokit, minisat, ocaml-lua, stb_image, extunix, lz4, snappy, argon2, ocaml-expat, lmdb, glpk, fftw3, secp256k1-internal, tsdl...); fixes for what broke.
- wasm32: integer literals beyond 31 bits in the bytecode's DATA section are wrapped to 31 bits instead of aborting the program at startup (optint, hence checkseum and decompress, holds such literals in its 32-bit code).
- Node: the standard streams are the process's, as bytes (binary output and input were decoded as UTF-8, NUL bytes dropped).
- C++ stubs and libraries with exceptions (
-fwasm-exceptions). -pthreaddropped from the flags of stubs (lwt):caml_statebecame a thread-local variable there.- wasm64:
caml_ba_alloc_dimsgets its dimensions asintnat(stubs passingints corrupted the heap); libffi writes widened return values at their address (ctypes-foreign calls returningintread 0). linkshows the C functions whose calls wasm-ld turned into traps (signature mismatches between C files), it used to hide them.dune -x: the tools of the host next to the toolchain'socamlc(%{ocaml}...);pkg-configonly shows the sysroot's libraries to the wasm context; C tests see the sysroot's headers and libraries; archives holding host objects (a vendored library built bymake) are rebuilt from the native build's C files; a kept_buildof another tool is left alone; a package that produced no archive is not marked as built.- Packages without dune: the C files behind the native build's archive.
linkbuilds the libraries named by-lXin.cmafiles (custom-only libraries such as stb_image, C libraries bound with ctypes-foreign).- System recipes are also found from the
-lXof a package's.cmafiles. - New recipes: liblz4, snappy, expat, fftw3, glpk, libargon2, liblmdb, sdl2; mirage-crypto (-ec, -rng), lwt, cryptokit, secp256k1-internal, lmdb.
0.1.0
First release.
ocaml-emcc build OUT.js: a dune executable and the C stubs of its dependencies, compiled to WebAssembly with emscripten (wasm32 or wasm64).- wasm OCaml runtime rebuilt from the sources of the switch's compiler, OCaml 4.08 to 5.5.
- Step-by-step commands (
setup,dune,deps,link,doctor...) for projects without dune. - Recipes for C libraries: gmp, mpfr, zlib, sqlite3, libffi, libbz2, libpcre2-8, zstd, gsl, camlidl, mlgmpidl, apron, clang; project recipes in
emcc-recipes/. - ctypes and ctypes-foreign support.