OCaml Multicore - August 2021
Multicore OCaml: August 2021
Welcome to the August 2021 Multicore OCaml monthly report! The following update and the previous updates have been compiled by me, @ctk21, @kayceesrk and @shakthimaan. This month's update is a bit quieter as August is also a period of downtime in Europe (and our crew in India also took well-deserved time off), but we all participated in the online OCaml Workshop which was held virtually this year, so there are plenty of videos to watch!
The multicore effort is all on track for integration into OCaml 5.0 early next year, with the core team currently organising the upstreaming code review strategy over the coming winter months. Meanwhile, there are some blog posts and videos from the OCaml Workshop which give more detailed updates on both domains-parallelism and effects.
- Adapting the OCaml ecosystem for Multicore OCaml
- This talk covers how our community can adapt to the forthcoming OCaml 5.0 with parallelism.
- Blog post 1, Blog post 2
- Parafuzz coverage guided Property Fuzzing for Multicore OCaml programs
- We develop ParaFuzz, an input and concurrency fuzzing tool for Multicore OCaml programs. ParaFuzz builds on top of Crowbar which combines AFL-based grey box fuzzing with QuickCheck and extends it to handle parallelism.
- Experiences with Effects
- The multicore branch of OCaml adds support for effect handlers. In this talk, we report our experiences with effects, both from converting existing code, and from writing new code.
As always, the Multicore OCaml updates are listed first, which are then followed by the updates from the Ecosystem libraries and Sandmark benchmarking.
ocaml-multicore/ocaml-multicore#632 Str module multi domain safety
An issue on stdlib safety in the OCaml
Strmodule to work concurrently with Multicore OCaml.
ocaml-multicore/ocaml-multicore#633 Error building 4.12.0+domains with no-flat-float-arrays
A linker error observed by
Adrián Montesinos González(
debugnik) when installing 4.12.0+domains
ocaml-multicore/ocaml-multicore#634 Strange type errors from merlin (This expression has type string/1)
Type errors reported from merlin (4.3.1-412) when using the effects version of the Multicore OCaml compiler.
ocaml-multicore/ocaml-multicore#624 core v0.14: test triggers a segfault in the GC
The root cause of the segfault when running
core.v0.14test suite with Multicore OCaml 4.12.0+domains has been identified.
ocaml-multicore/ocaml-multicore#573 Backport trunk safepoints PR to multicore
This is an on-going effort to backport the Safepoints implementation to Multicore OCaml.
ocaml-multicore/parallel-programming-in-multicore-ocaml#10 Edited for flow/syntax/consistency
The Parallel Programming in Multicore OCaml chapter has been updated for consistency, syntax flow and grammar.
ocaml-multicore/retro-httpaf-bench#15 Optimise Go code
nethttp-go/httpserv.gobenchmark has been optimised with use of
fmt.Fprintf, and the removal of yield().
poll_addtest hangs on s390x
The use of
poll_addcauses a hang on
s390xarchitecture. A backtrace with GDB is provided for reference:
(gdb) bt #0 0x000003ffb63ec01e in __GI___libc_write (nbytes=<optimized out>, buf=<optimized out>, fd=<optimized out>) at ../sysdeps/unix/sysv/linux/write.c:26 #1 __GI___libc_write (fd=<optimized out>, buf=0x3ffffdee8e0, nbytes=1) at ../sysdeps/unix/sysv/linux/write.c:24 #2 0x000002aa0dbb0ca2 in unix_write (fd=<optimized out>, buf=<optimized out>, vofs=<optimized out>, vlen=<optimized out>) at write.c:44 #3 0x000002aa0dbd4d3a in caml_c_call ()
@UnixJunkie has provided a simplified version for the interface for scientific parallel programming as recommended by the parany library.
val run: ?csize:int -> ~nprocs: int -> demux:(unit -> 'a) -> work:('a -> 'b) -> mux:('b -> unit) -> unit
ocaml-multicore/domainslib#39 Add a fast path in parallel scan
A patch that performs a sequential scan when the number of elements is less than or equal to the pool size or if the number of domains is one.
ocaml-multicore/domainslib#40 Parallel map
A PR that implements
parallel_mapin lib/task.ml that includes an optional chunk size parameter.
ocaml-multicore/retro-httpaf-bench#13 Update EIO for performance improvements, multiple domains
httpf-eiohas been enhanced with performance improvements when running with multiple domains. The results on an 8-core VM with 100 connections and 5 second runs is shown below:
The following illustration is from a VM for 1000 connections and 60 second runs:
The results with
GOMAXPROCS=3for three OCaml domains is as follows:
ocaml-multicore/ocaml-uring#36 Update to cstruct 6.0.1
Cstruct.shiftvand has been updated to use cstruct.6.0.1.
ocaml-multicore/domainslib#41 Use the master branch in the link to usage examples
The README.md file has been updated to point to the sample programs in the master branch that use the new
The Multicore OCaml concurrency bug detection tool named ParaFuzz is now available in GitHub as Free/Libre and Open Source Software.
Tezos is a proof-of-stake distributed consensus platform designed to evolve, and is written in OCaml. The version of the Tezos daemon that now runs on Multicore OCaml is also available in GitHub as a work-in-progress fork.
eio library provides an effects-based parallel IO stack for
ocaml-multicore/eio#68 Add eio_luv backend
We now use
luv, which has OCaml/Reason bindings to libuv, to provide a cross-platform default backend for eio.
ocaml-multicore/eio#72 Add non-deterministic to abstract domain socket test
The inclusion of
non-deterministic=commandto disable a regular
dune runtestfor the failing abstract domain socket test.
ocaml-multicore/eio#73 Work-around for
io_uringbug reading from terminals
A work-around to fix
io_uring_enterto block the entire process when reading from a terminal.
ocaml-multicore/eio#74 Don't crash when receiving a signal
A patch to receive a signal and not crash in
ocaml-multicore/eio#75 Add Eio.Stream
Streammodule has been added to Eio that implements bounded queues with cancellation.
ocaml-multicore/eio#76 Link to some eio examples
The README.md has been updated to point to existing eio example project sources.
ocaml-multicore/eio#77 Disable opam file generation due to dune bug
The opam file generation with dune.2.9.0 is broken as dune does not have the
subst --rootoption. Hence, the same is now disabled in the eio build steps.
ocaml-multicore/eio#79 Initial edits for consistency, formatting and clarity
Changes in the README.md file for consistency, syntax formatting and for clarity.
Update dependencies to work with 4.14.0+trunk
A series of patches that update the dependencies in Sandmark to build 4.14.0+trunk with dune.2.9.0.
We are continuing to integrate and test building of 4.12.0 OCaml variants with Sandmark-2.0 with
current-benchfor both sequential and parallel benchmarks.
Our thanks to all the OCaml users, developers and contributors in the community for their valuable time and support to the project. Stay safe!
- GC: Garbage Collector
- GDB: GNU Project Debugger
- HTTP: Hypertext Transfer Protocol
- ICFP: International Conference on Functional Programming
- IO: Input/Output
- PR: Pull Request
- OPAM: OCaml Package Manager
- VM: Virtual Machine