As Core is built on top of Base and Core_kernel you might want to have a look at Base's changelog and Core_kernel's changelog.
Time.Ofday.of_string
supports "AM and "PM" suffixes.- Change the innards of
Command.Flag
so it no longer uses mutable state. It now updates a Univ_map.t
instead. Renamed:
core/test --> core/test-bin
since these directories contain executables to run rather than libraries with standard unit tests. This is in preparation for moving the standard unit tests to a more "normal" test directory.
- Move unit tests from core/src to core/test .
- The output for the "version" subcommand and the "-version" flag is now the same.
Core.Std.Lock_file doesn't work properly on OSX, see this:
https://github.com/janestreet/jenga/issues/4#issuecomment-205176593
This fix the issue by requiring both flock and lockf only on Linux.
Closes janestreet/jenga#4
- Unix.fork_exec needs to handle exceptions raised by exec
- Remove
Core.Core_list
, which was not in Core.Std
and had only an unused to_sequence
. - Array.random_element
Shift operations are unspecified outside of the range 0 <= x < bitsize
.
This fixes unit tests for core/src/core_unix.ml in 32bit.
Replace all occurences (in Core) of Time_ns.Span.to_int_ns
and Time_ns.to_int_ns_since_epoch
with their 63bit counterpart.
Time_ns.Span.to_int_ns
and Time_ns.to_int_ns_since_epoch
are not implemented in 32bit.
The is one step toward have unit tests to pass in 32bit.
- Wrap Unix.strptime in Date and Time
New arg types in Command.Param:
sexp = Arg_type.create Sexp.of_string
sexp_conv a_of_sexp = Arg_type.create (fun s -> a_of_sexp (Sexp.of_string s))
Time_ns.Ofday.create
presently makes 2 function calls to caml_int_compare
. Changed to use the native int comparisons and avoid the calls in create
(and other comparisons).
N.B. Some interface change were made which are not listed here, as they are only cascading from core_kernel
. Look at core_kernel's CHANGES.md file to get a complete history of the changes made for this release.
- Add
Core.Command.Arg_type.Export.time_zone
Fix Command.shape
to run external programs only once to get their sexp.
Introduces a new variant of Command.t called Proxy. The Exec variant represents the top-level command of an external executable that has not yet been run; the Proxy variant represents an arbitrary subcommand that has been extracted by running an external executable. Command.exec constructs an Exec variant; Command.shape of an Exec variant runs the executable and generates a tree of Proxy variants representing all the information from the generated sexp, so the executable will not need to be re-run.
A version of recvmmsg that pre-allocates and reuses the iovec record. Profiling indicates this is a non-trivial amount of our I/O loop (under very heavy load).
Since nobody is using the heavyweight features of the existing recvmmsg, replace it with the lightweight one. This leads to minor but important changes to the interfaces of Iobuf.recvmmsg_assume_fd_nonblocking
and Udp.recvmmsg_loop
.
- Switch to ppx.
Time.set_sexp_zone
affects t_of_sexp
If we're willing to read sexps without zones, we should be willing to let you control what timezone they're read with.
- Sped up
Llimiter
. - Make
Interval.Int
implement Container
and Binary_searchable
. - Add
Identifiable.S
to Unix.Cidr
so that it supports hash tables. Update Unix.Cidr.t
to normalize values, e.g. "192.168.1.101/24" ==> "192.168.1.0/24". - Added "noalloc" attribute to
Linux_ext.unsafe_timerfd_settime
. In Iobuf, made some functions take:
(> write
, _) Iobuf.t
rather than:
(read_write, _) Iobuf.t
if they only need to write to the iobuf and don't need to read it.
Time.of_string_abs
didn't support ISO 8601 time zone strings without colons, or those specified as locations. This version adds support for time zones without colonsUnix.Passwd.getpwents
takes the lock, partially applies Exn.protect
, then releases the lock, then completes the application and actually runs stuff.Command.file-completion
Files are now completed correctly when paths contain a directory.
Previously, completion when pointed at a directory would put a space at the end. This would cause the user to hit backspace every time a directory was in the path.
- Add
diff_weekdays
and diff_weekend_days
functions to date module. Time.to_date_ofday_precise
implements a complete inverse for of_date_ofday
. This is needed to give the DWIM-est semantics to Schedule.t that we can think of.Reduce allocation in Linux_ext.Epoll
- Add
Time_ns.Of_day.(add_exn, sub_exn, diff)
. - Adding
head_padded_fixed_string
to Iobuf. - Move
Core_extended.Std.Sys.home
to Core.Std.Sys.home_directory
. - Add
Iobuf.{read,write,input,output}
akin to the bigstring versions. - Add expert iobuf functions for extracting bigstrings and iovecs that share the iobuf's underlying storage.
- Add stable
Int63.t
conversions for types in Time_ns
. - Rename DNS-based
Inet_addr
sexp conversions to expose their blocking nature. - Add
Unix.Inet_addr.Stable
. - Add [Core.Std.Schedule].
Fixed Iobuf_packet.iter
, which behaved incorrectly if the packet didn't start at the lo_min of the iobuf. It used Iobuf.rewind
when it should have used Iobuf.Lo_bound.restore
.
Added @@deriving compare
to Iobuf.Bound
.
- Add ability to
Time.format
for a specific time zone Make more information available via Command.Shape.t
Expose machine-readable info on anonymous arguments.
- Remove unnecessary rebinding of
(^/)
in core_filename.ml
. It had one call site and wasn't exposed. The (^/)
everyone uses comes from std.ml Add getifaddrs
to Core.Std.Unix
.
Handles Packet (for interfaces that do not have an address on Linux systems only), IPv4 and IPv6 address families.
- Implement
Time_ns.Span.to_string_hum
by analogy to Time.Span.to_string_hum
. The code and tests are essentially copied from "lib/core/src/span.ml". Remove our stubs for Unix.stat
.
They were upstreamed in 4.02.2.
Tweaked Unix.stat
's C code to reduce float rounding error, by using double-precision rather than single-precision floats.
Following Xavier's comment: http://caml.inria.fr/mantis/view.php?id=6285
- Added
Date.O
module. - In
Interval
, exposed compare
in stable types by having the appropriate modules match the Stable
signature. - Added
Iobuf.Fill.decimal
and Poke.decimal
, for efficiently writing integers in decimal format. - Removed
Filename.O
(and /^
), since it's not used that much, and is inconsistent with the older operator for this that's exposed in Core.Std
: ^/
. Improved Command
autocompletion to work even if some arguments can't be parsed.
This is useful because the completion mode does not get fed precisely the same arguments that it would get if you hit RETURN.
As a simple example, if completion is set up for my-exe
which takes a sexp as its first argument, then:
my-exe '(a b)' <TAB>
will run my-exe
in completion mode with:
'(a b)'
as its first argument, rather than:
(a b)
as would be passed if RETURN had been pressed instead. Since Sexp.of_string "'(a b)'"
fails, in this example tab completion won't work.
Changed the internals a bit to make this possible. Most notably, the Parser
module is now applicative rather than monadic, which required only a few simple changes to support.
Made Time_ns.to_time
and Time_ns.Span.to_span
round to the nearest microsecond in all cases.
Previously, Time_ns.Span.to_span
sometimes rounded incorrectly for negative spans.
Added Time.Zone.prev_clock_shift
, the analog of next_clock_shift
.
val prev_clock_shift
: t
-> before:Time_internal.T.t
-> (Time_internal.T.t * Span.t) option
Implemented next_clock_shift
and prev_clock_shift
using Array.binary_search_segmented
.
- Added
Lock_file.get_pid : string -> Pid.t option
. - Added
val random: unit -> int
to Time_ns
and Time_ns.Span
. Renamed Iobuf.sub
as Iobuf.sub_shared
.
This more closely matches Bigstring
, and clarifies the semantics of Iobuf.sub
vs sub
in Blit_intf.S
.
- Added
Iobuf
blit modules: Blit
, Blit_consume
, Blit_fill
, Blit_consume_and_fill
. Added Piecewise_linear.first_knot
and last_knot
.
val first_knot : t -> (key * value) option
val last_knot : t -> (key * value) option
- Made
Unix.Cidr
match Comparable.S_binable
, and added Cidr.create
and Cidr.netmask_of_bits
. - Moved
Unix.tm
and Unix.strftime
from Core_kernel
to Core
. Made Crc.crc32
return Int63.t
rather than int64
, and added Crc.bigstring_crc32
and Iobuf.crc32
.
Cleaned up old cruft in the C stubs for CRC checking.
- Added
Iobuf.narrow_lo
and narrow_hi
, which comprise Iobuf.narrow
. Changed Linux_ext.Timerfd
, Epoll.wait
, and Unix.select
to use (int
) Time_ns
rather than (float
) Time
.
This avoids spurious float conversions and rounding problems.
Made all timeouts consistently treat negative timeouts as "timeout immediately".
This fixes an incorrect behavior of Linux_ext.Timerfd.set_after
and set
, which had been rounding to the nearest microsecond, which was particularly bad for time spans smaller than 500ns, which would be rounded to zero, and then would cause the timerfd to never fire. Now, the small span is directly fed to timerfd_settime
. We also changed a span of zero to be treated as 1ns
, to avoid the behavior of timerfd_settime 0
, which causes the timerfd to be cleared and never fire.
- Made
Or_error
match Applicative.S
. Added Command.Param
, with the intention of one day replacing Command.Spec
and providing an applicative interface for command-line parsing.
This change required lots of rearrangement of command.mli
so that Command.Param
and Command.Spec
could share large portions of their interface. As a side effect, the interface is more sweeksy than before.
- Added
Command.shape
, for exposing the shape of a command, including what subcommands its has. Changed Syscall_result.to_result
to return a preallocated object, and changed many uses to take advantage of this property.
Pattern matching on results is much clearer than if-analysis and avoids double-checking errors in many cases.
Syscall_result.to_result
can only return preallocated results for a few Ok
values from Syscall_result.Int
, of course, and likewise for other large ok_value
types. We have initially, and arbitrarily, limited preallocation to 64 errno
's and 2048 ok_value
's.
- Added
Sys.big_endian : bool
, from Caml.Sys
. Disabled unit tests in Time_ns
that started failing around 10:40pm NYC 2015-05-15.
The tests indicate an off-by-one-microsecond error in round tripping between Time.Span.t
and Time_ns.Span.t
.
- Renamed
Dequeue
as Deque
. - Added
Fdeque
, a functional deque (a double-ended Fqueue
, or a functional Deque
). Changed Fqueue
's bin-io format, and added a stable type.
Deprecated deque-like functions in favor of Fdeque
.
- Added
Fheap
, a functional heap implementation based on pairing heaps. - Reverted the change to
Or_error
's bin-io format made in 112.17, going back to the format in 112.16 and before. - Added to
Unix.env
type a ``Replace_rawvariant, as used in
execand
fork_exec`. - Added
Array.Permissioned
module, which has a permissioned array type and permissioned versions of all the regular array functions. - Added
Date.week_number : t -> int
. Added values to Day_of_week
: of_int_exn
, iso_8601_weekday_number
, weekdays
.
val of_int_exn : int -> t
val iso_8601_weekday_number : t -> int
val weekdays : t list (- [ Mon; Tue; Wed; Thu; Fri ] *)
- Switched
Float
IEEE functions to use Int63.t
for the mantissa rather than int
, so they work on 32-bit platforms. - Added a
length
field to the Map.t
record, making Map.length
O(1)
rather than O(n)
. - Moved a fragment of
Time_ns
from Core
to Core_kernel
, enough so that Async_kernel
can use Core_kernel.Time_ns
and ultimately only depend on Core_kernel
. - Fixed compilation of
Time_ns
32-bit Linux. - Added
Bounded_int_table.clear
. Fixed the module_name
passed to Identifiable.Make
for a number of modules.
The module name must be an absolute module path.
Reported here: https://github.com/janestreet/core/issues/52
- Added
Tuple.Binable
functor, for making binable tuples. Sped up a Time_stamp_counter
unit test.
Time_stamp_counter
unit test has an 18s unit test, which seems excessive. Take a couple of orders of magnitude off the number of iterations.
Added Time_ns.pause
, whose implementation is the same as Time.pause
.
This involved moving the nanosleep
C code from Core
to Core_kernel
.
This was necessary so that Async_kernel
can pause without introducing a dependence of Async on Core.
Made Core_kernel.Time_ns.Alternate_sexp
use a similar format to Core.Time_ns
.
This was needed so that Async_kernel
can use a nice sexp format for time spans.
Changed Timing_wheel
implementation to use Time_ns
, and moved to Core_kernel.Timing_wheel_ns
; made Core.Timing_wheel
a wrapper around Timing_wheel_ns
.
Generalized the timing-wheel interface to be parametric in Time
, so that one interface applies to both Timing_wheel
and Timing_wheel_ns
.
Generalized the timing-wheel unit tests to a functor, Timing_wheel_unit_tests.Make
, that is used to test both Timing_wheel_ns
and Timing_wheel_float
. Moved a few tests that depend on Time
and Date
from the functor into timing_wheel_float_unit_tests.ml
.
Split out Timing_wheel.Debug
into a separate functor, Timing_wheel_debug.Make
.
This was done in so that Async_kernel
can depend only on Core_kernel
and not Core
.
Added optional arguments to Command.group
: ?body
and ?preserve_subcommand_order
.
preserve_subcommand_order : unit
causes subcommands to be in the order they are specified, rather than sorted.
body : (path:string list -> unit)
is called when no additional arguments are passed.
- Added accessor function
Command.summary : t -> string
. - Fixed a bug in
Time.Span
robust comparison. Changed Command
's tab-completion bash code so that it is possible for programs to return completions containing spaces.
Actually knowing when and how to do so is difficult, because of course there's escaping to worry about. Adding helper functions to make that sort of thing manageable is left for future work.
- In
Command
, made the -version
and -build-info
flags work at the top level when there are subcommands. - Added
Sequence.interleaved_cartesian_product
, which implements cartesian product of potentially infinite sequences.
Deprecated the single-line files that simply include
the corresponding Core_kernel module. Those are unnecessary, because people should use Core.Std
.
We will keep these aliases around for a version before deleted them entirely.
- Changed finalizers and signal handlers to, upon an unhandled exception, exit nonzero rather than asynchronously raise.
Removed Time.Zone.find_office
.
Replaced uses with the still-blocking Time.Zone.find_exn
Made many changes to Time
to make the time zone explicit instead of implicitly using the local timezone.
Added zone:Time.Zone.t
parameters to many functions. In almost all cases, used ~zone:Time.Zone.local
where previously it was implicit.
Removed of_local_ofday
and to_local_ofday
in favor of the explicit versions (with Time.Zone.local
).
Removed Time.Zone.machine_zone ()
in favor of local
.
- Exported
Core.Std.With_return
. - Exposed
Core.Std.with_return_option
. - Fixed
Time_ns.Ofday.of_span_since_start_of_day
to check its input. - Changed
Time_ns.to_span
and of_span
to round to microseconds, for round trippability. - Added
Unix.Error
module, for the Unix.error
type. Added Unix.Syscall_result
, a new abstract type representing the result of a Unix system call as an int
, to avoid allocation.
A lot of Unix system calls return an integer on success, so for ones that are called a lot, we can encode errors as -errno
. This module abstracts this concept.
- Changed
Iobuf.recvmmsg
functions to return the new Unix.Syscall_result
. - Changed
Unix.exec
's ?env
argument to support extending the environment in addition to replacing it. - Added
with compare
to Unix.Exit.t
and Unix.Exit_or_signal.t
. Moved Backtrace
to Core_kernel
.
Deleted backtrace_stubs.c
, now that we have Printexc.get_callstack
.
- Changed
Bigstring.read_assume_fd_is_nonblocking
and send_nonblocking_no_sigpipe
to return Unix.Syscall_result.t
, to reduce allocation. - Changed
Iobuf.send_nonblocking_no_sigpipe
to handle EINTR
like EAGAIN
, instead of raising. - Added
Command.Spec.char
. - Changed
Process_env.parse_ssh_client
to accept an SSH_CLIENT
that is just IP address without ports.
Renamed Linux_ext.gettid
as Unix.gettid
, and added OpenBSD support.
SYS_gettid
is not available on OpenBSD, but is used in Core_extended
. See the mailing list discussion about this here:
https://groups.google.com/forum/#!topic/ocaml-core/51knlnuJ8MM
Seems like the OpenBSD alternative is:
pid_t getthrid(void);
although it's not defined in any header file, which is a bit unfortunate.
- Added
Piecewise_linear.precache
, which computes a lookup table that speeds up subsequent calls to Piecewise_linear.get
. - Added
Time_ns
module, representing times as 63-bit integers of nanoseconds since the epoch. - Fixed build of
unix_stubs.c
on OpenBSD. - In
Daemon
, fixed an error message regarding WSTOPPED
(fixes #47). Added Time.Span.Stable.V2
, with sexps that use new suffixes for microseconds (us
) and nanoseconds (ns
).
Time.Span.of_string
supports the new format, but Time.Span.to_string
doesn't yet produce it -- we plan to change that later, after the new of_string
has made it out more widely.
- Added
Time.Span.to_string_hum
, which gives more options for rendering time spans. Merged the recvmmsg
stubs in Bigstring
and Iobuf
.
Factored out a shared underlying recvmmsg
call that both stubs use.
Restored -pedantic
by avoiding a C99 feature (variable-length stack arrays).
- Made
Date.t
abstract, and changed its representation from a 4-word record to an immediate int (packing year, month, day). - In
Daemon
, changed the permissions of the std{err,out}
files generated during daemonization from 0o777
to 0o644
. Moved Thread_safe_queue
from core
to core_kernel
.
This was done so that Async_kernel
can use it, eliminating one of Async_kernel
's dependencies on Core
.
Thread_safe_queue_unit_tests
remains Core
, at least for now, because it has some dependencies on other stuff in Core
.
- Removed vestigial code supporting OCaml 4.00.
Added Command
support for flags that are passed one or more times.
Added Command.Spec.one_or_more
and Command.Spec.non_empty_sequence
to deal with the cases where you expect a flag or anonymous argument (respectively) to be passed one or (optionally) more times. This is common enough and distinct from the case where you want the argument passed zero or more times that it seems like we should canonize it in the library.
In Lock_file
, made stale lock detection more robust.
Made Lock_file.create foo
succeed if foo
is absent and foo.nfs_lock
file is present and stale. Previously, it would fail.
- Removed
Syslog.syslog
's add_stderr
argument; use the PERROR
option instead. Fixed unix_stubs.c
compilation on NetBSD
Closes #45
- Added
Filename
operators /^
and /@
, and of_parts
, like the same functions for Catalog paths. Changed Iobuf
functions that advance the iobuf to not also return a redundant number of bytes processed.
This avoids a small allocation (in the case of the int option
functions) and normalizes the result (so the same information isn't returned two ways). Actually, it doesn't yet avoid the allocation in the implementation, as the corresponding Bigstring
functions must still return the number of bytes processed, and currently do so as an option. We hope to eventually change that.
In the future I expect we will change unit
to some error
variant to also avoid the exception construction for EWOULDBLOCK/EAGAIN
. We can even make Unix syscalls noalloc
if we're careful.
- In
Unix
module, added unit tests for Cidr.does_match
.
Fixed a bug in Bigstring.really_recv
if recv
doesn't receive all the data it wants.
This bug has been around forever; it may not have caused trouble because Bigstring.really_recv
(1) is barely used (the only use is in Bigstring.unmarshal_from_sock
) and (2) passes recv
the MSG_WAITALL
flag, so it will read the full amount unless it gets interrupted by a signal.
Fixed Bigstring.read
's handling of EINTR
so that it retries rather than returning zero.
This fixes a bug introduced in 111.09 in the interaction between Bigstring.read
and Async.Reader
. Prior to 111.09, Bigstring.read
would raise on EINTR
, and Async.Reader
would propagate the exception. From 111.09 to 111.16, Bigstring.read
would return zero, which would confuse Async.Reader
into thinking it reached EOF when it hadn't. From 111.17, Bigstring.read
will retry and not return zero when not at EOF.
We believe the bug was rare, because otherwise we would have frequently seen EINTR
exceptions prior to 111.09.
Added Command.Spec.apply
and pair
, which allow one to program more with Spec.param
rather than Spec.t
.
val apply : ('a -> 'b) param -> 'a param -> 'b param
val pair : 'a param -> 'b param -> ('a * 'b) param
Added Command.Spec.file
, which builds an Arg_type
value with the same autocompletion as Spec.file
.
(** [file] defines an [Arg_type.t] that completes in the same way as
[Command.Spec.file], but perhaps with a different type than [string] or with an
autocompletion key. *)
val file
: ?key:'a Univ_map.Multi.Key.t
-> (string -> 'a)
-> 'a t
Change some Bigstring
functions to retry on EINTR
rather than raise.
The following functions (and their unsafe versions) were affected:
read
really_read
really_recv
really_write
really_send_no_sigpipe
Some other Bigstring
functions, like input
and output
, already retried on EINTR
, so this change has precedent.
All of the affected stubs raise Bigstring.IOError
on failure, rather than Unix_error
, which means the normal method for retrying on EINTR
doesn't work. In particular Async.Reader
didn't retry them, even though it was supposed to.
Additionally, the documentation for the following functions was corrected to say that they raise =Unix_error= rather than =IOError=:
pread_assume_fd_is_nonblocking
pwrite_assume_fd_is_nonblocking
Eliminated global binary-to-decimal tables computed at startup for converting Date
and Of_day
to string.
Used an efficient implementation of division by 10, rather than the sprintf
tables in time_internal.ml
. This result in much less allocation at startup and it is a bit faster:
Name | Time/Run | mWd/Run | Percentage |
---|
Date.to_string | 69.39ns | 3.00w | 100.00% |
Name | Time/Run | mWd/Run | Percentage |
---|
Date.to_string | 53.38ns | 3.00w | 100.00% |
- Fixed
Time.Zone
tests so that they are deterministic. - Added
Iobuf.to_string_hum
, which produces a readable, multi-line representation of an iobuf, intended for debugging. - Fixed brittle unit tests of
Command
.
Added inline benchmarks for =Iobuf= and =Time=.
Hera are some of the results from the new benchmarks, with some indexed tests dropped.
Name | Time/Run | mWd/Run | Percentage |
---|
[time.ml:Time] Time.to_string | 848.74ns | 249.98w | 100.00% |
[time.ml:Time] Time.to_ofday | 59.66ns | 38.00w | 7.03% |
[time.ml:Time] Time.now | 39.78ns | 2.00w | 4.69% |
[time.ml:Time] Time.Zone.find_office | 83.64ns | 4.00w | 9.85% |
[time.ml:Time] Time.Span.of_hr | 3.71ns | 2.00w | 0.44% |
[time.ml:Time] Time.Span.of_min | 3.69ns | 2.00w | 0.44% |
[time.ml:Time] Time.Span.of_sec | 2.72ns | | 0.32% |
[time.ml:Time] Time.Span.of_ms | 6.02ns | 2.00w | 0.71% |
[time.ml:Time] Time.Span.of_ns | 5.98ns | 2.00w | 0.71% |
Name | Time/Run | Percentage |
---|
[iobuf.ml:Blit tests] functor blit:5 | 15.53ns | 7.66% |
[iobuf.ml:Poke tests] char:0 | 4.11ns | 2.03% |
[iobuf.ml:Poke tests] uint8:0 | 5.35ns | 2.64% |
[iobuf.ml:Poke tests] int8:0 | 4.59ns | 2.26% |
[iobuf.ml:Poke tests] int16_be:0 | 5.19ns | 2.56% |
[iobuf.ml:Poke tests] int16_le:0 | 5.14ns | 2.53% |
[iobuf.ml:Poke tests] uint16_be:0 | 5.11ns | 2.52% |
[iobuf.ml:Poke tests] uint16_le:0 | 5.12ns | 2.53% |
[iobuf.ml:Poke tests] int32_be:0 | 5.17ns | 2.55% |
[iobuf.ml:Poke tests] int32_le:0 | 4.91ns | 2.42% |
[iobuf.ml:Poke tests] uint32_be:0 | 5.73ns | 2.83% |
[iobuf.ml:Poke tests] uint32_le:0 | 5.74ns | 2.83% |
[iobuf.ml:Poke tests] int64_be:0 | 5.33ns | 2.63% |
[iobuf.ml:Poke tests] int64_le:0 | 4.93ns | 2.43% |
[iobuf.ml:Peek tests] char:0 | 5.50ns | 2.71% |
[iobuf.ml:Peek tests] uint8:0 | 4.68ns | 2.31% |
[iobuf.ml:Peek tests] int8:0 | 4.91ns | 2.42% |
[iobuf.ml:Peek tests] int16_be:0 | 5.19ns | 2.56% |
[iobuf.ml:Peek tests] int16_le:0 | 4.90ns | 2.42% |
[iobuf.ml:Peek tests] uint16_be:0 | 5.17ns | 2.55% |
[iobuf.ml:Peek tests] uint16_le:0 | 5.10ns | 2.51% |
[iobuf.ml:Peek tests] int32_be:0 | 5.17ns | 2.55% |
[iobuf.ml:Peek tests] int32_le:0 | 4.92ns | 2.42% |
[iobuf.ml:Peek tests] uint32_be:0 | 5.45ns | 2.69% |
[iobuf.ml:Peek tests] uint32_le:0 | 5.46ns | 2.69% |
[iobuf.ml:Peek tests] int64_be:0 | 6.61ns | 3.26% |
[iobuf.ml:Peek tests] int64_le:0 | 6.31ns | 3.11% |
Re-implemented Thread_safe_queue
to improve performance and reduce allocation.
The new implementation requires 3 words per element, down from the 7 words required by the old implementation.
The new implementation pools elements so that they can be reused, so there is no allocation in steady-state use.
The new implementation has dequeue_exn
rather than dequeue
, so that one can dequeue without allocating 2 words.
Eliminated create'
. One should just use create
and explicit calls to enqueue
and dequeue_exn
.
Eliminated dequeue_until_empty
. One should use an explicit while loop guarded by length
and using dequeue_exn
.
Moved Thread_safe_queue
from Core_kernel
to Core
, since it's thread related.
All in, there is now no allocation in a steady-state usage of enqueueing and dequeueing elements, as opposed to 9 words per enqueue+dequeue
in the old implementation. This reduces the cost from enqueue+dequeue
taking 166-216ns to enqueue+dequeue_exn
taking 48-82ns (plus eliminating gc impacts). Here are some BENCH
results, the first table being the old implementation, and the second table the new.
Name | Time/Run | mWd/Run | mjWd/Run |
---|
[thread_safe_queue.ml] enqueue + dequeue of immediate | 183.89ns | 9.00w | 7.02w |
[thread_safe_queue.ml] enqueue + dequeue of young object | 216.69ns | 11.00w | 9.01w |
[thread_safe_queue.ml] enqueue + dequeue_exn of old object | 166.75ns | 9.00w | 7.02w |
Name | Time/Run | mWd/Run |
---|
[thread_safe_queue.ml] enqueue + dequeue_exn of immediate | 48.20ns | |
[thread_safe_queue.ml] enqueue + dequeue_exn of young object | 81.96ns | 2.00w |
[thread_safe_queue.ml] enqueue + dequeue_exn of old object | 48.30ns | |
Changed {Bigstring,Iobuf}.recvmmsg_assume_fd_is_nonblocking
, when no message is available, to return a negative number rather than raise.
This was done for performance reasons, because raising an exception is expensive, due to the stashing of the backtrace and the string creation.
- Added
Iobuf.unsafe_resize
. Changed Bigstring.blit
so that it doesn't release the OCaml lock on map_file
bigstrings.
The old behavior of releasing the lock for blits of (small) bigstrings involving mmapped files was problematic and inconsistent. Its cost is high, and fundamentally any access to a mapped bigstring could cause some level of blocking.
- Added time-related
Arg_type.t
values to Command.Spec
. Added module Type_immediacy
, which has witnesses that express whether a type's values are always, sometimes, or never immediate.
This code used to be in the Typerep_immediate
library in typerep.
- Added
Debug.should_print_backtrace : bool ref
, to control whether Debug.am*
functions print backtraces. - Added to
Float
inline benchmarks. Moved all of the Gc
module into Core_kernel
.
Part of the Gc
module used to be in Core
because it used threads. But it doesn't use threads anymore, so can be all in Core_kernel
.
Improved Iobuf
support for copying to/from strings and bigstrings.
The new modules are Iobuf.{Consume,Peek}.To_{bigstring,string}
. They match a Blit
-like interface. We don't yet implement the Blit
interface in all applicable contexts, but do use Blit.Make
and expose some of the operations we want in the forms we expect them to take under a Blit
interface.
- Added
Linux_ext.Timerfd.to_file_descr
. - Added to
Time.next_multiple
an optional argument to control whether the inequality with after
is strict. Added Time.Zone.local
, a lazily cached Time.Zone.machine_zone ()
.
This is the first stage in a plan to make explicit timezones more pervasive. First, they are made more convenient, by replacing the relatively wordy Time.Zone.machine_zone ()
with Time.Zone.local
. This involves making the underlying timezone type lazy.
The next stage will be to remove machine_zone
and use Time.Zone.local
everywhere instead. Then (it is hoped) instead of of_local_whatever
, we just say e.g. of_date_ofday Time.Zone.local
and currently-implicitly-local functions will be able to switch over to explicit timezones without becoming too verbose.
- Added
Timing_wheel.Alarm.null
. Made Unix.File_descr.t
have with sexp
.
Closes janestreet/async_unix#3
- Fixed OpenBSD compilation failures in C stubs.
- Fixed
Lock_file.is_locked
to require read permission, not write permission, on the lock file. Added to Unix.mcast_join
an optional ?source:Inet_addr.t
argument.
From pull-request on bitbucket: https://bitbucket.org/janestreet/core/pull-request/1/receive-source-specific-multicast/diff
- Added
Unix.File_descr.equal
. - Added
Lock_file.Nfs.unlock
, the Or_error
version of unlock_exn
. - Improved the detail of the error messages exposed by
Lock_file.Nfs.create{,_exn}
. Added Unix.set_mcast_ifname
, to control the interface used for UDP multicast traffic.
Added bindings for setsockopt IP_MULTICAST_IF
.
See 6.3 in: http://www.tldp.org/HOWTO/Multicast-HOWTO-6.html
Changed Command
argument processing to treat a single dash (-
) as an anonymous argument rather than a flag.
This change follows the unix convention of passing -
as an anonymous argument meaning stdin
.
Added more bin-prot support to Iobuf
: Consume.bin_prot
, Fill.bin_prot
, Peek.bin_prot
, Poke.bin_prot
.
Framing doesn't do much for Iobuf
, so these are to be more standard, unframed accessors, as opposed to fill_bin_prot
.
Added Core.Debug.am
, amf
, and ams
, for outputting debugging messages showing the current source-code position.
Unfortunately, these aren't available in Core.Std.Debug
, but only in Core.Debug
. That will be fixed in 109.49.
- Made
Time_stamp_counter
compile on non x86-64 platforms. Made Core.Std.Debug
be Core.Debug
rather than Core_kernel.Debug
.
This exposes the Debug.am*
functions added in 109.48.
Added Time_stamp_counter
module, which has fast (< 10 nanos) access to the hardware time-stamp counter.
This module provides the fast function Time_stamp_counter.now ()
which is our best effort high-performance cycle counter for a given platform. For x86 systems this retrieves the CPU's internal time stamp counter using the RDTSC
instruction. For systems that do not have a RDTSC instruction, we fallback to using clock_gettime(CLOCK_MONOTONIC)
.
Here is a benchmark of execution time in nanos and allocations in words:
Name Time/Run Minor
------------------------------- ---------- -------
Time.now 39.02 2.00
TSC.now 7.54
TSC.to_time 4.88 2.00
TSC.to_time (TSC.now ()) 8.54 2.00
TSC.to_time_nanos 4.49
TSC.to_time_nanos(TSC.now ()) 8.95
Calibrator.calibrate 279 34.00
Historically, the rate of increment of the TSC (sometimes referred to as the TSC frequency) varied based of CPU overclocking, temperature, load etc. On modern Intel CPU's the TSC is expected to be stable. On Linux systems, the "constant_tsc" in /proc/cpuinfo
indicates that the machine has a stable TSC rate. While this module assumes that the TSC is relatively stable, it can adapt to small variations in the TSC frequency.
Changed Daemon.daemonize
and daemonize_wait
to leave the umask
alone by default.
Previously, these had alwasy set the umask to 0
, which means that all app-harness programs and all binaries run from grass were creating world-writeable (0666
) files by default.
Normalized Command
's help messages.
Made anonymous argument names uppercase and subcommand names lowercase.
In Iobuf
, added duals to flip
and snapshot
to work on the high end of the window.
flip
has been renamed to flip_lo
. The dual of flip_lo
is the newly added flip_hi
, and shifts the window forward to continue reading, rather than back to switch from writing to reading, as flip_lo
does.
flip_hi
, in practice, needs snapshots of the upper bound of the window, we split Snapshot
into Lo_bound
and Hi_bound
and introduced bounded versions of flip_lo
, compact
, and flip_hi
to support buffers which are only partially filled, but have substructure, like packet buffers.
Here's the new API.
module type Bound = sig
type ('d, 'w) iobuf
(** Expose =t = private int= only if a =t= is stored in a mutable data structure
somewhere and leads to a measurable =caml_modify= performance problem. *)
type t with sexp_of
val window : (_, _) iobuf -> t
val limit : (_, _) iobuf -> t
val restore : t -> (_, seek) iobuf -> unit
end
module Lo_bound : Bound
module Hi_bound : Bound
val flip_lo : (_, seek) t -> unit
val bounded_flip_lo : (_, seek) t -> Lo_bound.t -> unit
val flip_hi : (_, seek) t -> unit
val bounded_flip_hi : (_, seek) t -> Hi_bound.t -> unit
- Disabled use of
recvmmssg
, which isn't available on our CentOS 5 machines. - Defined
Option.compare
using with compare
so that their comparisons are consistent. Cleaned up the Dequeue
module's interface and implementation.
The interface now matches the conventions used elsewhere in Core
. The new implementation is also cleaner and more efficient.
Reimplemented the Stack
module to improve performance, and renamed the old implementation as Linked_stack
.
The new Stack
is implemented with this type:
type 'a t `
{ dummy : 'a;
mutable length : int;
mutable elts : 'a array;
}
Linked_stack
is implemented with this type:
type 'a t `
{ mutable length : int;
mutable elts : 'a list;
}
Using an array rather than a linked list is a more efficient and traditional implementation. Pushing to the stack now does not require allocation, except in the rare case when the stack grows.
One downside is that Stack.clear
is now O(n) rather than O(1).
This change also eliminates the Stack.Empty
exception, so any code matching on that exception should fail to compile, and should be changed to depend on option-returning top
and pop
operations.
Improved Lock_file.Nfs
.
- Allowed an arbitrary message to be stored and retreived.
- Fixed a case where
create
might throw an exception. - Delete both files used for locking when we unlock.
- Split
Core
into Core_kernel
and Core
. Core_kernel
is composed of all modules of Core
that do not depend on unix or threads, and Core
contains the rest and depends on Core_kernel
.
The goal is to have a very portable standard library that can especially run on exotic environment such as Javascript.
So that code that directly refers to Core
(rather than Core.Std
) for modules that have moved to Core_kernel
, we included "proxy" modules in Core
that simply include the corresponding module from Core_kernel
.
Fixed Core.Flags
to build on 32-bit machines.
It had contained a unit test with an integer literal too large to be accepted by the compiler when building on a 32-bit machine.
Massively improved the signatures of Map
and Set
, both for readability and ocamldoc, as well as improved type error messages.
For instance the type of Int.Set.singleton
was:
('a, 'comparator, 'a Core.Std.Int.Set.elt_ -> ('a, 'comparator) Core.Std.Int.Set.t_) Core.Core_set_intf.without_comparator
Now it is simply:
int -> Int.Set.t
Added an optional argument to Command.run
that can be used to specify default flags from a user config file.
The optional argument can extend the command line based on the path to the command.
Rename module Weekday
as Day_of_week
.
The name Weekday
conflicted with ordinary usage of "weekday" to mean Monday through Friday.
Changed sexp_of_t
for {Month,Ofday,Time,Time.Span}.{Set,Map}
to use the nice sexp format of the underlying atomic type.
Previously, the converter had used thes raw type (float
, int
, etc.). t_of_sexp
still accepts both formats; we will remove the ability to accept the raw format in the distant future.
This output-format change was planned when we originally in 108.06b improved those t_of_sexp
functions to accept both formats.
- Added
Unix.remove
. - Removed some
IFDEF
's connected to OCaml <4 support.
- Wrapped
Unix.wordexp
in an Or_error.t
since it is not available on all systems. - Added function
Process_env.parse_ssh_client
. This gets the address from which you're currently ssh'd in. - Added to
Unix
module the ability to get and set IP_MULTICAST_LOOP
and IP_MULTICAST_TTL
. - Exposed module
Core.Std.Ref
, which was previously only available via Core.Ref
. Remove Mutex.am_holding_mutex
and changed the type of Mutex.try_lock
.
With NPTL it is impossible to determine which thread is holding the lock. So, Mutex.am_holding_mutex
is unimplementable. Also, Mutex.try_lock
was incorrect because it claimed to raise if one was attempting to recursively lock. Since it's not possible to distinguish between recursive locking and the lock being held by another thread, we changed the type to make this clear:
val try_lock : t -> [ `Already_held_by_me_or_other | `Acquired ]
Removed our custom version of the OCaml runtime's core_sys_open
function.
There used to be a bug in the OCaml runtime, PR#5069, in which open_{in,out}_gen
could block while holding the OCaml lock, because they made a call to fcntl
outside the blocking section. We had our own C code with the bug fix and re-exposed the fixed versions of the functions in Core
.
The bug in OCaml has been fixed, so we have removed our patched function from Core
.
In unix_stubs.c
, switched from using FNM_FILE_NAME
to FNM_PATHNAME
.
The GNU project introduced FNM_FILE_NAME as a non-portable synonym for FNM_PATHNAME.
We were using pre-processor macros to define FNM_FILE_NAME as FNM_PATHNAME if unavailable, but it is simpler to just use the more portable FNM_PATHNAME everywhere.
Changed Time.to_string
and Time.sexp_of_t
to include the timezone.
This is an incompatible change with very old programs in which Time.of_string
and Time.t_of_sexp
did not support the timezone.
If you have programs that are:
- very old and do Time string/sexp handling
- rely on reading in time values without using
Time.of_string
and Time.t_of_sexp
. - rely on chains of writing/reading/writing times across machines and timezones where the time is always intended to be taken as the local time on the currently reading machine
you should recompile/review your code to make sure you won't have issues.
Added function List.remove_consecutive_duplicates : 'a t -> equal:('a -> 'a -> bool) -> 'a t
.
This returns the input list with consecutive duplicates removed, and doesn't change the order of the remaining elements.
Added module User_and_group
, which is a pair of a unix username and primary unix group.
The string/sexp converters follow the usual unix convention of <user>:<group>
.
Added function Date.first_strictly_after : t -> on:Weekday.t -> t
.
first_strictly_after t ~on:day_of_week
returns the first occurrence of day_of_week
strictly after t
.
Added functor Type_equal.Lift
.
It is always safe to conclude that if type a
equals b
, then type a X.t
equals b X.t
, for any type X.t
. The OCaml type checker uses this fact when it can. However, sometimes, e.g. when using Type_equal.conv
, one needs to explicitly use this fact to construct an appropriate Type_equal.t
. The Type_equal.Lift*
functors do this.
module Type_equal : sig
type ('a, 'b) t
...
module Lift (X : T1) : sig
val lift : ('a, 'b) t -> ('a X.t, 'b X.t) t
end
end
Fixed major performance problem with hashing in Int.Table
.
Our Int.Table.replace
was 3 times slower than polymorphic hash table and find
was 8 times slower.
This was caused by using:
external seeded_hash_param : int -> int -> int -> 'a -> int = "caml_hash" "noalloc"
in Int.Table
but:
external old_hash_param : int -> int -> 'a -> int = "caml_hash_univ_param" "noalloc"
everywhere else.
The seeded_hash_param
was introduced in Caml 4.
We fixed this problem by changing Int.hash
from:
let hash (x : t) = Hashtbl.hash x
to:
let hash (x : t) = if x >= 0 then x else ~-x
- Added
Bigstring.{pread,pwrite}
, which allow reading and writing at a specific file offset. Added module Nothing
, which is a type with no values.
This is useful when an interface requires you to specify a type that you know will never be used in your implementation.
Changed Identifiable.Make
so that it registers a pretty printer.
Identifiable.Make
now uses Pretty_printer.Register
. This requires all calls to Identifiable.Make
to supply a val module_name : string
.
- Made
Core.Zone
match the Identifiable
signature. Made polymorphic equality always fail on Core.Map.t
and Core.Set.t
.
Before this change, polymorphic equality on a Core.Map
or a Core.Set
could either raise or return false
. It returnd false
if the data structures were unequal, and raised if the data structures were equal.
This is because their type definitions looked like:
type ('k, 'v, 'comparator) t =
{ tree : ('k, 'v) Tree0.t;
comparator : ('k, 'comparator) Comparator.t;
}
and polymorphic equality visits a block's fields in order. So, it will detect unequal trees and return false, but if the trees are equal, it will compare the comparators and raise because of the functional value.
This change reversed the order of the fields so polymorphic equality always fails.
Added Command.Spec.flags_of_args_exn
, for compatibility with OCaml's standard library.
This function converts a Core.Std.Arg.t
into a Command.Spec.t
.
Made various modules Identifiable
: Char
, String
, and the various Int
modules.
In particular, Int
being identifiable is useful, because one can now write:
module My_numeric_identifier : Identifiable ` Int
You might think that we could now delete String_id
, and just write:
module My_string_identifier : Identifiable ` String
But this is not quite equivalent to using String_id
, because String_id.of_string
enforces that its argument is nonempty.
Removed module Space_safe_tuple
, which became unnecessary in OCaml 4.00.0.
OCaml 4.00.0 included Fabrice's patch to fix the space leak that Space_safe_tuple
was circumventing (PR#5288, commit SVN 11085).
- Added
Exn.to_string_mach
, for single-line output. Added Linux_ext.bind_to_interface
, to improve security of UDP applications.
val bind_to_interface : (File_descr.t -> string -> unit) Or_error.t
This uses the linux-specifc socket option BINDTODEVICE
to prevent packets being received from any interface other than one named.
- Fixed
Unix.mkdir_p
on Mac OS X.
Add some functions to Byte_units
.
- Added functions:
to_string_hum
, scale
, Infix.//
. - Eliminated the notion of "preferred measure", so a
Byte_units.t
is just a float
.
Improved the performance of Array.of_list_rev
.
The new implementation puts the list elements directly in the right place in the resulting array, rather that putting them in order and then reversing the array in place.
Benchmarking shows that the new implementation runs in 2/3 the time of the old one.
Fixed Fqueue.t_of_sexp
, which didn't work with sexp_of_t
.
There was a custom sexp_of_t
to abstract away the internal record structure and make the sexp look like a list, but there wasn't a custom t_of_sexp
defined, so it didn't work.
- Added
Stable.V1
types for Host_and_port
. Removed Identifiable.Of_sexpable
and Identifiable.Of_stringable
, in favor of Identifiable.Make
Identifiable.Of_sexpable
encouraged a terrible implementation of Identifiable.S
. In particular, hash
, compare
, and bin_io were all built by converting the type to a sexp, and then to a string.
Identifiable.Of_stringable
wasn't as obviously bad as Of_sexpable
. But it still used the string as an intermediate, which is often the wrong choice -- especially for compare
and bin_io
, which can be generated by preprocessors.
Added Identifiable.Make
as the replacement. It avoids using sexp conversion for any of the other operations.
Added List.intersperse
and List.split_while
.
These came from Core_extended.List
.
val intersperse : 'a list -> sep:'a -> 'a list
val split_while : 'a list -> f:('a -> bool) -> 'a list ** 'a list
Added a functor, Pretty_printer.Register
, for registering pretty printers. The codifies the idiom that was duplicated in lots of places:
let pp formatter t = Format.pp_print_string formatter (to_string t)
let () = Pretty_printer.register "Some_module.pp")
- In
Core.Std
, exposed Or_error.ok_exn
and Or_error.error
Removed some values exported by Core.Std
.
Removed some values from Core.Std
that weren't widely used, or we didn't think should be exposed, including ascending
, descending
, and equal
, which use polymorphic comparison, and we want to discourage.
Here's a guide to some of what was removed, and what one should now use instead.
removed | replace with |
-----------------------------------+--------------------------------------- | |
Int_replace_polymorphic_compare
| Int.Replace_polymorphic_compare
|
ascending
| Polymorphic_compare.ascending
|
descending
| Polymorphic_compare.descending
|
equal
| Polymorphic_compare.equal
|
ifprintf
| Printf.ifprintf
|
sscanf
| Scanf.sscanf
|
Scan_failure
| Scanf.Scan_failure
|
string_of__of__sexp_of
| Sexplib.Conv.string_of__of__sexp_of
|
of_string__of__of_sexp
| Sexplib.Conv.of_string__of__of_sexp
|
type vec
| type float64_vec
|
- Disallowed
<:sexp_of<
with two underscores; using a single underscore instead. - Added
Command.Spec.Arg_type.of_alist_exn
as an alternative for of_map
. This captures the common pattern to create the map from an alist. - Improved the performance of
Hashtbl
. Constrained hashtbl size to a power of two and used a bitmask rather than mod operation for finding hash buckets. Improved the performance of Univ
, using the Type_equal
GADT. The new implementation improves the run-time and space usage over the old one. In the old implementation, a Univ.t
was represented as record with three fields: an exception, a string, and a closure. Creating a univ required allocating three heap blocks, the exception (3 words), the closure (3 words), and the three-field record (4 words). In the new implementation, a Univ.t
is represented as a 2-field heap block containing the Constr.t
and the value. Creating a univ allocates that single 3-word block, improving on the 10 words needed previously.
Matching on univs is also faster. In the old implementation, matching on a univ required making a function call, testing exception equality, and allocating a Some
block. Now, it does just the test and allocation. Furthermore, it is possible to use does_match
and match_exn
to avoid the allocation.
- Added
Version_util.build_info_as_sexp
. - Added
_squelch_unused_module_warning_
to Comparable.S.Replace_polymorphic_compare
.