package async_unix

bytes_read t returns how many bytes have been read by this reader from the underlying file descriptor. This includes bytes that have not been read by the user from the reader yet (sitting in the internal buffer).

Returns time of the most recent read system call that returned data.

stdin is a reader for file descriptor 0. It is lazy because we don't want to create it in all programs that happen to link with Async.

open_file file opens file for reading and returns a reader reading from it.

This may raise an exception for the typical reasons that an open(2) system call may fail. If it does raise, it's guaranteed to be a Unix_error variant.

transfer t pipe_w transfers data from t into pipe_w one chunk at a time (whatever is read from the underlying file descriptor without post-processing). The result becomes determined after reaching EOF on t and the final bytes have been transferred, or if pipe_w is closed.

This function will normally not be needed (see pipe).

pipe t returns the reader end of a pipe that will continually be filled with chunks of data from the underlying Reader.t. When the reader reaches EOF or the pipe is closed, pipe closes the reader, and then after the reader close is finished, closes the pipe.

of_pipe info pipe_r returns a reader t that receives all the data from pipe_r. If pipe_r is closed, t will see an EOF (but will not be automatically closed). If t is closed, then pipe_r will stop being drained.

of_pipe is implemented by shuttling bytes from pipe_r to the write-end of a Unix pipe, with t being attached to the read end of the Unix pipe.

create ~buf_len fd creates a new reader that is reading from fd.

buf_len specifies the initial size of the internal buffer, which is the most data the Reader will buffer from the file descriptor (and consequently, the most that the Reader will read from a single read syscall). This buffer will be automatically increased in size if more data is requested from functions like read_one_chunk_at_a_time, peek, or {peek,read}_bin_prot. Note that buffers at least 128 KiB in size will be allocated with mmap (see bigstring_unix_stubs.c); buffers smaller than that will go on the C heap directly, which can cause C heap fragmentation in programs that allocate lots of buffers, e.g. RPC servers, possibly resulting in unintuitive higher overall program memory usage.

with_file file f opens file, creates a reader with it, and passes the reader to f. It closes the reader when the result of f becomes determined, and returns f's result.

This may raise an exception for the typical reasons that an open(2) system call may fail. If it does raise before f is called, it's guaranteed to be a Unix_error variant.

Note: You need to be careful that all your IO is done when the deferred you return becomes determined. If for example you use with_file and call lines, make sure you return a deferred that becomes determined when the EOF is reached on the pipe, not when you get the pipe (because you get it straight away).

exclusive = true uses a filesystem lock to try and make sure that the file is not read while it's being modified. This is an advisory lock, which means that the writer must be cooperating by taking a relevant lock when writing (see Writer.with_file). This is unrelated and should not be confused with the O_EXCL flag in open systemcall. Note that the implementation uses Unix.lockf, which has known pitfalls. It's recommended that you avoid the exclusive flag in favor of using a library dedicated to dealing with file locks where the pitfalls can be documented in detail.

See create for the meaning of buf_len.

close t prevents further use of t and closes t's underlying file descriptor. The result of close becomes determined once the underlying file descriptor has been closed. It is an error to call other operations on t after close t has been called, except that calls of close subsequent to the original call to close will return the same deferred as the original call.

close_finished t becomes determined after t's underlying file descriptor has been closed, i.e., it is the same as the result of close. close_finished differs from close in that it does not have the side effect of initiating a close.

is_closed t returns true iff close t has been called.

with_close t ~f runs f (), and closes t after f finishes or raises.

id returns a name for this reader that is unique across all instances of the reader module.

fd returns the Fd.t used to create this reader.

read t ?pos ?len buf reads up to len bytes into buf, blocking until some data is available or EOF is reached. The resulting i satisfies 0 < i <= len.

peek t ~len peeks exactly len bytes from t's buffer. It blocks until len bytes are available or EOF is reached.

Reports how many bytes of data are currently in the reader's buffer.

Consumes data from the reader's buffer without performing any additional I/O.

Reads up to len bytes from the reader's buffer without consuming it and without performing any additional I/O.

drain t reads and ignores all data from t until it hits EOF, and then closes t.

read_one_chunk_at_a_time t ~handle_chunk reads into t's internal buffer, and whenever bytes are available, applies handle_chunk to them. It waits to read again until the deferred returned by handle_chunk becomes determined. If handle_chunk returns `Consumed, then read_one_chunk_at_a_time will wait for additional data to arrive before calling handle_chunk again. Thus, handle_chunk should consume as much as possible.

read_one_chunk_at_a_time continues reading until it reaches `Eof or handle_chunk returns `Stop or `Stop_consumed. In the case of `Stop and `Stop_consumed, one may read from t after read_one_chunk_at_a_time returns.

`Stop a or `Continue respects the usual Iobuf semantics where data up to the Iobuf.Lo_bound is considered consumed.

read_one_iobuf_at_a_time is like read_one_chunk_at_a_time, except that the user-supplied handle_chunk function receives its data in an Iobuf.t, and uses the Iobuf position to communicate how much data was consumed. read_one_iobuf_at_a_time is implemented as a wrapper around read_one_chunk_at_a_time.

read_substring t ss reads up to Substring.length ss bytes into ss, blocking until some data is available or EOF is reached. The resulting i satisfies 0 < i <= Substring.length ss.

really_read t buf ?pos ?len reads until it fills len bytes of buf starting at pos, or runs out of input. In the former case it returns `Ok. In the latter, it returns `Eof n where n is the number of bytes that were read before end of input, and 0 <= n < String.length ss.

read_until t pred ~keep_delim reads until it hits a delimiter c such that:

• if pred = `Char c' then c = c'
• if pred = `Pred p then p c

`Char c' is equivalent to `Pred (fun c -> c = c') but the underlying implementation is more efficient, in particular it will not call a function on every input character.

read_until returns a freshly-allocated string consisting of all the characters read and optionally including the delimiter as per keep_delim.

read_until_bounded is just like read_until, except you have the option of specifying a maximum number of chars to read (not including the separator).

When `Max_exceeded str is returned, the length of str will be equal to max + 1.

read_line t reads up to and including the next newline (\n) character (or \r\n) and returns a freshly-allocated string containing everything up to but not including the newline character. If read_line encounters EOF before the newline char then everything read up to but not including EOF will be returned as a line.

really_read_line ~wait_time t reads up to and including the next newline (\n) character and returns an optional, freshly-allocated string containing everything up to but not including the newline character. If really_read_line encounters EOF before the newline char, then a time span of wait_time will be used before the input operation is retried. If the descriptor is closed, None will be returned.

read_sexp t reads the next sexp.

read_sexps t reads all the sexps and returns them as a pipe. When the reader reaches EOF or the pipe is closed, read_sexps closes the reader, and then after the reader close is finished, closes the pipe.

read_bin_prot ?max_len t bp_reader reads the next binary protocol message using binary protocol reader bp_reader. The format is the "size-prefixed binary protocol", in which the length of the data is prefixed as a 64-bit integer to the data. This is the format that Writer.write_bin_prot writes.

For higher performance, consider iter_bin_prot or read_bin_prot_into_pipe.

Similar to read_bin_prot, but doesn't consume any bytes from t.

iter_bin_prot t bp_reader ~f reads all size-prefixed binary protocol messages in t using binary protocol reader bp_reader, calling f for each message. Closes t after reaching EOF or an error.

Similar to iter_bin_prot, but returns messages as a pipe. The reader will be closed when EOF is reached or when the pipe is closed.

read_marshal_raw reads and returns a buffer containing one marshaled value, but doesn't unmarshal it. You can just call Marshal.from_string on the string, and cast it to the desired type (preferably the actual type). Similar to Marshal.from_channel, but suffers from the String-length limitation (16MB) on 32-bit platforms.

read_marshal is like read_marshal_raw, but unmarshals the value after reading it.

recv returns a string that was written with Writer.send.

read_all t read_one returns a pipe that receives all values read from t by repeatedly using read_one t.

When close_when_finished is true, the reader is automatically closed when it reaches EOF. Otherwise, the caller is responsible for closing the reader.

The returned pipe is closed when the reader reaches EOF.

lseek t offset ~mode clears t's buffer and calls Unix.lseek on t's file descriptor. The `Cur mode is not exposed because seeking relative to the current position of the file descriptor is not the same as seeking relative to the current position of the reader.

ltell t returns the file position of t from the perspective of a consumer of t. It uses Unix.lseek to find the file position of t's underlying file descriptor, and then subtracts the number of bytes in t's buffer that have been read from the OS but not from t.

lines t reads all the lines from t and puts them in the pipe, one line per pipe element. The lines do not contain the trailing newline. When the reader reaches EOF or the pipe is closed, lines closes the reader, and then after the reader close is finished, closes the pipe.

contents t returns the string corresponding to the full contents (up to EOF) of the reader. contents closes t before returning the string.

file_contents file returns the string with the full contents of the file.

file_lines file returns a list of the lines in the file. The lines do not contain the trailing newline.