When renovating a building, it is common to do this floor by floor. In
this sense, Ocamlnet 3.0.0 focused on the foundation and the first
floor. Also, the renovation is not yet finished - many features still
need to be added, like supporting SSL for more protocols. This is now
easier thanks to some new basic APIs that have been introduced in the
first step.
Netsys
One of the parts that got most attention is Netsys, the
library adding the missing links to the operating system (OS). One of
the driving forces was the port to Win32. This lead to the introduction
of generalized versions of Unix.read and Unix.write
calls (defined in Netsys):
val gread : fd_style -> Unix.file_descr -> string -> int -> int -> int
val gwrite : fd_style -> Unix.file_descr -> string -> int -> int -> int
For getting some Win32-specific emulations right, it is sometimes
required to call other functions instead of
Unix.read
and
Unix.write, e.g.
Netsys_win32.pipe_read
and
Netsys_win32.pipe_write. In order to avoid that such
case distinctions are scattered over the whole library, the idea of
defining these generic functions was born. In
fd_style the
user passes in how to handle the
descriptor. Usually
fd_style is automatically determined
by another function
get_fd_style (this requires a few
system calls and is factored out because of this). Although targeting mostly
at Win32, there are already
some benefits for POSIX systems, e.g. the
fd_style already
encodes whether a descriptor is a socket, and whether it is connected,
which is sometimes quite useful information. In the future, this system
will be extended:
Seekable files are currently not well supported by the
asynchronous I/O layer. The reason is that the select
and poll system calls cannot predict whether I/O would be
blocking or non-blocking (and thus always say non-blocking).
This can be improved by using special AIO calls of the OS.
Of course, files for which AIO is to be used need to be
flagged specially, and a new fd_style could
do so.
There are also some ideas for labeling SSL sockets by a special
fd_style. This would make it a bit easier to
support SSL thoughout the library. This is a bit more work
than just calling Ssl.read and Ssl.write,
though, because the SSL protocol allows renegotiations at any
time, and a read may also require writes on the socket level,
and vice versa.
Another new idea on the
Netsys level is a little object
definition called
pollset:
class type pollset =
object
method find : Unix.file_descr -> Netsys_posix.poll_req_events
method add : Unix.file_descr -> Netsys_posix.poll_req_events -> unit
method remove : Unix.file_descr -> unit
method wait : float ->
( Unix.file_descr *
Netsys_posix.poll_req_events *
Netsys_posix.poll_act_events ) list
method dispose : unit -> unit
method cancel_wait : bool -> unit
end
A
pollset represents a set of file descriptor events one
wants to poll. Again, this data structure was originally required
for the Win32 port (because Win32 is very different in this respect),
but there are also advantages for Unix systems. Nowadays, there are
various improved APIs for polling such as Linux epoll or BSD kqueue.
The
pollset abstraction will make it very easy to support
these - the user simply selects one of the advanced implementations
of
pollset, and thanks to dynamic binding of object
methods it is automatically used everywhere. (One of the next
versions of Ocamlnet will allow this.)
Another word about polling. The Ocaml runtime only provides
select. Although not as bad as claimed by some people,
it imposes artificial limitations, especially about the number of
supported file descriptors. Because of this, Netsys_posix
includes now a binding of the poll system call which is not
suffering from this disease. Of course, poll is now the
only poll API used throughout Ocamlnet (and, as noted, even better
APIs will be supported in one of the next releases).
Other additions on the OS level for Unix systems:
Netsys_posix.spawn is a new way of starting subprograms,
with special support for monitoring the subprocesses asynchronously
- There are now bindings for syslog in
Netsys_posix
- The system calls
fsync and fdatasync are
supported
- If the OS provides this call,
fadvise can be invoked
to control the page cache
- There is also
fallocate to allocate disk space, so far
the OS provides it
- POSIX semaphores are supported, so far the OS provides the complete
interface (i.e. named semaphores for synchronization between unrelated
processes)
- There is a coordinator module for signals,
Netsys_signal,
so that various users of signals do not mutually override their handlers
For all systems, Netsys implements:
- Wrappers for multicasting system calls on sockets
- In
Netsys_mem there is now special support for
using bigarrays of chars as efficient I/O buffers. Such
bigarray-backed buffers are called memory (reminding us
to the fact that these buffers are not relocatable like strings, but
bound to fixed memory addresses). There are functions for allocating
page-aligned or cache-line-aligned memory buffers. Also,
there is experimental support for copying Ocaml values into buffers
(used by the Camlbox module, see below). Finally, there are also
versions of read, write, recv
and send operating on memory buffers rather than strings.
These versions open the door to zero-copy network I/O (if supported by
the OS).
- For better support of multi-threading there is now a version of
the thread API that even exists when the thread library is not linked in,
so that especially critical sections are emulated as no-ops in the
single-threaded case. It is hoped that more functions can be made
thread-safe by this new feature (in
Netsys_oothr).
- The exception registry
Netexn is now almost outdated,
because the Ocaml standard library recently introduced a similar
feature (yes, sometimes feature wishes are honoured :-).
Equeue
As Netsys uses now pollsets to manage
polling, Equeue had to be rewritten to take advantage of
this. In particular, there is now Unixqueue_pollset which
is a port of the old Unixqueue API around pollsets. For
the user, there is absolutely no difference.
What's more important is the extension of the engine API. Ocamlnet 2
introduced engines as a way of expressing a suspended I/O possibility,
but there was only limited support for it in the library. This has now
changed - engines are now a first class member of Ocamlnet. In particular,
there are now much more synchronization primitives (e.g.
stream_seq_engine for executing an open number of engines
in sequence, or msync_engine for waiting for the
completion of multiple engines). This development was mostly driven by
another project of mine: Plasma (see other blog articles on this
site). Plasma uses engines for all kinds of concurrent execution of
I/O code, and while I was developing Plasma, I extended the Ocamlnet
engine API step by step.
There is also now a way to call RPC procedures with an engine:
Rpc_proxy.ManagedClient.rpc_engine. This function has
originally also been developed for the Plasma project.
For simpler I/O needs, I added Uq_io. It contains
"engineered" versions of simple I/O functions like input,
input_line or flush. Uq_io is
not limited to file descriptors, but works also on top of a number
of other I/O devices (including virtual ones).
The operators ++ and >> have been
introduced as abbreviations for sequential execution, and result
mapping of engines, respectively. For example, the synchronous
code
let line1 = input_line ch_in in
let line2 = input_line ch_in in
output_string ch_out (line1 ^ line2 ^ "\n")
would now look in "engineered" code:
Uq_io.input_line_e d_in ++
(fun line1 ->
Uq_io.input_line_e d_in ++
(fun line2 ->
Uq_io.output_string_e d_out (line1 ^ line2 ^ "\n")
)
)
Not bad, if you compare with the previous solution (hand-coding a
scanner for lines, writing the event handler routines, etc., adding
up to 100-200 lines of code).
Netplex
The development in the
Netplex area was focused on easing
multi-processing. With
Netplex it is very easy to run
code in several worker processes, e.g. for network servers. What was
missing up to now, however, was an easy way to manage the
collaboration of the processes.
Netplex worker processes got now a number of ways to
talk to each other:
- It is now possible to store variables in a common place, so that
each process can get and set these (
Netplex_sharedvar).
Of course, this mechanism is typed.
- There are mutexes and semaphores for synchronization
(
Netplex_mutex and Netplex_semaphore)
- Each process can be directly contacted via a private channel,
the so-called container socket. This is also an RPC mechanism, but
unlike normal RPC servers the caller directly addresses a process
(and not only a service in general, and the Netplex machinery automatically
selects the destination process). There is also a directory so that
processes can see which other processes exist.
The implementation of these mechanisms is not yet optimal, but the APIs
are defined and backed by simple but robust modules. It is expected that
in the future more sophisticated implementations will become available,
e.g. the Netplex_sharedvar code use a shared memory object
if the OS supports that.
Another addition are "levers". This kind of handle exists within the
Netplex master process, but can be activated from the child processes.
It is a kind of little RPC function for a special purpose: Sometimes
the process model requires that certain functionality must be done
within the scope of the master process. An example would be the start
of another child process. By doing that via a lever, this action can
also be triggered from any child process.
Besides that there are numerous smaller enhancements. Especially
the module Netplex_cenv has been extended, e.g. there
are now timers that can be attached to the Netplex event queue.
RPC
The development went into two directions: First, it was aimed at a
more powerful RPC client implementation, and second, performance
performance performance.
The improved client is called Rpc_proxy. All experience
went in that I made at my Ocaml job at Mylife.com - lots of RPC calls
in an unreliable environment (if you have hundreds of machines, one
box is always down). Clients can now be recycled, they can react
better on errors, and even load balancing and fail-over to alternate
endpoints are now supported. (See the other blog posting, "The next
server, please!".)
Performance improvements were achieved by two means: First, the XDR
encoding and decoding was optimized. This has not yet come to an end
yet, but certain XDR types like arrays of strings are now processed a
lot faster. The other strategy was to replace many string buffers by
bigarrays of char (see under "memory" above). This allows it to get rid
of a number of copy operations, especially when large strings are
transmitted via RPC. This new string representation is even accessible
by user code via a new XDR type _managed string. This
may avoid even more copies.
Shell
The API of
Shell is mostly the same - only a few
suspicious functions have been removed. The implementation, however,
has changed a lot.
Shell now uses the new Netsys functions for
starting subprocesses. As these functions are written in C, one gets
some immediate benefits: Shell is now officially supported
for multi-threaded programs because it is possible to do the signal
handling right in C (but still, this is notoriously difficult). Also,
there is now no risk anymore that the Ocaml garbage collector wants to
clean up in the worst moment, namely between fork and exec.
Another benefit is that Shell works now also under Win32.
The C part is completely different, though.
Netcgi
Not much has changed here, only that the old version
Netcgi1
is gone now.
Camlboxes
An exciting but still experimental addition are Camlboxes. They are
designed as a fast way of sending messages between unrelated
processes. Camlboxes use shared memory for communication.
This works as follows: If process 1 want to send process 2 a message,
both have to map the same memory pages into their address space. The
message is orignally an Ocaml value somewhere in the private memory
of process 1. With the help of Camlbox this value is now copied to
shared memory so that, and this is the pivotal point, process 2 can
directly access the value without additional decoding step. This
reduces greatly the overhead of message sending - actually only a
relatively fast value copy is done, bypassing any kernel-controlled
I/O devices.
For passing a short message, this takes now only a few microseconds.
Most of that time is spent for synchronization, of course, not for
copying. (On the hardware level, the synchronization is mostly done by
moving cache lines from one CPU core to the other, so this is some
kind of hidden copying. It is worth noting that Camlboxes are way
faster on single-core machines than on multi-cores because this
low-level synchronization is not required then.)
Camlboxes have one downside, though. They are not perfectly integrated
into the garbage collecting machinery, and because of this, one has to
follow some programming rules. In particular, there is no way to
recognize that a message (or part of it) is no longer referenced, so
messages are manually deleted, and there is of course the danger that
bad code keeps references to (or into) deleted messages. For fixing
this, we would need more help by the Ocaml GC.
Another problem is missing integration with Equeue.
Camlboxes are synchronous by design - that's the price for their speed.
Where to get Ocamlnet 3
Look at the
project page for the newest version and links to the manual, mailing
list, etc.
