This new implementation of the UDS service is built on top of the
libsockevent library. It thereby inherits all the advantages that
libsockevent brings. However, the fundamental restructuring
required for that change also paved the way for resolution of a
number of other important open issues with the old UDS code. Most
importantly, the rewrite brings the behavior of the service much
closer to POSIX compliance and NetBSD compatibility. These are the
most important changes:
- due to the use of libsockevent, UDS now supports multiple suspending
calls per socket and a large number of standard socket flags and
options;
- socket address matching is now based on <device,inode> lookups
instead of canonized path names, and socket addresses are no longer
altered either due to canonization or at connect time;
- the socket state machine is now well defined, most importantly
resolving the erroneous reset-on-EOF semantics of the old UDS, but
also allowing socket reuse;
- sockets are now connected before being accepted instead of being
held in connecting state, unless the LOCAL_CONNWAIT option is set
on either the connecting or the listening socket;
- connect(2) on datagram sockets is now supported (needed by syslog),
and proper datagram socket disconnect notification is provided;
- the receive queue now supports segmentation, associating ancillary
data (in-flight file descriptors and credentials) with each segment
instead of being kept fully separately; this is a POSIX requirement
(and needed by tmux);
- as part of the segmentation support, the receive queue can now hold
as many packets as can fit, instead of one;
- in addition to the flags supported by libsockevent, the MSG_PEEK,
MSG_WAITALL, MSG_CMSG_CLOEXEC, MSG_TRUNC, and MSG_CTRUNC send and
receive flags are now supported;
- the SO_PASSCRED and SO_PEERCRED socket options are replaced by
LOCAL_CREDS and LOCAL_PEEREID respectively, now following NetBSD
semantics and allowing use of NetBSD libc's getpeereid(3);
- memory usage is reduced by about 250 KB due to centralized in-flight
file descriptor tracking, with a limit of OPEN_MAX total rather than
of OPEN_MAX per socket;
- memory usage is reduced by another ~50 KB due to removal of state
redundancy, despite the fact that socket path names may now be up to
253 bytes rather than the previous 104 bytes;
- compared to the old UDS, there is now very little direct indexing on
the static array of sockets, thus allowing dynamic allocation of
sockets more easily in the future;
- the UDS service now has RMIB support for the net.local sysctl tree,
implementing preliminary support for NetBSD netstat(1).
Change-Id: I4a9b6fe4aaeef0edf2547eee894e6c14403fcb32
The getnucred() function was used by UDS to obtain credentials of user
processes in a form used in the UDS API, namely the ucred structure.
Since the NetBSD merge, this structure has changed drastically (aside
from being renamed to "uucred"), and it is no longer in UDS's best
interest to use this structure internally. Therefore, getnucred() is
no longer a useful API either, and instead we directly use the
previously private getepinfo() function to obtain credentials.
Change-Id: I80bc809de716ec0a9b7497cb109d2f2708a629d5
This library provides an event-based abstraction model and dispatching
facility for socket drivers. Its main goal is to eliminate any and
all need for socket drivers to keep track of pending socket calls.
Additionally, this library takes over responsibility of a number of
other tasks that would otherwise be duplicated between socket drivers,
but in such a way that individual socket drivers retain a large degree
of freedom in terms of API behavior. The library's main features are:
- suspension, resumption, and cancellation of socket calls;
- an abstraction layer for select(2);
- state tracking of shutdown(2);
- pending (asynchronous) errors and the SO_ERROR socket option;
- listening-socket tracking and the SO_ACCEPTCONN socket option;
- generation of SIGPIPE signals; SO_NOSIGPIPE, MSG_NOSIGNAL;
- send and receive low-watermark tracking, SO_SNDLOWAT, SO_RCVLOWAT;
- send and receive timeout support and SO_SNDTIMEO, SO_RCVTIMEO;
- an abstraction layer for the SO_LINGER socket option;
- tracking of various on/off socket options as well as SO_TYPE;
- a range of pre-checks on socket calls that are required POSIX.
In order to track per-socket state, the library manages an opaque
"sock" object for each socket. The allocation of such objects is left
entirely to the socket driver. Each sock object has an associated
callback table for calls from libsockevent to the socket driver. The
socket driver can raise events on the sock object in order to flag
that any previously suspended operations of a particular type should
be resumed. The library may defer processing such raised events if
immediate processing could interfere with internal consistency.
The sockevent library is layered on top of libsockdriver, and should
be used by all socket driver implementations if at all possible.
Change-Id: I3eb2c80602a63ef13035f646473360293607ab76
This library provides abstractions for socket drivers, and should be
used as the basis for all socket driver implementations. It provides
the following functionality:
- a function call table abstraction, hiding the details of the
socket driver protocol with simple parameters and presenting the
socket driver with callback functions very similar to the BSD
socket API calls made from userland;
- abstracting data structures and helper functions for suspending
and resuming blocking calls;
- abstracting data structures and helper functions for copying data
from and to the caller.
Overall, the library is similar to lib{block,char,fs,input,net}driver
in concept. Some of the abstractions provided here should in fact be
applied to libchardriver as well. As always, for the case that the
provided message loop is too restrictive, a set of more low-level
message processing functions is provided.
Change-Id: I79ec215f5e195c3b0197e223636f987d3755fb13
A pair of manual pages were already present in /usr/share/man, but
not yet installed. Install them as well. Lots and lots more from
NetBSD's set of manual pages should be imported, though.
Change-Id: Ie2e8946967afcb2e71de563f06fa331586dcb31d
IMPORTANT: this change has a docs/UPDATING entry!
This patch performs an initial import of the infrastructure and a
subset of the NetBSD set of rc startup and shutdown scripts. The
"initial" refers to the fact that this is not yet a full switch to the
NetBSD rc system: the MINIX ramdisk rc script, which (typically) runs
as the first thing, is kept as is. After mounting the root file
system, the ramdisk rc script will start the NetBSD rc infrastructure
by invoking /etc/rc, however. The regular MINIX startup-and-shutdown
script has been moved from /etc/rc to /etc/rc.minix, and is now
invoked as part of the NetBSD rc infrastructure through a bridge rc
script /etc/rc.d/minixrc. /etc/rc.minix invokes /usr/etc/rc as before.
Switching over the ramdisk to the NetBSD system and decomposing the
MINIX rc.minix script into smaller components are left to future work.
Also, the current pkgsrc etc/rc.d auto-start functionality is left as
is, even though it should be removed (see the etc/usr/rc comment).
Change-Id: Ia96cae7c426e94b85c67978dc1307dacc4b09fc5
This requires importing a few files from mail(1) already. Importing
the rest of mail(1) is left to future work.
Change-Id: If96513a306245cd7fb64660758d0dbd29a36e87c
IMPORTANT: this change has a docs/UPDATING entry!
This rename is unfortunately necessary because NetBSD has decided to
create its own service(8) utility, and we will want to import theirs
as well. The two can obviously not coexist.
Also move ours from /bin to /sbin, as it is a superuser-only utility.
Change-Id: Ic6e46ffb3a84b4747d2fdcb0d74e62dbea065039
IMPORTANT: this change has a docs/UPDATING entry!
This change is a long overdue switch-over from the old MINIX set of
user and group accounts to the NetBSD set. This switch-over is
increasingly important now that we are importing more and more
utilities from NetBSD, several of which expect various user accounts
to exist. By switching over in one go, we save ourselves various
headaches in the long run, even if the switch-over itself is a bit
painful for existing MINIX users.
The newly imported master.passwd and group files have three exceptions
compared to their NetBSD originals:
1. There is a custom "service" account for MINIX 3 services. This
account is used to limit run-time privileges of various system
services, and is not used for any files on disk. Its user ID may
be changed later, but should always correspond to whatever the
SERVICE_UID definition is set to.
2. The user "bin" has its shell set to /bin/sh, instead of NetBSD's
/sbin/nologin. The reason for this is that the test set in
/usr/tests/minix-posix will not be able to run otherwise.
3. The group "operator" has been set to group ID 0, to match its old
value. This tweak is purely for transitioning purposes: as of
writing, pkgsrc packages are still using root:operator as owner and
group for most installed files. Sometime later, we can change back
"operator" to group ID 5 without breaking anything, because it does
not appear that this group name is used for anything important.
Change-Id: I689bcfff4cf7ba85c27d1ae579057fa3f8019c68
This small change makes it easier to do sorts without having to deal
with these entries over and over again.
Change-Id: Id5077a17733fa4b535cdc9881109286335d3cb17
This was a MINIX3-specific header file placed outside of the minix/
header subdirectory, with its definitions duplicated in the more
standard minix/sysutil.h header.
Also make env_prefix(3) take constant pointers.
Change-Id: I243c38eb38e24eb98f0c0dddf7f340e7fec255f4
Without this file, the NetBSD userland will fall back by default to the
old, insecure classic UNIX password hashing algorithm.
This is a big security issue. Please check docs/UPDATING for details.
Change-Id: Ib85646ee4678f91384bab238426ee55ff26da011
The way these options work is by creating files that contain debugging
symbols and stashing them in a dedicated set. The minix-debug set has
been created for this purpose, but it will probably have to be refined
since it has been tested only with the default options with an i386
cross-build.
LSC: Amended to support many combination of MKDEBUG, MKDEBUGLIB, with
and without X11, for both intel and arm.
Change-Id: I2901952e8229938f9ac79c8656484acf704ccd9b
Most of the nodes in the general sysctl tree will be managed directly
by the MIB service, which obtains the necessary information as needed.
However, in certain cases, it makes more sense to let another service
manage a part of the sysctl tree itself, in order to avoid replicating
part of that other service in the MIB service. This patch adds the
basic support for such delegation: remote services may now register
their own subtrees within the full sysctl tree with the MIB service,
which will then forward any sysctl(2) requests on such subtrees to the
remote services.
The system works much like mounting a file system, but in addition to
support for shadowing an existing node, the MIB service also supports
creating temporary mount point nodes. Each have their own use cases.
A remote "kern.ipc" would use the former, because even when such a
subtree were not mounted, userland would still expect some of its
children to exist and return default values. A remote "net.inet"
would use the latter, as there is no reason to precreate nodes for all
possible supported networking protocols in the MIB "net" subtree.
A standard remote MIB (RMIB) implementation is provided for services
that wish to make use of this functionality. It is essentially a
simplified and somewhat more lightweight version of the MIB service's
internals, and works more or less the same from a programmer's point
of view. The most important difference is the "rmib" prefix instead
of the "mib" prefix. Documentation will hopefully follow later.
Overall, the RMIB functionality should not be used lightly, for
several reasons. First, despite being more lightweight than the MIB
service, the RMIB module still adds substantially to the code
footprint of the containing service. Second, the RMIB protocol not
only adds extra IPC for sysctl(2), but has also not been optimized for
performance in other ways. Third, and most importantly, the RMIB
implementation also several limitations. The main limitation is that
remote MIB subtrees must be fully static. Not only may the user not
create or destroy nodes, the service itself may not either, as this
would clash with the simplified remote node versioning system and
the cached subtree root node child counts. Other limitations exist,
such as the fact that the root of a remote subtree may only be a
node-type node, and a stricter limit on the highest node identifier
of any child in this subtree root (currently 4095).
The current implementation was born out of necessity, and therefore
it leaves several improvements to future work. Most importantly,
support for exit and crash notification is missing, primarily in the
MIB service. This means that remote subtrees may not be cleaned up
immediately, but instead only when the MIB service attempts to talk
to the dead remote service. In addition, if the MIB service itself
crashes, re-registration of remote subtrees is currently left up to
the individual RMIB users. Finally, the MIB service uses synchronous
(sendrec-based) calls to the remote services, which while convenient
may cause cascading service hangs. The underlying protocol is ready
for conversion to an asynchronous implementation already, though.
A new test set, testrmib.sh, tests the basic RMIB functionality. To
this end it uses a test service, rmibtest, and also reuses part of
the existing test87 MIB service test.
Change-Id: I3378fe04f2e090ab231705bde7e13d6289a9183e
We do not support any PF functionality, nor do we intend to. However,
some NetBSD utilities rely on the presence of these files. Not all of
the files are installed. The NetBSD source seems rather inconsistent
in where from to include these files. We simply follow what NetBSD
does, though.
Change-Id: Ib244dfcc60b16ebc4697af22f71b7e014374b855
While still a small subset of the NetBSD headers, this set should
allow various additional utilities to be compiled without too many
MINIX3-specific changes (even if those utilities will not yet work).
Change-Id: Idc70e9901d584e960cd406f75f561dcc9a4ddb7d
Some functions in lib/libc/net were disabled on MINIX3 only, but with
a few added header files they build just fine, even though some of
them rely on system functionality that has not yet been implemented.
Since the functionality is unlikely to be used in practice (because
it typically requires the use of protocol families that themselves are
not yet supported, such as IPv6), already enabling it right now helps
in building packages that rely on the functionality being present at
compile time, while not posing any practical risk of breaking the same
packages at run time.
Change-Id: Idee8e3963c9e300bde9575429f0e77b0565acaef
This patch aims to synchronize the basic process user and group ID
management, as well as the set[ug]id(2) and sete[ug]id(2) behavior,
with NetBSD. As it turns out, the main issue was missing support for
saved user and group IDs. This support is now added.
Since NetBSD's userland, which we are importing, may rely on NetBSD
specifics when it comes to security, we choose not to deviate from
NetBSD's behavior in any way here. A new test, test89, verifies the
correct behavior - it has been confirmed to pass on NetBSD as is.
Change-Id: I023935546d97ed01ffd8090f7793d336cceb0f4a
If this directory doesn't exist, pid files are not created, which create
issues when shutting down or rebooting.
Change-Id: I52dddb57aca4368b1775606e22818fba99d05bf6
A few MINIX3-specific changes are necessary due to the fact that we
are missing the System V IPC message queue system calls.
Change-Id: Idd252984be9df69618cef79bcf6c676cbf915d85
- rewrite the semop(2) implementation so that it now conforms to the
specification, including atomicity, support for blocking more than
once, range checks, but also basic fairness support;
- fix permissions checking;
- fix missing time adjustments;
- fix off-by-one errors and other bugs;
- do not allocate dynamic memory for GETALL/SETALL;
- add test88, which properly tests the semaphore functionality.
Change-Id: I85f0d3408c0d6bba41cfb4c91a34c8b46b2a5959
Due to differences in (mainly) measuring and accumulating CPU times,
the two top programs end up serving different purposes: the NetBSD
top is a system administration tool, while the MINIX3 top (now mtop)
is a performance debugging tool. Therefore, we keep both.
The newly imported BSD top has a few MINIX3-specific changes. CPU
statistics separate system time from kernel time, rather than kernel
time from time spent on handling interrupts. Memory statistics show
numbers that are currently relevant for MINIX3. Swap statistics are
disabled entirely. All of these changes effectively bring it closer
to how mtop already worked as well.
Change-Id: I9611917cb03e164ddf012c5def6da0e7fede826d
No changes except for one cosmetic adjustment: NetBSD has chosen to
rename the standard TT column to TTY and not shorten tty names; we
undo those changes, making ps(1) behave more in accordance with the
specification and its manual page, and, most importantly for us, not
use an incredibly wide TTY column to print "console".
Change-Id: I3b3c198762f3eacf1b8e500557a803c1fedf2a61
Adapt libc devname(3) to make use of it, so that such device name
queries are now several orders of magnitude faster. The database
is created and updated at system bootup time.
Change-Id: I0cbcb24c7d47577d4d6af9c8290c21ee4df9a0ff
Imported with no changes, but not all parts are expected to be
functional. The libc nlist functionality is enabled for the
purpose of successful linking, although the nlist functionaly has
not been tested on MINIX3 nor is it needed for how we use libkvm.
In terms of function calls: kvm_getproc2, kvm_getargv2,
kvm_getenvv2, and kvm_getlwps are expected to work, whereas
kvm_getproc, kvm_getargv, kvm_getenvv, and kvm_getfiles are not.
Change-Id: I7539209736f1771fc0b7db5e839d2df72f5ac615
The new MIB service implements the sysctl(2) system call which, as
we adopt more NetBSD code, is an increasingly important part of the
operating system API. The system call is implemented in the new
service rather than as part of an existing service, because it will
eventually call into many other services in order to gather data,
similar to ProcFS. Since the sysctl(2) functionality is used even
by init(8), the MIB service is added to the boot image.
MIB stands for Management Information Base, and the MIB service
should be seen as a knowledge base of management information.
The MIB service implementation of the sysctl(2) interface is fairly
complete; it incorporates support for both static and dynamic nodes
and imitates many NetBSD-specific quirks expected by userland. The
patch also adds trace(1) support for the new system call, and adds
a new test, test87, which tests the fundamental operation of the
MIB service rather thoroughly.
Change-Id: I4766b410b25e94e9cd4affb72244112c2910ff67
