minix/distrib/sets
David van Moolenbroek 6f3e0bcd3d MIB/libsys: support for remote MIB (RMIB) subtrees
Most of the nodes in the general sysctl tree will be managed directly
by the MIB service, which obtains the necessary information as needed.
However, in certain cases, it makes more sense to let another service
manage a part of the sysctl tree itself, in order to avoid replicating
part of that other service in the MIB service.  This patch adds the
basic support for such delegation: remote services may now register
their own subtrees within the full sysctl tree with the MIB service,
which will then forward any sysctl(2) requests on such subtrees to the
remote services.

The system works much like mounting a file system, but in addition to
support for shadowing an existing node, the MIB service also supports
creating temporary mount point nodes.  Each have their own use cases.
A remote "kern.ipc" would use the former, because even when such a
subtree were not mounted, userland would still expect some of its
children to exist and return default values.  A remote "net.inet"
would use the latter, as there is no reason to precreate nodes for all
possible supported networking protocols in the MIB "net" subtree.

A standard remote MIB (RMIB) implementation is provided for services
that wish to make use of this functionality.  It is essentially a
simplified and somewhat more lightweight version of the MIB service's
internals, and works more or less the same from a programmer's point
of view.  The most important difference is the "rmib" prefix instead
of the "mib" prefix.  Documentation will hopefully follow later.

Overall, the RMIB functionality should not be used lightly, for
several reasons.  First, despite being more lightweight than the MIB
service, the RMIB module still adds substantially to the code
footprint of the containing service.  Second, the RMIB protocol not
only adds extra IPC for sysctl(2), but has also not been optimized for
performance in other ways.  Third, and most importantly, the RMIB
implementation also several limitations.  The main limitation is that
remote MIB subtrees must be fully static.  Not only may the user not
create or destroy nodes, the service itself may not either, as this
would clash with the simplified remote node versioning system and
the cached subtree root node child counts.  Other limitations exist,
such as the fact that the root of a remote subtree may only be a
node-type node, and a stricter limit on the highest node identifier
of any child in this subtree root (currently 4095).

The current implementation was born out of necessity, and therefore
it leaves several improvements to future work.  Most importantly,
support for exit and crash notification is missing, primarily in the
MIB service.  This means that remote subtrees may not be cleaned up
immediately, but instead only when the MIB service attempts to talk
to the dead remote service.  In addition, if the MIB service itself
crashes, re-registration of remote subtrees is currently left up to
the individual RMIB users.  Finally, the MIB service uses synchronous
(sendrec-based) calls to the remote services, which while convenient
may cause cascading service hangs.  The underlying protocol is ready
for conversion to an asynchronous implementation already, though.

A new test set, testrmib.sh, tests the basic RMIB functionality.  To
this end it uses a test service, rmibtest, and also reuses part of
the existing test87 MIB service test.

Change-Id: I3378fe04f2e090ab231705bde7e13d6289a9183e
2016-06-18 12:46:59 +00:00
..
lists MIB/libsys: support for remote MIB (RMIB) subtrees 2016-06-18 12:46:59 +00:00
attrs NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
checkflist Basic live rerandomization infrastructure 2015-09-17 17:15:03 +00:00
comments NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
culldeps Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
deps Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
descrs Split minix distribution set 2015-10-12 11:25:54 +02:00
getdirs.awk Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
join.awk NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
listpkgs Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
Makefile NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
makeflist Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
makeobsolete Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
makeplist Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
makesrctars NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
makesums Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
maketars NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
metalog.subr Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
mkvars.mk NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
README Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
regpkg NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
regpkgset Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
sets.subr NetBSD re-synchronization of the source tree 2016-01-13 20:32:14 +01:00
sort-list Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
syspkgdeps Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
TODO Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
versions Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00

# $NetBSD: README,v 1.13 2013/08/06 22:33:59 soren Exp $

the scripts should be run from the directory where they reside.

makeflist:	output the list of files that should be in a
		distribution, according to the contents of the
		'lists' directory.

checkflist:	check the file list (as internally generated
		by makeflist) against the tree living in $DESTDIR.
		(that tree should be made with 'make distribution'.)

maketars:	make tarballs of the various sets in the distribution,
		based on the contents of the lists, the tree in
		$DESTDIR, and put the tarballs in $RELEASEDIR.
		Note that this script _doesn't_ create the 'secr'
		distribution, because (for now) it requires
		manual intervention to get the binaries right...
		(i'll add another script to create that dist, later.)

what's in 'lists':

lists describing file sets.  There are two sets of lists per file
set: machine dependent and machine-independent files. (there's
also another file in the 'man' dir, which is used by the 'man'
and 'misc' sets, but that's explained later.)

There is one machine-independent file, named "mi".  There are
N machine-dependent files (one per architecture), named "md.${ARCH}".

the sets are as follows:

	base:	the base binary set.  excludes everything described
		below.

	comp:	compiler tools.  All of the tools relating to C, C++,
		and FORTRAN (yes, there are two!) that are in the
		tree.  This includes includes, the linker, tool chain,
		and the .a versions of the libraries.  (obviously,
		base includes ldd, ld.so, and the shared versions.
		base also includes 'cpp', because that's used by X11.)
		includes the man pages for all the binaries contained
		within.  Also, includes all library and system call
		manual pages.

	debug:	Debugging libraries (_g.a/MKDEBUGLIB) and (.debug/MKDEBUG)
		binaries.

	etc:	/etc, and associated files (/var/cron/tabs, /root,
		etc.).  things that shouldn't be blindly reinstalled
		on an upgrade.

	games:	the games and their man pages.

	man:	all of the man pages for the system, except those
		listed elsewhere (e.g. in comp, games, misc, text).
		Includes machine-dependent man pages for this CPU.

	misc:	share/dict, share/doc, and the machine-dependent
		man pages for other CPUs which happen to always
		be installed.

	modules:	stand/${MACHINE}/${OSRELEASE}/modules kernel modules

	tests:	unit, regression, integration and stress tests for the
		whole system.

	text:	text processing tools.  groff and all of its friends.
		includes man pages for all bins contained within.

Each set must contain "./etc/mtree/set.<set name>" within the mi
list.  Failure to add this will break unprivileged builds.