As /usr/pkg will be completely deleted at the start of bulk builds, make sure your login shell is placed somewhere else. Either drop it into /usr/local/bin (and adjust your login shell in the passwd file), or (re-)install it via pkg_add(1) from /etc/rc.local, so you can login after a reboot (remember that your current process won't die if the package is removed, you just can't start any new instances of the shell any more). Also, if you use NetBSD earlier than 1.5, or you still want to use the pkgsrc version of ssh for some reason, be sure to install ssh before starting it from rc.local:

    ( cd /usr/pkgsrc/security/ssh ; make bulk-install )
    if [ -f /usr/pkg/etc/rc.d/sshd ]; then
        /usr/pkg/etc/rc.d/sshd
    fi

Not doing so will result in you being not able to log in via ssh after the bulk build is finished or if the machine gets rebooted or crashes. You have been warned! :)

Make sure you don't need any of the packages still installed.

Be sure to remove all other things that might interfere with builds, like some libs installed in /usr/local, etc. then become root and type:

# cd /usr/pkgsrc
# sh mk/bulk/build

If for some reason your last build didn't complete (power failure, system panic, ...), you can continue it by running:

# sh mk/bulk/build restart

At the end of the bulk build, you will get a summary via mail, and find build logs in the directory specified by FTP in the build.conf file.

The bulk builds consist of three steps:

During the build, a list of broken packages will be compiled in /usr/pkgsrc/.broken (or .../.broken.${MACHINE} if OBJMACHINE is set), individual build logs of broken builds can be found in the package's directory. These files are used by the bulk-targets to mark broken builds to not waste time trying to rebuild them, and they can be used to debug these broken package builds later.

Currently, roughly the following requirements are valid for NetBSD 2.0/i386:

Note that all pkgs will be de-installed as soon as they are turned into a binary package, and that sources are removed, so there is no excessively huge demand to disk space. Afterwards, if the package is needed again, it will be installed via pkg_add(1) instead of building again, so there are no cycles wasted by recompiling.

If you don't want all the packages nuked from a machine (rendering it useless for anything but pkg compiling), there is the possibility of doing the package bulk build inside a chroot environment.

The first step is to set up a chroot sandbox, e.g. /usr/sandbox. This can be done by using null mounts, or manually.

There is a shell script called pkgsrc/mk/bulk/mksandbox which will set up the sandbox environment using null mounts. It will also create a script called sandbox in the root of the sandbox environment, which will allow the null mounts to be activated using the sandbox mount command and deactivated using the sandbox umount command.

To set up a sandbox environment by hand, after extracting all the sets from a NetBSD installation or doing a make distribution DESTDIR=/usr/sandbox in /usr/src/etc, be sure the following items are present and properly configured:

When the chroot sandbox is set up, you can start the build with the following steps:

# cd /usr/sandbox/usr/pkgsrc
# sh mk/bulk/do-sandbox-build

This will just jump inside the sandbox and start building. At the end of the build, mail will be sent with the results of the build. Created binary pkgs will be in /usr/sandbox/usr/pkgsrc/packages (wherever that points/mounts to/from).

In addition to building a complete set of all packages in pkgsrc, the pkgsrc/mk/bulk/build script may be used to build a subset of the packages contained in pkgsrc. By setting SPECIFIC_PKGS in /etc/mk.conf, the variables

will define the set of packages which should be built. The bulk build code will also include any packages which are needed as dependencies for the explicitly listed packages.

One use of this is to do a bulk build with SPECIFIC_PKGS in a chroot sandbox periodically to have a complete set of the binary packages needed for your site available without the overhead of building extra packages that are not needed.

This section describes how pkgsrc developers can upload binary pkgs built by bulk builds to ftp.NetBSD.org.

If you would like to automatically create checksum files for the binary packages you intend to upload, remember to set MKSUMS=yes in your mk/bulk/build.conf.

If you would like to PGP sign the checksum files (highly recommended!), remember to set SIGN_AS=username@NetBSD.org in your mk/bulk/build.conf. This will prompt you for your GPG password to sign the files before uploading everything.

Then, make sure that you have RSYNC_DST set properly in your mk/bulk/build.conf file, i.e. adjust it to something like one of the following:

RSYNC_DST=ftp.NetBSD.org:/pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch/upload 

Please use appropriate values for "pkgsrc-200xQy", "NetBSD-a.b.c" and "arch" here. If your login on ftp.NetBSD.org is different from your local login, write your login directly into the variable, e.g. my local account is "feyrer", but for my login "hubertf", I use:

RSYNC_DST=hubertf@ftp.NetBSD.org:/pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch/upload 

A separate upload directory is used here to allow "closing" the directory during upload. To do so, run the following command on ftp.NetBSD.org next:

nbftp% mkdir -p -m 750 /pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch/upload

Please note that /pub/NetBSD/packages is only appropriate for packages for the NetBSD operating system. Binary packages for other operating systems should go into /pub/pkgsrc.

Before uploading the binary pkgs, ssh authentication needs to be set up. This example shows how to set up temporary keys for the root account inside the sandbox (assuming that no keys should be present there usually):

# chroot /usr/sandbox
chroot-# rm $HOME/.ssh/id-dsa*
chroot-# ssh-keygen -t dsa
chroot-# cat $HOME/.ssh/id-dsa.pub 

Now take the output of id-dsa.pub and append it to your ~/.ssh/authorized_keys file on ftp.NetBSD.org. You can remove the key after the upload is done!

Next, test if your ssh connection really works:

chroot-# ssh ftp.NetBSD.org date 

Use "-l yourNetBSDlogin" here as appropriate!

Now after all this works, you can exit the sandbox and start the upload:

chroot-# exit
# cd /usr/sandbox/usr/pkgsrc
# sh mk/bulk/do-sandbox-upload 

The upload process may take quite some time. Use ls(1) or du(1) on the FTP server to monitor progress of the upload. The upload script will take care of not uploading restricted packages and putting vulnerable packages into the vulnerable subdirectory.

After the upload has ended, first thing is to revoke ssh access:

nbftp% vi ~/.ssh/authorized_keys
Gdd:x! 

Use whatever is needed to remove the key you've entered before! Last, move the uploaded packages out of the upload directory to have them accessible to everyone:

nbftp% cd /pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/arch
nbftp% mv upload/* .
nbftp% rmdir upload
nbftp% chmod 755 . 

Complete documentation for cdpack is found in the cdpack(1) man page. The following short example assumes that the binary packages are left in /usr/pkgsrc/packages/All and that sufficient disk space exists in /u2 to hold the ISO 9660 images.

# mkdir /u2/images
# pkg_add /usr/pkgsrc/packages/All/cdpack
# cdpack /usr/pkgsrc/packages/All /u2/images

If you wish to include a common set of files (COPYRIGHT, README, etc.) on each CD in the collection, then you need to create a directory which contains these files. e.g.

# mkdir /tmp/common
# echo "This is a README" > /tmp/common/README
# echo "Another file" > /tmp/common/COPYING
# mkdir /tmp/common/bin
# echo "#!/bin/sh" > /tmp/common/bin/myscript
# echo "echo Hello world" >> /tmp/common/bin/myscript
# chmod 755 /tmp/common/bin/myscript

Now create the images:

# cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images

Each image will contain README, COPYING, and bin/myscript in their root directories.