Table of Contents
This chapter gives a detailed description on how a package
is built. Building a package is separated into different
phases (for example
fetch, build, install), all of which are described in the
following sections. Each phase is splitted into so-called
stages, which take the
name of the containing phase, prefixed by one of pre-, do- or
post-. (Examples are
pre-configure, post-build.) Most of the actual work is done
in the do-* stages.
The basic steps for building a program are always the same. First the program's source (distfile) must be brought to the local system and then extracted. After any pkgsrc-specific patches to compile properly are applied, the software can be configured, then built (usually by compiling), and finally the generated binaries, etc. can be put into place on the system.
Before outlining the process performed by the NetBSD package system in the next section, here's a brief discussion on where programs are installed, and which variables influence this.
The automatic variable PREFIX
indicates where all files of the final program shall be
installed. It is usually set to LOCALBASE (/usr/pkg), or CROSSBASE for pkgs in the
“cross” category. The
value of PREFIX needs to be put
into the various places in the program's source where paths
to these files are encoded. See Section 8.3,
“patches/*” and Section 16.3.1,
“Shared libraries - libtool” for more
details.
When choosing which of these variables to use, follow the following rules:
PREFIX always points to
the location where the current pkg will be installed.
When referring to a pkg's own installation path, use
“${PREFIX}”.
LOCALBASE is where all
non-X11 pkgs are installed. If you need to construct a
-I or -L argument to the compiler to find includes and
libraries installed by another non-X11 pkg, use
“${LOCALBASE}”.
X11BASE is where the
actual X11 distribution (from xsrc, etc.) is installed.
When looking for standard X11 includes (not
those installed by a pkg), use “${X11BASE}”.
X11-based packages are special in that they may be
installed in either X11BASE or LOCALBASE.
Usually, X11 packages should be installed under
LOCALBASE whenever
possible. Note that you will need to include
../../mk/x11.buildlink3.mk in
them to request the presence of X11 and to get the
right compilation flags.
Even though, there are some packages that cannot be
installed under LOCALBASE:
those that come with app-defaults files. These packages
are special and they must be placed under X11BASE. To accomplish this, set
either USE_X11BASE or
USE_IMAKE in your
package.
Some notes: If you need to find includes or
libraries installed by a pkg that has USE_IMAKE or USE_X11BASE in its pkg Makefile, you need to look in
both
${X11BASE} and ${LOCALBASE}. To force installation
of all X11 packages in LOCALBASE, the pkgtools/xpkgwedge package is
enabled by default.
X11PREFIX should be
used to refer to the installed location of an X11
package. X11PREFIX will be
set to X11BASE if
xpkgwedge is not installed, and to LOCALBASE if xpkgwedge is
installed.
If xpkgwedge is installed, it is possible to have
some packages installed in X11BASE and some in LOCALBASE. To determine the prefix of
an installed package, the EVAL_PREFIX definition can be used. It
takes pairs in the format “DIRNAME=<package>”, and the
make(1) variable
DIRNAME will be set to the
prefix of the installed package <package>, or
“${X11PREFIX}”
if the package is not installed.
This is best illustrated by example.
The following lines are taken from pkgsrc/wm/scwm/Makefile:
EVAL_PREFIX+= GTKDIR=gtk+
CONFIGURE_ARGS+= --with-guile-prefix=${LOCALBASE:Q}
CONFIGURE_ARGS+= --with-gtk-prefix=${GTKDIR:Q}
CONFIGURE_ARGS+= --enable-multibyte
Specific defaults can be defined for the packages
evaluated using EVAL_PREFIX, by using a definition of
the form:
GTKDIR_DEFAULT= ${LOCALBASE}
where GTKDIR
corresponds to the first definition in the EVAL_PREFIX pair.
Within ${PREFIX}, packages should install
files according to
hier(7), with the
exception that manual pages go into ${PREFIX}/man, not ${PREFIX}/share/man.
When building a package, a number of directories is used to store source files, temporary files, pkgsrc-internal files, and so on. These directories are explained here.
Some of the directory variables contain relative
pathnames. There are two common base directories for these
relative directories: PKGSRCDIR/PKGPATH is used for directories
that are pkgsrc-specific. WRKSRC
is used for directories inside the package itself.
PKGSRCDIRThis is an absolute pathname that points to the pkgsrc root directory. Generally, you don't need it.
PKGPATHThis is a pathname relative to PKGSRCDIR that points to the current
package.
WRKDIRThis is an absolute pathname pointing to the directory where all work takes place. The distfiles are extraced to this directory. It also contains temporary directories and log files used by the various pkgsrc frameworks, like buildlink or the wrappers.
WRKSRCThis is an absolute pathname pointing to the
directory where the distfiles are extracted. It is
usually a direct subdirectory of WRKDIR, and often it's the only
directory entry that isn't hidden. This variable may be
changed by a package Makefile.
You can run a particular phase by typing
make phase,
where phase is the
name of the phase. This will automatically run all phases
that are required for this phase. The default phase is
build, that is, when you run
make without
parameters in a package directory, the package will be built,
but not installed.
This will check if the file(s) given in the variables
DISTFILES and PATCHFILES (as defined in the package's
Makefile) are present on the local system in /usr/pkgsrc/distfiles. If they are not
present, an attempt will be made to fetch them using commands
of the form:
${FETCH_CMD} ${FETCH_BEFORE_ARGS} ${site}${file} ${FETCH_AFTER_ARGS}
where ${site} varies through several possibilities in
turn: first, MASTER_SITE_OVERRIDE is tried, then the
sites specified in either SITES_file if defined, else MASTER_SITES or PATCH_SITES, as applies, then finally the
value of MASTER_SITE_BACKUP. The
order of all except the first can be optionally sorted by the
user, via setting either MASTER_SORT_AWK or MASTER_SORT_REGEX.
After the distfile(s) are fetched, their checksum is generated and compared with the checksums stored in the distinfo file. If the checksums don't match, the build is aborted. This is to ensure the same distfile is used for building, and that the distfile wasn't changed, e.g. by some malign force, deliberately changed distfiles on the master distribution site or network lossage.
When the distfiles are present on the local system, they need to be extracted, as they usually come in the form of some compressed archive format.
By default, all DISTFILES are
extracted. If you only need some of them, you can set the
EXTRACT_ONLY variable to the
list of those files.
Extracting the files is usually done by a little program,
mk/scripts/extract, which already knows how
to extract various archive formats, so most likely you will
not need to change anything here. But if you need, the
following variables may help you:
EXTRACT_OPTS_{BIN,LHA,PAX,RAR,TAR,ZIP,ZOO}Use these variables to override the default options
for an extract command, which are defined in
mk/scripts/extract.
EXTRACT_USINGThis variable can be set to pax, tar
or an absolute pathname pointing to the command with
which tar archives should be extracted.
If the extract program doesn't serve your needs,
you can also override the EXTRACT_CMD variable, which holds the
command used for extracting the files. This command is
executed in the ${WRKSRC} directory. During execution of
this command, the shell variable extract_file holds the absolute pathname of
the file that is going to be extracted.
And if that still does not suffice, you can override the
do-extract target in the package
Makefile.
After extraction, all the patches named by the
PATCHFILES, those present in the
patches subdirectory of the package as well as in
$LOCALPATCHES/$PKGPATH (e.g. /usr/local/patches/graphics/png) are
applied. Patchfiles ending in .Z or .gz are uncompressed before they are
applied, files ending in .orig or .rej are ignored. Any special options to
patch(1) can be handed in
PATCH_DIST_ARGS. See Section 8.3,
“patches/*” for more details.
By default patch(1) is given special args to make it fail if the patches apply with some lines of fuzz. Please fix (regen) the patches so that they apply cleanly. The rationale behind this is that patches that don't apply cleanly may end up being applied in the wrong place, and cause severe harm there.
This is covered in Chapter 15, Tools needed for building or running.
Most pieces of software need information on the header files, system calls, and library routines which are available on the platform they run on. The process of determining this information is known as configuration, and is usually automated. In most cases, a script is supplied with the distfiles, and its invocation results in generation of header files, Makefiles, etc.
If the package contains a configure script, this can be
invoked by setting HAS_CONFIGURE
to “yes”. If the
configure script is a GNU autoconf script, you should set
GNU_CONFIGURE to
“yes” instead. What
happens in the configure phase is roughly:
.for d in ${CONFIGURE_DIRS}
cd ${WRKSRC} && cd ${d} && env ${CONFIGURE_ENV} \
${CONFIGURE_SCRIPT} ${CONFIGURE_ARGS}
.endfor
CONFIGURE_DIRS (default:
“.”) is a list of
pathnames relative to WRKSRC. In
each of these directories, the configure script is run with
the environment CONFIGURE_ENV
and arguments CONFIGURE_ARGS.
The variables CONFIGURE_ENV,
CONFIGURE_SCRIPT (default:
“./configure”) and
CONFIGURE_ARGS may all be
changed by the package.
If the program uses an Imakefile for configuration, the
appropriate steps can be invoked by setting USE_IMAKE to “yes”. (If you only want the package
installed in ${X11PREFIX} but
xmkmf not being run, set USE_X11BASE instead.)
For building a package, a rough equivalent of the following code is executed.
.for d in ${BUILD_DIRS}
cd ${WRKSRC} && cd ${d} && env ${MAKE_ENV} \
${MAKE_PROGRAM} ${BUILD_MAKE_FLAGS} \
-f ${MAKEFILE} ${BUILD_TARGET}
.endfor
BUILD_DIRS (default:
“.”) is a list of
pathnames relative to WRKSRC. In
each of these directories, MAKE_PROGRAM is run with the environment
MAKE_ENV and arguments
BUILD_MAKE_FLAGS. The variables
MAKE_ENV, BUILD_MAKE_FLAGS, MAKEFILE and BUILD_TARGET may all be changed by the
package.
The default value of MAKE_PROGRAM is “gmake” if USE_TOOLS contains “gmake”, “make” otherwise. The default value of
MAKEFILE is “Makefile”, and BUILD_TARGET defaults to “all”.
Once the build stage has completed, the final step is to install the software in public directories, so users can access the programs and files.
In the install phase, a rough equivalent of the following code is executed. Additionally, before and after this code, much magic is performed to do consistency checks, registering the package, and so on.
.for d in ${INSTALL_DIRS}
cd ${WRKSRC} && cd ${d} && env ${MAKE_ENV} \
${MAKE_PROGRAM} ${INSTALL_MAKE_FLAGS} \
-f ${MAKEFILE} ${BUILD_TARGET}
.endfor
The variable's meanings are analogous to the ones in the
build phase.
INSTALL_DIRS defaults to
BUILD_DIRS. INSTALL_TARGET is “install” by default, plus
“install.man” if
USE_IMAKE is defined.
In the install
phase, the following variables are useful. They are all
variations of the
install(1) command that
have the owner, group and permissions preset. INSTALL is the plain install command. The
specialized variants, together with their intended use,
are:
INSTALL_PROGRAM_DIRdirectories that contain binaries
INSTALL_SCRIPT_DIRdirectories that contain scripts
INSTALL_LIB_DIRdirectories that contain shared and static libraries
INSTALL_DATA_DIRdirectories that contain data files
INSTALL_MAN_DIRdirectories that contain man pages
INSTALL_PROGRAMbinaries that can be stripped from debugging symbols
INSTALL_SCRIPTbinaries that cannot be stripped
INSTALL_GAMEgame binaries
INSTALL_LIBshared and static libraries
INSTALL_DATAdata files
INSTALL_GAME_DATAdata files for games
INSTALL_MANman pages
Some other variables are:
INSTALLATION_DIRSA list of directories relative to PREFIX that are created by pkgsrc at
the beginning of the install phase. If this
variable is set, NO_MTREE=“yes” is assumed, which means that
the package claims to create all needed directories
itself before installing files to it. Therefore this
variable should only be set in Makefiles that are under control of
the package's author.
For any of the main targets described in the previous section, two auxiliary targets exist with “pre-” and “post-” used as a prefix for the main target's name. These targets are invoked before and after the main target is called, allowing extra configuration or installation steps be performed from a package's Makefile, for example, which a program's configure script or install target omitted.
Should one of the main targets do the wrong thing, and should there be no variable to fix this, you can redefine it with the do-* target. (Note that redefining the target itself instead of the do-* target is a bad idea, as the pre-* and post-* targets won't be called anymore, etc.) You will not usually need to do this.
If you did a make install and you noticed some file was not installed properly, you can repeat the installation with this target, which will ignore the “already installed” flag.
This target does a pkg_delete(1) in the current directory, effectively de-installing the package. The following variables can be used to tune the behaviour:
PKG_VERBOSEAdd a "-v" to the pkg_delete(1) command.
DEINSTALLDEPENDSRemove all packages that require (depend on)
the given package. This can be used to remove any
packages that may have been pulled in by a given
package, e.g. if make deinstall
DEINSTALLDEPENDS=1 is done in
pkgsrc/x11/kde, this is likely
to remove whole KDE. Works by adding
“-R” to
the
pkg_delete(1)
command line.
This target causes the current package to be updated
to the latest version. The package and all depending
packages first get de-installed, then current versions
of the corresponding packages get compiled and
installed. This is similar to manually noting which
packages are currently installed, then performing a
series of make
deinstall and make install (or whatever
UPDATE_TARGET is set to)
for these packages.
You can use the “update” target to resume package
updating in case a previous make update was interrupted
for some reason. However, in this case, make sure you
don't call make
clean or otherwise remove the list of
dependent packages in WRKDIR. Otherwise, you lose the
ability to automatically update the current package
along with the dependent packages you have
installed.
Resuming an interrupted make update will only work as long as the package tree remains unchanged. If the source code for one of the packages to be updated has been changed, resuming make update will most certainly fail!
The following variables can be used either on the
command line or in /etc/mk.conf to alter the behaviour
of make
update:
UPDATE_TARGETInstall target to recursively use for the
updated package and the dependent packages.
Defaults to DEPENDS_TARGET if set,
“install”
otherwise for make
update. e.g. make update
UPDATE_TARGET=package
NOCLEANDon't clean up after updating. Useful if you want to leave the work sources of the updated packages around for inspection or other purposes. Be sure you eventually clean up the source tree (see the “clean-update” target below) or you may run into troubles with old source code still lying around on your next make or make update.
REINSTALLDeinstall each package before installing
(making DEPENDS_TARGET). This may be
necessary if the “clean-update” target (see
below) was called after interrupting a running
make
update.
DEPENDS_TARGETAllows you to disable recursion and hardcode
the target for packages. The default is
“update”
for the update target, facilitating a recursive
update of prerequisite packages. Only set
DEPENDS_TARGET if
you want to disable recursive updates. Use
UPDATE_TARGET
instead to just set a specific target for each
package to be installed during
make
update (see above).
Clean the source tree for all packages that would
get updated if make
update was called from the current
directory. This target should not be used if the
current package (or any of its depending packages) have
already been de-installed (e.g., after calling
make
update) or you may lose some packages
you intended to update. As a rule of thumb: only use
this target before the first time you
run make
update and only if you have a dirty
package tree (e.g., if you used NOCLEAN).
If you are unsure about whether your tree is clean, you can either perform a make clean at the top of the tree, or use the following sequence of commands from the directory of the package you want to update (before running make update for the first time, otherwise you lose all the packages you wanted to update!):
#make clean-update#make clean CLEANDEPENDS=YES#make update
The following variables can be used either on the
command line or in /etc/mk.conf to alter the behaviour
of make
clean-update:
CLEAR_DIRLISTAfter make
clean, do not reconstruct the
list of directories to update for this package.
Only use this if make update
successfully installed all packages you wanted to
update. Normally, this is done automatically on
make
update, but may have been
suppressed by the NOCLEAN variable (see
above).
This target invokes pkg_info(1) for the current package. You can use this to check which version of a package is installed.
This target generates a README.html file, which can be viewed
using a browser such as www/mozilla or www/links. The generated files
contain references to any packages which are in the
PACKAGES directory on the
local host. The generated files can be made to refer to
URLs based on FTP_PKG_URL_HOST and FTP_PKG_URL_DIR. For example, if I
wanted to generate README.html files which pointed to
binary packages on the local machine, in the directory
/usr/packages, set FTP_PKG_URL_HOST=file://localhost and
FTP_PKG_URL_DIR=/usr/packages. The
${PACKAGES} directory and
its subdirectories will be searched for all the binary
packages.
Use this target to create a file README-all.html which contains a list
of all packages currently available in the NetBSD
Packages Collection, together with the category they
belong to and a short description. This file is
compiled from the pkgsrc/*/README.html files, so be
sure to run this after a make readme.
This is very much the same as the
“readme” target
(see above), but is to be used when generating a pkgsrc
tree to be written to a CD-ROM. This target also
produces README.html files, and can be made to
refer to URLs based on CDROM_PKG_URL_HOST and CDROM_PKG_URL_DIR.
This target shows which distfiles and patchfiles are
needed to build the package. (DISTFILES and PATCHFILES, but not patches/*)
This target shows nothing if the package is not installed. If a version of this package is installed, but is not the version provided in this version of pkgsrc, then a warning message is displayed. This target can be used to show which of your installed packages are downlevel, and so the old versions can be deleted, and the current ones added.
This target shows the directory in the pkgsrc hierarchy from which the package can be built and installed. This may not be the same directory as the one from which the package was installed. This target is intended to be used by people who may wish to upgrade many packages on a single host, and can be invoked from the top-level pkgsrc Makefile by using the “show-host-specific-pkgs” target.
This target shows which installed packages match the
current package's DEPENDS.
Useful if out of date dependencies are causing build
problems.
After a package is installed, check all its binaries
and (on ELF platforms) shared libraries to see if they
find the shared libs they need. Run by default if
PKG_DEVELOPER is set in
/etc/mk.conf.
After a “make
install” from a new or upgraded pkg, this
prints out an attempt to generate a new PLIST from a find -newer
work/.extract_done. An attempt is made
to care for shared libs etc., but it is strongly recommended to
review the result before putting it into PLIST. On upgrades, it's useful to
diff the output of this command against an already
existing PLIST file.
If the package installs files via
tar(1) or other
methods that don't update file access times, be sure to
add these files manually to your PLIST, as the “find -newer” command used by this
target won't catch them!
See Section 10.3, “Tweaking output of make print-PLIST” for more information on this target.
Used to do bulk builds. If an appropriate binary
package already exists, no action is taken. If not,
this target will compile, install and package it (and
its depends, if PKG_DEPENDS is set properly. See
Section 6.3.1,
“Configuration”). After creating the
binary package, the sources, the just-installed package
and its required packages are removed, preserving free
disk space.
Beware that this target may deinstall all packages installed on a system!
Used during bulk-installs to install required packages. If an up-to-date binary package is available, it will be installed via pkg_add(1). If not, make bulk-package will be executed, but the installed binary won't be removed.
A binary package is considered “up-to-date” to be installed via pkg_add(1) if:
None of the package's files (Makefile, ...) were modified
since it was built.
None of the package's required (binary) packages were modified since it was built.
Beware that this target may deinstall all packages installed on a system!