Package Building Procedures
		Next

Package Building Procedures

The FreeBSD Ports Management Team

Revision: 43184

Last modified on 2013-11-13 by hrs.

[ Split HTML / Single HTML ]

Table of Contents

1. Introduction
2. Build Client Management
3. Jail Build Environment Setup
4. Customizing Your Build
5. Starting the Build
6. Anatomy of a Build
7. Build Maintenance
8. Monitoring the Build
9. Dealing With Build Errors
10. Release Builds
11. Uploading Packages
12. Experimental Patches Builds
13. How to configure a new package building node
14. How to configure a new FreeBSD branch
15. How to delete an unsupported FreeBSD branch
16. How to rebase on a supported FreeBSD branch
17. How to configure a new architecture
18. How to configure a new head node (pointyhat instance)
19. Procedures for dealing with disk failures

1. Introduction

In order to provide pre-compiled binaries of third-party applications for FreeBSD, the Ports Collection is regularly built on one of the “Package Building Clusters.” Currently, the main cluster in use is at http://pointyhat.FreeBSD.org.

This article documents the internal workings of the cluster.

Note:

Many of the details in this article will be of interest only to those on the Ports Management team.

1.1. The codebase

Most of the package building magic occurs under the /a/portbuild directory. Unless otherwise specified, all paths will be relative to this location. ${arch} will be used to specify one of the package architectures (e.g., amd64, arm, i386™, ia64, powerpc, SPARC64®), and ${branch} will be used to specify the build branch (e.g., 7, 7-exp, 8, 8-exp, 9, 9-exp, 10, 10-exp). The set of branches that portmgr currently supports is the same as those that the FreeBSD security team supports.

Note:

FreeBSD no longer builds packages for branches 4, 5, or 6, nor for the alpha architecture.

The scripts that control all of this live in either /a/portbuild/scripts/ or. /a/portbuild/admin/scripts/. These are the checked-out copies from the Subversion repository at base/projects/portbuild/ .

Typically, incremental builds are done that use previous packages as dependencies; this takes less time, and puts less load on the mirrors. Full builds are usually only done:

right after release time, for the -STABLE branches
periodically to test changes to -CURRENT
for experimental ("exp-") builds

Packages from experimental builds are not uploaded.

1.2. Historical notes on the codebase

Until mid-2010, the scripts were completely specific to pointyhat.FreeBSD.org as the head (dispatch) node. During the summer of 2010, a significant rewrite was done in order to allow for other hosts to be head nodes. Among the changes were:

removal of the hard-coding of the string pointyhat
factoring out all configuration constants (which were previously scattered throughout the code) into configuration files (see below)
appending the hostname to the directories specified by buildid (this will allow directories to be unambigious when copied between machines.)
making the scripts more robust in terms of setting up directories and symlinks
where necessary, changing certain script invocations to make all the above easier

Note:

Also during this process, the codebase was migrated to the Subversion repository. For reference, the previous version may still be found in CVS.

1.3. Notes on privilege separation

As of January 2013, a rewrite is in progress to further separate privileges. The following concepts are introduced:

Server-side user portbuild assumes all responsiblity for operations involving builds and communicating with the clients. This user no longer has access to sudo.
Server-side user srcbuild is created and given responsiblity for operations involving both VCS operations and anything involving src builds for the clients. This user does not have access to sudo.
The server-side ports-arch users go away.
None of the above server-side users have ssh keys. Individual portmgr will accomplish all those tasks using ksu. (This is still work-in-progress.)
The only client-side user is also named portbuild and still has access to sudo for the purpose of managing jails.

		Next
		2. Build Client Management