1 / 30

FreeBSD Release Engineering: Code Review, CVS Operations, and Release Building

Learn about the FreeBSD development model, release process, code review, and release building techniques, including terminology, branch points, and the release checklist.

gwalker
Download Presentation

FreeBSD Release Engineering: Code Review, CVS Operations, and Release Building

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BSDCon Europe November 11, 2001 FreeBSD Release Engineering Murray Stokely Wind River Systems murray@FreeBSD.org

  2. Outline • Introduction • Terminology • FreeBSD Development Model • Release Process Overview • Code Review • Release Checklist • CVS Operations • Release Building

  3. Outline • Extensibility • FreeBSD 4.4 • Future Directions

  4. SCM Terminology • SCM - Software Configuration Management • Branch -A variant codeline that is separate from the main line of development. • Merge - The process of integrating tested changes across branches. • Tag - An identifier for the code in an SCM system at a given point in time. • CVS - The spawn of satan that we're forced to use.

  5. FreeBSD Development Model • FreeBSD development continues along two parallel branches; FreeBSD-CURRENT and FreeBSD-STABLE. • FreeBSD-CURRENT is the main trunk of our CVS repository. All new development should happen here. • FreeBSD-STABLE is the branch from which major releases are made. Changes enter this branch at a different pace, and with the general assumption that they have been well tested by our user community running -CURRENT.

  6. FreeBSD Development Model Recent FreeBSD Branch Points

  7. FreeBSD Development Model • Thousands of developers around the world contribute code to FreeBSD. • Only the "committers" have write access to the CVS repository • 290 total committers in FreeBSD (includes source, documentation, and ports committers). • 197 active committers to src/ in the last 12 months. • 139 active committers to src/sys in the last 12 months. (active is loosely defined as making a single commit.)

  8. FreeBSD Development Model • Users and external developers should submit bug fixes, enhancements, or problem reports with GNATS. • Online submission forms, send-pr command installed by default on FreeBSD systems, etc..

  9. Open PRs vs Time

  10. Release Process Overview Releases are made from the -STABLE development branch at approximately 4 month intervals. • Reminder / Announcement email sent to developers <45 days> • MFC Sweeps ("Merge from -CURRENT") • Code slush / freeze <30 days> • Release Candidates <15 days> • Eventually a final release

  11. Release Checklist • Creating the Release Branch • Branch point tag $ cvs rtag -rRELENG_4 RELENG_4_4_BP src • Creating the branch $ cvs rtag -b -rRELENG_4_4_BP RELENG_4_4 src

  12. Bumping up the Version Number • Before the final release can be tagged, built, and released, the following files need to be modified to reflect the correct version of FreeBSD. • src/sys/conf/newvers.sh • src/sys/sys/param.h • src/release/doc/share/sgml/release.ent • src/gnu/usr.bin/groff/tmac/mdoc.local • doc/share/sgml/freebsd.ent • doc/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml

  13. Bumping up the Version Number (continued) • www/en/releases/* • src/UPDATING

  14. Creating the Final Release Tag • When the final release is ready, the following command will create the RELENG_4_4_0_RELEASE tag. $ cvs rtag -rRELENG_4_4 RELENG_4_4_0_RELEASE • The Documentation and Ports managers are responsible for tagging the respective trees with the RELEASE_4_4_0 tag. • "Sliding" tags after the fact : $ cvs tag -d tagname filename

  15. Release Building • Tools necessary for building a FreeBSD release are available in src/release/ • These tools aim to provide a consistent way to build FreeBSD releases. • A complete release can be built with only a single command, including the creation of ISO images suitable for burning onto CDROM, installation floppies, and an FTP install directory. This command is aptly named "make release"

  16. Release Building • To successfully build a release, you must first populate /usr/obj by running "make world" or simply "make buildworld". The release target requires that several variables be set properly : • CHROOTDIR - The directory to be used as the chroot environment for the entire release build. • BUILDNAME - The name of the release to be built. • CVSROOT - The location of a CVS repository. • RELEASETAG - The tag corresponding to the sources to build.

  17. "make release" (1/4) There are many other variables that can be used to customize the release build process. Most of these are documented at the top of src/release/Makefile. The exact command used to build FreeBSD 4.4 (x86) was : make release CHROOTDIR=/local3/release \ BUILDNAME=4.4-RELEASE CVSROOT=/host/cvs/usr/home/ncvs RELEASETAG=RELENG_4_4_0_RELEASE

  18. "make release" (2/4) • The release makefile can be broken down into several distinct steps. • Creation of a sanitized system environment in a separate directory hierarchy with "make installworld". • Checkout from CVS of a clean version of the system source, documentation, and ports into the release build hierarchy. • Population of /etc and /dev in the chrooted environment. • chroot into the release build environment, to make it harder for the outside world to taint the build.

  19. "make release" (3/4) • Steps of "make release" (continued) • "make world" in the chrooted environment. • Build Kerberos-related binaries. • Build 'GENERIC' kernel. • Creation of a staging directory tree where the binary distributions will be built and packaged. • Build and installation of the documentation toolchain needed to convert the documentation source (SGML) into the HTML and text documents that will accompany the release.

  20. "make release" (4/4) Steps of "make release" (continued) • Build and install of the actual documentation (user manuals, release notes, tutorials, hardware compatibility lists, etc..) • Build of the "crunched" binaries to be used on the installation floppies • Creation of the distribution tarballs of the binaries and sources. • Create the boot media and a fixit floppy. • Create the FTP installation hierarchy. • (optionally) Create ISO images for CDROM media.

  21. Ports and Packages • The FreeBSD Ports Collection is a collection of nearly 6,000 third-party software packages available for FreeBSD. • The ports team (portmgr@FreeBSD.org) is responsible for maintaining a consistent ports tree that can be used to create the binary packages that accompany a given FreeBSD release.

  22. Ports and Packages • In order to provide a consistent set of third-party packages, every port is built in a separate chroot environment, starting with an empty /usr/local and /usr/X11R6. • The requisite dependencies are installed as packages before the build proceeds. • By starting the package build in a pristine environment, we can assure that the package metadata (such as required dependencies) are accurate, and so we will never generate packages that might work on some systems and not on others depending on what software was previously installed.

  23. Ports Build Cluster • The "Ports Cluster" for the x86 architecture currently consists of a master node (Dual Pentium III 733 Mhz) and 8 slave nodes (Pentium III 800Mhz) to do the actual package builds. With this configuration, a complete package build takes over 24 hours. • The Ports Cluster for the Alpha architecture consists of 7 PWS 500A machines donated by Compaq. • The clusters for both architectures are co-located with the other FreeBSD Project equipment on Yahoo's campus in Santa Clara, California.

  24. Ports and Packages • For FreeBSD 4.4, over 4.1 gigabytes of packages were created. This causes a problem for CDROM distributions because we would obviously like to ship as many packages as possible without making the user insert another disc to satisfy dependencies. • The solution is to put "clusters" of like packages with similar dependencies onto specific discs. • The package split is performed by Steve Price and the ports team in coordination with the general wishes of the general user community with respect to which packages get to appear on the first CD.

  25. FreeBSD CDROMs • Starting with FreeBSD 4.4, the FreeBSD Project decided to release all four ISO images that were previously sold on the BSDi/WRS "official" CDROM distributions. • Each of the four discs must contain a README.TXT, CDROM.INF, and filename.txt file. • The file manifest can be created with : find . -type f | sed -e 's/\^.\///' | sort > filename.txt

  26. Disc #1 • Output from "make release" with the following additions : • Addition of a 'tools' directory to aid would-be new users working in other operating systems. • XFree86 • As many software packages as will fit.

  27. Disc #2 • Also largely created by "make release". • Contains a "live filesystem" that can be used from sysinstall to troubleshoot a FreeBSD installation. • Contains a compressed copy of the CVS repository in the CVSROOT directory • Contains commercial software demos (such as Perforce, Accelerated-X, etc..) in the commerce directory. • Discs #3 and 4 simply contain as many additional packages as will fit.

  28. Extensibility • A release can be customized in many different ways. • If you have access to the staging directory of an existing "make release" build hierarchy then you may find it easiest to apply patches or additional files as necessary to the chroot build directory. rm ${CHROOTDIR}/usr/obj/usr/src/release/release.[48] • Rebuild sysinstall, the kernel, or whatever parts of the system your change affected. chroot ${CHROOTDIR} ./mk release.4 release.8

  29. FreeBSD 4.4 • Code slush began on August 1, 2001. After which time all commits to the RELENG_4 branch (-STABLE) had to be approved by re@FreeBSD.org • The first release candidate for the x86 architecture was released on August 16. • 4 more release candidates were released leading up to the final release on September 18th. • Over 500 emails sent to re@FreeBSD.org in little over a month.

  30. Future Work • More documentation • Regression Testing • Installation Tools

More Related