Using daemontools (supervise) on FreeBSD

What is daemontools?

From the daemontools website:

daemontools is a collection of tools for managing UNIX services.

supervise monitors a service. It starts the service and restarts the service if it dies. Setting up a new service is easy: all supervise needs is a directory with a run script that runs the service.

At first when I was introduced to this tool at work, I thought “What a typical Linux-admin. FreeBSD’s rc. system is superior.” Despite my personal preferences, whatever software is used at work is what I have to use and learn to use, too. After getting a little more familiar with supervise, and installing it on a FreeBSD server, I was finally convinced that this may also have a place on FreeBSD machines.

Have you ever needed to know that a process is 100% sure to be running no matter what? Well, some of our applications need that extra little safety net, and you might too. Just right of the bat I can mention things like httpd, sshd, denyhosts, and syslog(-ng). While the theoretical risk of these applications crashing randomly and still being able to run again without any direct editing of some configuration file seems to be very low, in a production environment where loads are extremely high and all processes are pushed into a stage where their theoretical load-handling capacity is on the edge with what has practically been tested, this may happen to you – and you can’t afford the service being down until you figure out a way to fix it permanently.

Either way, if some application crashes in a recoverable manner, it’s most likely that either 1) supervise is still running and will try to revive the process or 2) your box is so broken, it doesn’t even matter that supervise is still running. It’s all about that extra little factor of reliability.

Convinced? Here’s the walkthrough

Portsnap; 3 years old. Happy birthday

Three years ago today, Colin Percival announced the availability of the first version of FreeBSD Portsnap. Almost a year later, in August 2005, Portsnap was added to the FreeBSD base system, and since then it has grown to four official mirrors and now supports almost 40 thousand users. Happy birthday Portsnap!

CVSup is slow, insecure, and a memory hog. However, until now it’s been the only option for keeping an up-to-date ports tree, and (thanks to all of the recent work on vuxml and portaudit) it has become quite obvious that keeping an up-to-date ports tree is very important.

To provide a secure, lightweight, and fast alternative to CVSup, I’ve written portsnap. As the name suggests, this is a system for building, *signing*, and distributing compressed snapshots of the ports tree, which can then be extracted into /usr/ports as needed.

Portsnap is:

  • Lightweight. It’s a 15kB shell script which uses under 50kB of other binaries.
  • Designed for frequent updating. Unlike CVSup, it doesn’t need to transmit a complete list of files in the ports tree each time it runs; in fact, if there are no updates available, it only needs to fetch a single file of 256 bytes.
  • Secure. Using code from FreeBSD Update, the ports snapshots are signed using a 2048-bit RSA key.
  • HTTP-only. That’s right, you don’t need to beg your network maintainer to allow outgoing connections on port 5999 any more. :-)

FreeBSD handbook chapter on how to use portsnap can be found here.

Absolute FreeBSD – Auction

Absolute BSDThe FreeBSD Foundation is pleased to announce the kicking off of the 2007 Fall Fundraising campaign by auctioning off the first copy of the book Absolute FreeBSD, 2nd Edition. You can be the first one to own this book, while helping the FreeBSD Project and community. This book was generously donated by Michael Lucas, the author, and he will include a signed authentic laser-printed Certificate of Authenticity, and a signed bookplate.To bid on this phenomenal guide to FreeBSD go to:

All proceeds will go to the Foundation. If you’re not interested in bidding on the book you can still support the FreeBSD Foundation by donating.


pfSense hackathon wrap up

The third annual pfSense hackathon has been a great success. There was a lot of cleaning up code and cleaning up the many new features that are already in the development branches, rather than adding more new features. This leaves pfSense in a better position to get out future releases.

FreeBSD 7 – a taste of things to come

Kris Kennaway from the FreeBSD project has created an interesting 37-page PDF showing some of the good things to come with FreeBSD 7.0. (including some much promising graphs!). Especially the speed of MySQL on a FreeBSD server is impressive. It now even beats Linux!

This is the table of contents:

Introducing FreeBSD 7.0

Part I: The SMPng project: A 7 year journey

  • Multi processor support, old and new: FreeBSD 4.x
  • Multi processor support, old and new: The SMPng project
  • SMPng and the Universal Development Model
  • SMPng, step 1: First, make it work; FreeBSD5.x
  • SMPng, step 2: Then make it work well; FreeBSD 6.x
  • SMPng, step 3: Then make it fast; FreeBSD7.0
  • A case study: SQL database performance
  • FreeBSD PostgreSQL performance: 5.5, 6.2 and 7.0
  • Performance of PostgreSQL
  • FreeBSD7.0: Scaling with varying number of CPUs
  • FreeBSD MySQL performance: 5.5, 6.2 and 7.0
  • Understanding MySQL performance
  • FreeBSD vs other operating systems: PostgreSQL
  • FreeBSD vs other operating systems: MySQL
  • Comments onother operating systems

Part II: New features debuting in FreeBSD7.0

  • Filesystem and storage subsystem changes
  • Storage subsystem changesNetwork stack changes
  • Network drivers
  • Wireless (802.11) ; much improved in 7.0
  • New CPU architectures
  • Security subsystems
  • User-level changes
  • Growth of FreeBSD Ports Collection
  • Performance
  • Other kernel changes
  • Developer tools/internals

Part III: What the future holds for FreeBSD

FreeBSD 7.0 brings FreeBSD back to the forefront of OS performance on modern hardware (it’s good to be back).

These and some other features can also be found on the “What’s cooking for FreeBSD 7 page

FreeBSD 7.0-BETA1 Available

FreeBSD Logo (big)Now that FreeBSD 7.0-beta1 has been released, the final stage of the FreeBSD-7.0 Release cycle has begun. The FreeBSD 7.0-BETA1 ISO images for Tier-1 architectures are now available for download on most of the FreeBSD mirror sites. The more people that test and report bugs, the better FreeBSD 7.0-RELEASE will be. For more information about the FreeBSD 7.0 release process, please check the official schedule and the todo list.

Those of you that are confident enough to try beta versions, please test this beta version and report any problems to the developers. The following is the official announcement to the FreeBSD Current mailinglist:

“We have entered the final phases of the FreeBSD-7.0 Release cycle which also means the beginning of the FreeBSD-6.3 Release cycle. Because the people who support the ports for FreeBSD also need to go through a freeze cycle as part of releases we had decided to combine the two releases to try and minimize the impact on the ports maintainers.

The current plan is to interleave the BETAs/RCs of the 7.0 and 6.3 releases, trying to follow this for the dates when the builds will get started (with them becoming available on the FTP mirrors a day or two after the builds start):

Version – 7.0 – 6.3
BETA1 – 10/17 – 10/24
BETA2 – 10/31 – 11/7
RC1 – 11/14 – 11/21
RC2 – 11/28 – 12/5
REL – 12/12 – 12/19

Tomorrow (10/23) the RELENG_6 branch will be marked “6.3-PRERELEASE” to note that we have entered the 6.3 release cycle.

The schedule dates are, as usual, tentative. At this point RELENG_6 is pretty mature so that schedule should be fairly accurate. Being a new branch it is at least somewhat likely the dates for 7.0 will wind up slipping.

The 7.0-BETA1 builds have completed and are on many of the FreeBSD mirror sites. If you want to update an existing machine using cvsup use RELENG_7 as the branch tag. Instructions on using FreeBSD Update to perform a binary upgrade from FreeBSD 6.x to 7.0-BETA1 will be provided via the freebsd-stable list when available.

BSDConTR – Turkey BSD conference

The BSD Conference in Turkey is over now. Diomidis Spinellis, a FreeBSD committer since 2003, wrote about the conference on his blog

This has been a wonderful experience. The conference’s location, the Rectorate of the Marmara University, (SultanAhmet, Istanbul), was impressive, and near many even more remarkable sights, the guesthouse of the University were extremely convenient, and the conference’s program was had many interesting talks. The translators did a superb job translating from and to Turkish. Our hosts organized our day to the last detail, taming successfully the difficulties we encountered from the city’s sometimes chaotic traffic and the daunting (to us) public transport.

Although I’ve been a (fairly undistinguished) FreeBSD committer since May 2003, at the conference I met for the first time face to face other fellow members of the FreeBSD project: Pawel Dawidek, the person behind the port of the ZFS file system to FreeBSD, and Kris Kennaway, who presented the impressive progress that the release of FreeBSD 7.0 represents.

Read the whole post here: International BSD Conference in Turkey

DesktopBSD weekly snapshots (dev)

The release cycle of DesktopBSD is rather slow, since the developers spend a lot of time making sure the release is almost bug-free. For those who are always excited about trying the latest and greatest features, DBSD provides now weekly snapshot ISOs. They are built every Saturday from the latest DesktopBSD Tools, the most recent FreeBSD 6-STABLE sources and an up-to-date ports collection. The ISO contains a live system that can be booted without installing first, an installer that copies the operating system to your hard disk and a large selection of packages for most of your every-day needs.

For now, the snapshots are only available for the AMD64 architecture, but i386 snapshots will soon folow. You can download the ISO files here.