FreeBSD now has NUMA? Why’d it take so long?

Adrian Chadd, known for his extensive WiFI work, writes about his findings of NUMA (non-uniform memory access) in FreeBSD.


I just committed “NUMA” to FreeBSD. Well, no, I didn’t. I did almost no actual NUMA-y work in FreeBSD. I just exposed the existing NUMA stuff in FreeBSD out and re-enabled it.

FreeBSD-9 introduced basic NUMA awareness in the physical allocator (sys/vm/vm_phys.c.) It implemented first-touch page allocation, and then fell back to searching through the domains, round-robin style. It wasn’t perfect, for some workloads it was apparently okay. But it had some shortcomings – it wasn’t configurable, UMA and other subsystems didn’t know about NUMA domains, and the scheduler really didn’t know about NUMA domains. So I’m sure there are plenty of workloads which it didn’t work for.

That was all ripped out before FreeBSD-10. FreeBSD-10 NUMA just implements round-robin physical page allocation. It still tracks the per-domain physical memory regions, but it doesn’t do any kind of NUMA aware allocation. From what I can gather, it was removed until something ‘better’ would land.

However, nothing (yet) has landed. So I decided I’d take a look into it. I found that for a lot of simple workloads (ie, where you’re doing lots of anonymous memory allocation – eg, you’re doing math crunching) the FreeBSD-9 model works fine. It’s also a perfectly good starting point for experimenting.

So all my NUMA work in -HEAD does is provide an API to exactly the above. It doesn’t teach the kernel APIs about domain aware allocations – there’s currently no way to ask for memory from a specific domain when calling UMA, or contigmalloc, etc. The scheduler doesn’t know about NUMA, so threads/processes will migrate off-socket very quickly unless you explicitly limit things. Devices don’t yet do NUMA local work – the ACPI code is in there to enumerate which NUMA domain they’re in, but it’s not used anywhere just yet.

Then what is it good for?

[Read more…]

Ethereum on Different Operating Systems (self.ethereum)

FreeBSD user /u/aedigix shows us how to get Ethereum set up on FreeBSD. Ethereum is a virtualized platform for decentralized applications.



For go-ethereum If you install go 1.4 you can just do the following:

git clone
cd go-ethereum && git checkout tags/<latest stable> && make geth 

For cpp-ethereum/solidity compiler This is a little bit more involved but I am able to build on FreeBSD 10 and 11 using the following steps:

Dependencies which should be available in Fedora’s yum repo, FreeBSD/Gentoo ports, etc….

llvm35 gcc48 cmake boost-all-1.55.0 ncurses automake libtool unzip gmp leveldb yasm miniupnpc readline curl cryptopp libmicrohttpd libjson++ argtable opencl ode npm doxygen gmake cmake

On FreeBSD I have to build jsoncpp and libjson-rpc-cpp manually with the following steps:


git clone
cd jsoncpp && git checkout tags/1.6.2
mkdir build && cd build
cmake -DCMAKE_CXX_FLAGS="-fPIC" ..
gmake install


cd libjson-rpc-cpp
git checkout tags/v0.5.0
mkdir build && cd build
cmake -DCMAKE_CXX_FLAGS="-fPIC" ..
gmake install

Then build cpp-ethereum:

git clone
cd cpp-ethereum && git checkout develop
mkdir build && cd build
cmake -DFATDB=1 -DUSENPM=1 -DETHASHCL=1 -DEVMJIT=1 -DLLVM_DIR=/usr/local/lib/llvm-3.5/share/llvm/cmake ..
gmake -j8

FreeBSD 10.2-BETA2 Now Available

The developers of FreeBSD have made available version 10.2-BETA2. See below for the documented changes.

Download link:



The second BETA build of the 10.2-RELEASE release cycle is now

Installation images are available for:

o amd64 GENERIC
o i386 GENERIC
o ia64 GENERIC
o powerpc GENERIC
o powerpc64 GENERIC64
o sparc64 GENERIC

The image checksums follow at the end of this email.

FreeBSD/arm SD card images are available for:


Note:  For convenience for those without console access to supported arm
devices, a default 'freebsd' user exists for ssh(1) login.  The password
is 'freebsd', which it is strongly recommended to change after gaining
access to the system.  Additionally, the 'root' user password is 'root',
which is also recommended to change.

[Read more…]

How To Find Out FreeBSD Version and Patch Level Number

This short tutorial by user Vivek Gite (nixCraft) shows us how to figure out your FreeBSD version and patch level number.


How do I find out my FreeBSD server version and patch level of the installed system using command line option?

Tutorial details
Difficulty Easy (rss)
Root privileges No
Requirements None
Estimated completion time 1m

You can use any one of the following command to display the FreeBSD operating system version and patch level:a) freebsd-version command – Show the version and patch level of the installed system

b) uname command – Show information about the system

Say hello to freebsd-version command

The freebsd-version command appeared in FreeBSD version 10.0. To see the version and patch level of the installed kernel:
freebsd-verion -k
To see the version and patch level of the installed userland:
freebsd-verion -u
Sample outputs:

Fig.01: freebsd-version and other commands in action
(Fig.01: freebsd-version and other commands in action)

You can see a system being repaired using a live CD:
mount -rt ufs /dev/ada0p2 /mnt
env ROOT=/mnt /mnt/bin/freebsd-version -ku

You can also use the following command to find out if FreeBSD kernel is running in 32bit or 64bit mode:
# getconf LONG_BIT

Sample outputs:

Old good uname command

Type the following command (it works on all versions of FreeBSD):
umame -a
uname -mrs

FreeBSD 10.1-RELEASE-p10 amd64

Using bhyve on FreeBSD

This tutorial by will show you how to setup and use bhyve on FreeBSD. bhyve is a BSD hypervisor and virtual machine manager which relies on features such as Extended Page Tables and VirtIO network/storage drivers.


freebsd_bhyveI wanted to write an article about bhyve for a long time now, and fortunately I recently had the time to do just that. Bhyve has the potential to became a very sophisticated, and advanced hypervisor.

1. Introduction

I’m not sure since when, but bhyve now supports libvirt. That’s great news, given that libvirt is a universal tool which helps in managing VMs. There are quite a few hypervisors compatible with libvirt, for example KVM, Xen, Linux containers (LXC) and a bunch of others. On the other hand, there are only a few graphical fronteds compatible with it, for example Virtual Machine Manager (virt-manager), which is a general purpose VM management tool. This can be installed to a computer and the hypervisors can be managed from there. Virt-manager is generally good for a few hypervisors, with 10-20 VMs, but managing a cloud infrastructure is quite a challenge with it. There a few tools compatible with libvirt, which are targeted at clouds or greater deployment.

In this article, I will cover the Virtual Machine Manager (virt-manager). This tool allows for the basic management of VMs, such as installing, starting, deleting etc. The goal of this document is to introduce the installation and configuration of bhyve to a FreeBSD host, libvirt, and the virt-manager to a remote computer, then to explain how you can connect from the virt-manager to the remote host. This setup allows the management of VMs remotely. This article is customized for CentOS guest installation as a guest, however other Linux guests could be installed and used in a similar way.

As its current state, libvirt is not fully compatible with bhyve and therefore interacting with VMs with libvirt is not possbile. However, this article provides instructions for installing and managing VMs with bhyve itself.

2. Additional informations for this article

[Read more…]

BSDCan 2015 Trip Report

bsdcan2015FreeBSD developers Christian Brueffer and Warren Block summarize their experiences at BSDCan 2015, which was held at the University of Ottawa in Ontario.  The two were sponsored by the FreeBSD Foundation to attend developer summits and presentations, gathered by developers around the world. Head on over to the links to view the full reports.

Christian Brueffer:

All in all, the conference and the devsummit were a great success for me.  I learned a lot, and it’s always astonishing how seeing the cool stuff other people have been working on motivates me to work on FreeBSD better myself.  Thank you to the FreeBSD Foundation for making this trip possible!


Warren Block:

BSDCan is not what most people expect.  It is not a boring computer conference.  Sure, there are presentations and talks and the standard conference stuff.  But this is a place where BSD nerds are the norm, not the exception; where the other people there speak your language, share and understand your problems, and know that you understand theirs.  It is a feeling of family, a chance to share and solve problems, and gather inspiration for the rest of the year.


Using ZFS replication features in FreeBSD to improve offsite backups

This tutorial by user iceflatline shows us how to improve offsite backups by using ZFS replication features in FreeBSD.


Recently I decided to improve the reliability of my file system backups by using the data replication capabilities inherent in the FreeBSD Zettabyte File System (ZFS). ZFS provides a built-in serialization feature that can send a stream representation of a ZFS file system (Which ZFS refers to as a “dataset”) to standard output. Using this technique, it is possible to not only store the dataset(s) on another ZFS storage pool (zpool) connected to the local system, but also to send it over a network to another FreeBSD system. ZFS dataset snapshots serve as the basis for this replication, and the essential ZFS commands used for replicating the data are zfs send and zfs receive.

This post describes how I used this ZFS feature to perform replication of ZFS dataset snapshots from my home FreeBSD server to another FreeBSD machine located offsite. I’ll also discuss how I manage the quantity of snapshots stored locally and offsite, as well as a couple of options for recovering my files should it become necessary.

For purposes of example, I’ll refer to the FreeBSD system hosting the snapshots I want to send as “server”, and the offsite FreeBSD system that I will send snapshots to as “backup”. Unless otherwise noted, all steps were performed as the user root. However a non-root user, “iceflatline”, was created on both machines and is used for many of the commands. The versions for the software used in this post were as follows:

  • FreeBSD 10.1-RELEASE


Configure server

On server I had created a simple mirror vdev for my zpool consisting of (2) two terabyte disks. The mirror and the zpool were created using the following commands:

[Read more…]

The importance of mentoring, or “how I got involved in FreeBSD”..

Adrian Chadd, known for his extensive WiFi work and research in FreeBSD, recounts his humble beginnings of FreeBSD from youth — starting with meeting Julian Elischer.


adrianchaddHere’s how I was introduced into this UNIX world, or “wait, WHO was your WHAT?”

So, here’s 11ish or so year old Adrian. It’s the early 90s. I was hiding in my bedroom, trying to make another crystal set out of random parts and scraping away the paint at my windowsill. In walks my Aunty, who introduces her new boyfriend.

“Hi, I’m Julian.” he said. That wasn’t all that interesting.

“Oh, are you making a crystal set?” .. ok, so that was interesting.

And, that was that. Suddenly, someone role-model-y shows up in my life out of the blue. There I was, an 11 year old who felt very mostly alone most of the time, and someone shows up who I can look up to and think I can relate to. So, I’m a sponge for everything he shows me. Whenever he comes over, he has some new story to tell, some new thing to show me. He would show me better ways of building transistor switch circuits when I was in the “make large arcs with car alternator” phase of my early teens. And, when I saved up and bought a PC, he started to show me programming.

Now, I was already programming. My parents had saved up and bought me an Amstrad CPC464. We had a second-hand commodore 64 for a short while, but that eventually somehow stopped working and I didn’t have the clue to fix it. But I was programming Locomotive BASIC and dabbling in Z80 assembly when I was 12, and had “upgraded” to Turbo Pascal 6 when I hit high school. (Yes, school taught Turbo Pascal at Grade 10 level, and I decided to learn it a bit earlier. That’s .. wow, that dates me.) I hadn’t yet really stumbled into C yet. I had heard about it, but I didn’t have anything that could write it.

[Read more…]