Since the conclusion of the SMPng project, the focus of SMP development in FreeBSD has shifted from deploying locking infrastructure to careful profiling and optimization of kernel SMP strategies for increased performance on common workloads. FreeBSD 7.0 was the first release to benefit from this optimization work.
The status of this work includes MySQL workload benchmarks and memory allocator performance in the new FreeBSD 8 branch. Also, here is a recent presentation showing FreeBSD compared to several other operating systems like NetBSD, DrangonFly, Solaris, and Linux.
Source: osnews.com (11/03/2008) where you can also read a lively FBSD vs Linux comments thread.
Awesome. slide 12 of Kris’ new slides is great: it’s like the holy grail of scaling!
We should also relate this back to the earlier “I’m
faster …” story. There Nick P. strictly used MySQL for all of his benchmarking. In Kris’ latest slides, he points out some of the MySQL scaling problems
above 8 threads are caused by MySQL’s architecture itself (slide 15).
We can also say now that there may be a disconnect between slide 17 and Nick’s
results (see his last slide at http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/) comparing 7.0 and 2.6.22. Kris said it was i386 for his
Xeons. Perhaps, Nick was using 7.0-amd64? (Too many variables!)
Please let’s
be fair..
1. Take a look at Kris’s graph -> http://people.freebsd.org/%7Ekris/scaling/os-mysql.png
You can see a dropoff in the 2.6.22 line between
12 and 15 threads.
Now take a look at Nick’s graph -> http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/16-2.png
The same dropoff! Even
though it seems to be between 11 and 13 threads. That alone proves that the benchmarks are similar!
2. Kris’s graph stops at 20 threads, whereas Nick
goes up to 60 threads. FreeBSD only starts to decline after 20 threads, http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/64-2.png
Now take
a look at Jeff’s blog. http://jeffr-tech.livejournal.com/18706.html
When asked about the low-point he says: “I’m glad you asked about the low point. We
actually just discovered today that it was simply our pthreads implementation not adaptive spinning for long enough. Changing an env var LIBPTHREAD_SPINLOOPS
to 2000 puts our performance back in line with linux at these higher thread counts.”
Now the question is.. did Jeff and Kris know about this dropoff after
20 threads in advance and hence cut their graphs there? I’m not going to speculate about it but the question is valid!
3. Kris used 4 dualcore Opterons
whereas Nick used two quadcore Barcelonas. Jeff even admits that they are a bit slower than Linux: “On this processor I still think we’re about 4-5% slower
than linux in 7.x due to non-core aware scheduling algorithms. In 8.0 we should be doing better again in a week or two.”
I have no reason not to trust
Nick’s results. Think about it.. why should you value Kris’s results over Nicks? Kris represents the FreeBSD project and Nick is a Linux kernel developer.
I guess if you want 100% objective results you have to run those benchmarks yourself.
Thanks
I have no reason not to trust Nick’s results. But, there is a disconnect in the two result sets, where one shows linux 2.26.22 on top across the
board (less one point) and the other shows FBSD 7.0 on top across the board. There are many possible explanations. The integrity of the results is NOT an
explanation that I even suggested.
I think it’s
one of these:
– different kernel configs (maybe some debugging options left in by Kris? or Nick tuned it a bit)
– 4 x dualcore vs. 2 x quadcore, just
take look at Jeff’s blog, the cpu topology patches aren’t in 7.0, they will be in 7.1 or later.