April 09, 2013

Holger

Hackweek 9 @ SUSE

It’s hackweek time again at SUSE, and we are using the time to improve SMB Traffic Analyzer. At some point we decided to move the whole SMB Traffic Analyzer software suite to QT4. This movement is fronted by Nanuk Krinner, who already implemented our realtime component smbtamonitor-qt. His work at hackweek 9 will improve the graph display, and it’s realtime movement.  I am working on a port of the object selector of webSMBTA to QT4, and a solid integration of smbtaquery with smbtamonitor-qt. Once we have reached a certain point of integration, we will work on porting webSMBTA features to smbtamonitor-qt. We found it hard to deal with framework updates for webSMBTA. Since webSMBTA is based on smbtaquery too, it won’t be dead anytime soon, but we certainly will give more attention to the QT port in future.

Check out Hackweek 9, and explore all the cool ideas people have put on the table to hack on for a week!

 

hhetter123

April 09, 2013 07:08 PM

April 03, 2013

Simo

Why depending on DNS Reverse resolution is bad

I have been recently involved in a discussion about why I go around trying to stop applications from using and sometime even depending on DNS Reverse resolution (PTR records lookups).

There are 2 main reasons:

• In many networks you cannot really control the PTR records, so reverse resolution is simply broken
• Reverse resolution is bad when used for security protocols like GSSAPI

Let's start from the first point, which is easy to argue about. In a lot of cases the person setting up a service is not the same person controlling the DNS. Even more the DNS person/organization controlling the Forward Zone may not be at all the same one that controls the Reverse Zone.

This is true for the general internet usage (try asking your ISP to set a special PTR record for your residential public IP address ... laughs) but also for some corporate environments where the Network Ops may be so separate from the user installing a machine and rules to ask changes to DNS so complex that it is sometime simply too inconvenient to ask for changes, especially in temporary settings like Proof of Concept trials, etc.. This is not hypothetical, in my past life as consultant I've seen it all, and I can tell PTR records are broken more often than not.

So by this reason alone depending on a PTR record to obtain the actual name of a server is a pretty high bar and will inevitably be a barrier for adoption. It gets to silly levels if an application actually gets the 'right' name in input and then translates it into an IP only to attempt reverse resolution and fail. Users legitimately get pissed the app is so stupid as to throw away the name they just gave it. I just gave the name to you! Don't you see it!

It is surprising how many applications do this silly game when it comes with providing the target name to GSSAPI, which introduces the second point.

Why is it bad from a security point of view ? We understand that it is unfortunate for cases were reverse resolution is broken, but if reverse resolution is properly configured what is so bad depending on it ?

This is a little scenario I wrote up on the linux-nfs mailing list to explain how the fact rpc.gssd (the client that handles GSSAPI authentication on the kernel behalf in user space for the nfs client module) depends on reverse resolution can actually be exploited by an attacker.

Assume the following scenario:

• User Alice has access to secret documents that are automatically backed up daily by mounting a NFS share from secure.server.name
• Only Alice has write or read access to this server
• User Eve wants to get hold of those documents and is in a position to intercept and modify Alice's traffic

There is a similar server available on the network that Alice has write access to called public.server.name

Eve has read access on public.server.name

The NFS servers use RPCSEC_GSS (Kerberos) to secure the communications and perform mutual authentication.

Note that Eve does not need to be controlling any of the servers, and it is sufficient for her to be able to spoof DNS replies.

Now the attack: Eve wants to fool Alice's computer to mount the public server's NFS share instead of the secure server one, so that the automatic backup job will copy Alice's secret documents to the public server where Eve has read access and can grab a copy.

Normally this is not possible, because the Kerberos protocol implies mutual authentication. Not only the user authenticates to a server by using a ticket, but the ticket is only usable by the right target server, therefore authentication fail if either the user or the server are not who they say they are.

In our case normally Alice will grab a ticket for nfs@secure.server.name (GSSAPI Naming notation), which can be used exclusively to authenticate against the secure server. If Eve tries to redirect communication to the public server the authentication will fail because the public server is not able to decrypt the ticket.

However, rpc.gssd does a very bad thing(TM). When the client runs the mount command it uses the name provided on the command line to obtain the server's IP address, then ignores the fact we already have a name and proceeds to perform a reverse lookup to 'find' the server name.

What this means is that Eve can simply spoof the DNS to redirect Alice's computer to contact the wrong server and then later rpc.gssd will 'find' that the 'real' name of the server is public.server.name (Either because Eve spoofed the original forward resolution reply or by spoofing the reverse resolution reply later on).

Now Alice's computer will call into GSSAPI with the constructed name of nfs@public.server.name and when it connects to that server mutual authentication is successful because the ticket can be decrypted by the target server.

Eve just waits for Alice's computer to complete its backup on the wrong server on which she has read access, and grabs the documents.

This type of attack obviously is not limited to the NFS protocol but can be performed against any client that trusts DNS Reverse resolution to determine the target server's name. It is also not limited to GSSAPI, an SSL client might also be fooled the same way if it doesn't check the name that was provided in the URL but instead uses DNS Reverse resolution to validate the server certificate. Luckily I am not aware of any client doing that for HTTPS at least.

And that is all folks!

April 03, 2013 11:00 PM

April 01, 2013

Rusty

Thanks for the Bitcoin donation!

Last week I used 2 BTC to support Jupiter Broadcasting’s Unfilter show (and their other shows, but only Unfilter takes BTC so far).  Just now I noticed that someone made a 0.5BTC donation to my blog (I’ve had a BTC donation address in the sidebar of my blog for a few years now).  Thanks!

As I promised to pass donations onwards, I googled for bitcoin donations, and chose the following places to give 0.05 BTC each:

1. Juice Rap News for making high-baud political commentary (Unfilter in rap form)
2. Freedom Box for actually doing something about Internet freedom.
3. Torservers.net (as recommended by torproject.org) for the same.
4. f-droid.org for keeping a healthy Open alternative.
5. Bitcoin Foundation to support and strengthen the infrastructure that made this possible.
6. The Free Software Foundation even though I don’t always agree with them.
7. Wikileaks for recognizing something society needs, even if they stumble at delivery.
8. The Internet Archive for something that only gets more useful over time.

There are two left to go, so I’ll keep an eye out for more opportunities to donate in the next few weeks…

-0.05

April 01, 2013 02:36 AM

March 28, 2013

Andreas

Writing and reading code

You’ve probably heard that a developer of an established software project writes an average of 100 lines of code (loc) a day. I can say that this applies to me. So if you write 100 loc per day, how many do you read? I would estimate that the amount of time you spend on reading and understanding code is significantly more than on writing code. You probably also spend quite some time debugging code.

If you spend so much more time on reading and debugging code than writing code, shouldn’t you put more effort in writing clean and debuggable code? The Samba codebase is pretty old, more than 15 years now. I would say we have some experience with bad code and we have started to write much better and cleaner code, because we have wasted so much time trying to understand and debug code. However there is still room for improvement. Lets take a look at the following C code snippet.

if (!a) {
    return;
}

if (!b) {
    return;
}

if (!c) {
    return;
}

if (!*d) {
    return;
}

Can you guess from the code above what types of variables a, b, c and d are? The answer is no? Ok, lets take a look at the following code:

if (a == NULL) {
    return;
}

if (b == 0) {
    return;
}

if (c == false) {
    return;
}

if (d[0] == '\0') {
    return;
}

If you look at the code now, you can probably guess what types they are. Well not exactly which type, but in which superset they are. ‘a’ is a pointer, ‘b’ is an integer type, ‘c’ is a bool and ‘d’ is a string (char array). If you write code the way shown above, you don’t have to scroll up to find out as which type the variable is defined. Most of the time it is enough to know what type of superset you are checking and why.

Think about this: If you spend just a bit more time on writing clean code now, you will spend less time on reading, understanding and debugging the code later if you have to find a bug.

Lets look at some more best practices we do in the Samba code:

bool ok;
int rc;

/* bool return codes should always have the name 'ok' or start with 'is_' or 'do_' */
ok = fn_returning_a_bool();
if (!ok) {
    return;
}

/* We use rc or ret for an integer return code */
rc = do_something();
if (rc  0) {
    return;
}

You can see that we have variables for the return codes and check them with an if-clause. The reason is that it is easy on the eyes and in a debugger you can simply print the return code variable. If you write it like this: if (do_something() 0) You have a hard time in the debugger to find out the actual return code. You have to step into and through the function to get it. We allow the !ok syntax for bool types, cause ok is by definition in our code a bool.

To be continued ...

March 28, 2013 03:33 PM

March 21, 2013

Kai

Model vs. Reality

My twitter feed currently is ablaze with discussions about sexism in tech and science. As a member of the social group that goes through life with difficulty set to "easy", of course none of this has happened to me. I don't know what it feels like to go to a conference where people constantly comment on my looks or gender instead of my work. As I lack the experience, and I also lack solid data, I don't want to write about that in this post.

However, as a computational biologist, I do have some experience with model vs. reality clashes, and I believe that might be the reason why people on the internet are surprised about the existence of female scientists or engineers. People also tend to get upset when they realize their mental model doesn't match reality, which might explain some of the emotional upset males show in the discussions I mentioned in the first paragraph.

Now, instead of taking the easy way out and blaming this on internet stupidity, I want to put another theory out there. People often don't get statistics. In my day-to-day work, I frequently run into publications where there is at best a loose correlation between the data and the model supposed to explain said data. And I believe that's what is happening here as well.

So by the time-proven method of pulling data out of my hat, I propose that when you graph the "ability to do science" against the sex chromosome layout of individuals, you'll get something similar to this:

So far, so good. The problem is that despite "ability to do science" not clustering for any sane definition of a cluster, the mental model of many people seems to look like this:

If you have settled for a given model, there is quite some inertia to stay with your chosen model, even if the data doesn't back it up. If reality dares to come up with conflicting data, blame reality! The ripples of a lot of mental models running into reality hard are currently washing over my twitter feed. The inertia of sticking with your model makes it hard to realize it, but in the end when reality and your model disagree, it is easier to change your model. In my example graph, a lot of lines I could put in there would likely have a similar quadratic error. To me, this is a warning sign that my model probably is bad. In the example, the conclusion should be that not only "women are bad at science, men are great" (the red line) is wrong, but also every other attempt at constructing a linear correlation between the parameters. "Ability to do science" and "sex chromosome layout" are orthogonal characteristics*. Also, why is there an arrow on the x axis, when we're looking at discrete parameters?

* According to my hat, of course.

March 21, 2013 09:34 AM

March 20, 2013

Rusty

GCC and C vs C++ Speed, Measured.

With the imminent release of gcc 4.8, GCC has finally switched to C++ as the implementation language.  As usual, LWN has excellent coverage.  Those with long memories will remember Linux trying to use g++ back in 1992 and retreating in horror at the larger, slower code.  The main benefit was stricter typechecking, particularly for enums (a great idea: I had -Wstrict-enum patches for gcc about 12 years ago, which was a superset of the -Wenum-compare we have now, but never got it merged).

With this in mind, and Ian Taylor’s bold assertion that “The C subset of C++ is as efficient as C”, I wanted to test what had changed with some actual measurements.  So I grabbed gcc 4.7.2 (the last release which could do this), and built it with C and C++ compilers:

1. ../gcc-4.7.2/configure –prefix=/usr/local/gcc-c –disable-bootstrap –enable-languages=c,c++ –disable-multiarch –disable-multilib
2. ../gcc-4.7.2/configure –prefix=/usr/local/gcc-cxx –disable-bootstrap –enable-languages=c,c++ –disable-multiarch –disable-multilib –enable-build-with-cxx

The C++-compiled binaries are slightly larger, though that’s mostly debug info:

1. -rwxr-xr-x 3 rusty rusty 1886551 Mar 18 17:13 /usr/local/gcc-c/bin/gcc
   text       data        bss        dec        hex    filename
   552530       3752       6888     563170      897e2    /usr/local/gcc-c/bin/gcc
2. -rwxr-xr-x 3 rusty rusty 1956593 Mar 18 17:13 /usr/local/gcc-cxx/bin/gcc
   text       data        bss        dec        hex    filename
   552731       3760       7176     563667      899d3    /usr/local/gcc-cxx/bin/gcc

Then I used them both to compile a clean Linux kernel 10 times:

1. for i in `seq 10`; do time make -s CC=/usr/local/gcc-c/bin/gcc 2>/dev/null; make -s clean; done
2. for i in `seq 10`; do time make -s CC=/usr/local/gcc-cxx/bin/gcc 2>/dev/null; make -s clean; done

Using stats –trim-outliers, which throws away best and worse, and we have the times for the remaining 8:

1. real    14m24.359000-35.107000(25.1521+/-0.62)s
   user    12m50.468000-52.576000(50.912+/-0.23)s
   sys    1m24.921000-27.465000(25.795+/-0.31)s
2. real    14m27.148000-29.635000(27.8895+/-0.78)s
   user    12m50.428000-52.852000(51.956+/-0.7)s
   sys    1m26.597000-29.274000(27.863+/-0.66)s

So the C++-compiled binaries are measurably slower, though not noticably: it’s about 865 seconds vs 868 seconds, or about .3%.  Even if a kernel compile spends half its time linking, statting, etc, that’s under 1% slowdown.

And it’s perfectly explicable by the larger executable size.  If we strip all the gcc binaries, and do another 10 runs of each (… flash forward to the next day.. oops, powerfail, make that 2 days later):

1. real    14m24.659000-33.435000(26.1196+/-0.65)s
   user    12m50.032000-57.701000(50.9755+/-0.36)s
   sys    1m26.057000-28.406000(26.863+/-0.36)s
2. real    14m26.811000-29.284000(27.1308+/-0.17)s
   user    12m51.428000-52.696000(52.156+/-0.39)s
   sys    1m26.157000-27.973000(26.869+/-0.41)s

Now the difference is 0.1%, pretty much in the noise.

Summary: so whether you like C++ or not, the performance argument is moot.

March 20, 2013 10:52 PM

March 04, 2013

Simo

IMAPD via SSH and Thunderbird

I have been using Evolution for many years, and one of the key features that kept me using it was the ability to run imapd on another machine via ssh. This was done using a simple command in Evolution's option:

ssh -l <user> <server> exec /usr/sbin/imapd

This ssh command will allow Evolution to connect directly to a pre-authenticated imapd process on my server avoiding the need to run a network facing service and the need for password based authentication. Everything is accessed via my ssh connection that uses key based authentication

(the option is not directly available anymore and you have to fiddle with gsettings to use it now, which is a real shame as it is completely undiscoverable.)

I recently decided to try out Thunderbird again and found out that this is one of the features that is still missing, after all these years ...

This was a blocker for me, so I decided to find a workaround that would allow me to use Thunderbird and still use ssh to reach the imapd daemon on my server, like I have done for the last decade.

After some tinkering and reading on all the SSH options for Nth time I came to the conclusion that ssh alone cannot run a remote command and wire it's STIDN/STDOUT to a local port even though it can do pretty much any other forwarding you may think of, including forwarding your local STIDN/STDOUT to a remote host/port ... a real shame.

The most I could achieve was to make SMTP available this way, as I do have an MTA listening to an actual TCP port on the server. Making the MTA available is easy, you just need to run the following command on your client:

ssh -f -N -C -L 10025:localhost:25 -o ExitOnForwardFailure=yes -l <user> <server>

This command, makes available locally on port 10025 the server's port 25 through a simple forward on a SSH encrypted channel. The -f and -N options, are used to put ssh in the background without running any command or shell. The -C option turns on compression and the ExitOnForwardFailure option makes ssh fail to start if it cannot establish the forwarding. This way if I run the command multiple times only one tunnel stays up as the other shells will simply silently exit.

This is quite cool already but doesn't solve my imap problem, to solve it I need to employ one of those little know yet very powerful tools available on Linux (and other *nix OSs as well): netcat

The version I have installed is the one distributed with the Nmap project.

Netcat (ncat or nc) is an incredibly useful tool. I've used it countless of times for all sort of things over the network. And it is the perfect tool to solve my problem when used this way:

ncat -k --sh-exec "ssh -C -l <user> <server> exec /usr/sbin/imapd" -l localhost 10143

This command does a wonderful thing. It keeps (-k) listening (-l) on the local port 10143 and every time there is a connection it will run the command provided by the --sh-exec option in a shell and wire it's STDIN/STDOUT to the connection that has been just opened over TCP.

This is exactly what I needed. Now every time Thunderbird connects to my local port 10143, netcat will run the ssh command that will connect to the remote server as my user and run the imapd server.

Although Thunderbird's configuration doesn't seem to allow for 'non' authenticated connections, everything seem to work fine if I just leave the password empty. (Remember the imapd server is pre-authenticated via my ssh connection as my remote user and requires no additional authentication)

So what is missing here ? The Security paranoids among my readers should have spotted one glaring issue! Everybody on my local machine can now connect to my local port 10143 and access my remote mailbox without authentication!!

Let me fix that with a single firewall instruction:

ipatables -A OUTPUT -p tcp --dport 10143 -d 127.0.0.1 -m owner ! --uid-owner simo -j REJECT

Yep, it is a simple as that (on Linux at least). But what does it do ?

This command uses a very nifty feature of iptables that allows the kernel to recognize who is the owner of any outbound connection and will prevent any connection to port 10143 for any user on the system that is not me. Of course iptables filters any non local connection to my machine as well.

Problem solved!

Now I can start playing with Thunderbird and see what else I need to tweak to make it useful for me (one thing I already found is an add-on to import/export entire folders, a feature I always wanted and missed in Evolution)

March 04, 2013 04:10 AM

March 01, 2013

Simo

Talking to people

The new year started with a lot of talks at various conferences.

For the past few years I had slowed down on attending conferences, but this year started with my attendance to 2 conferences I like a lot.

The first is FOSDEM, probably the best Free Software conference and certainly the biggest one in the world.

I just love FOSDEM, I love Belgium for the Beers and Chocolate, so it is always a pleasure for me to go there. Plus I have friends in Brussels, where I have been multiple times in the past so it is always a pleasure to go back there for a full immerse week end.

This year I presented 2 talks at FOSDEM.

One in the main track about Identity Management on Linux and a second in the Legal Devroom about a Veteran's perspective on various legal matters surrounding Free and Open Source Software. I organized this talk as an open discussion between me and the public and I absolutely loved the conversation.

The IdM talk in contrast was a classic solo speech on a 30 kilometers high overview about the problem of building an IdM system on Linux and for Linux. It does have references to the FreeIPA project but does not go in deep technical details beyond explaining why we choose certain technologies.

This actually led to criticism after the talk: Not technical enough!

And it is a fair one, too bad that when I presented the initial abstract to FOSDEM I got the opposite reply: Too technical!, so I had to water down and broaden the initial proposal :-).

I guess you can never win this game, so my resolution is to oscillate between the two extremes ...

... which brings me to the other talk at DevConf.cz. This is a very nice conference, organized by Red Hat in Brno.

DevConf.cz is a developer conference so I presented a pretty technical talk on GSSAPI and privilege separation using Gss-Proxy which is the latest project I launched together with Nico and later the help of Günther.

This time I got the: Too technical! red flag. Hopefully, though, it was still interesting enough for the audience.

All in all, I enjoyed these conferences very much, I won't list all the excellent talks I attended, there were too many. Most importantly I was able to finally meet face to face with some people I interact every day or I needed to have a more interactive discussion to hash out some problems an ideas. So fun and very productive time, what more can you ask for as a nerd type ?

March 01, 2013 09:00 PM

February 26, 2013

Andreas

vim modelines in git config

I’m working on different Open Source projects and most of them have different coding style guidelines. Mostly spaces or tabs or different tabwidth. The easiest thing would be to store these information in the git config of the project. So here is a easy and secure way to have modelines in the git config.

So first I set the modelines (here for the Samba project):
git config --add vim.modeline "tabstop=8 shiftwidth=8 noexpandtab cindent"

Then copy this plugin into ~/.vim/plugin folder.

The modeline you defined in your git config will be appended to the :set command of vim. It only allows a limited set of set commands to be used. It will not execute any arbitrary code and you probably are the only person modifying the git config. You can change the allowed commands by adding the following to your ~/.vimrc file

    let g:git_modelines_allowed_items = [
                \ "textwidth",   "tw",
                \ "softtabstop", "sts",
                \ "tabstop",     "ts",
                \ "shiftwidth",  "sw",
                \ "expandtab",   "et",   "noexpandtab", "noet",
                \ "filetype",    "ft",
                \ "foldmethod",  "fdm",
                \ "readonly",    "ro",   "noreadonly", "noro",
                \ "rightleft",   "rl",   "norightleft", "norl",
                \ "cindent",     "cin",  "nocindent", "nocin",
                \ "smartindent", "si",   "nosmartindent", "nosi",
                \ "autoindent",  "ai",   "noautoindent", "noai",
                \ "spell",
                \ "spelllang"
                \ ]

UPDATE:
* New script which only allows a specified list
* Use sandbox command for set
* Added git repository
* Set only locally when reading the buffer

February 26, 2013 10:44 AM

February 08, 2013

Jelmer

OpenChange 2.0 released

Apparently 'tis the season for major software releases.

Julien has just announced the release of OpenChange 2.0, codenamed quadrant. This release fixes a number of important bugs and enables integration with SOGo.

With the SOGo backend, it is now possible to set up an Exchange-compatible groupware server that can be accessed from Outlook without the need to connect any connectors.

See the release notes for more details.

February 08, 2013 07:00 PM

January 26, 2013

Michael

Running Debian Squeeze as libvirt/kvm Guest on Ubuntu

I struggeled for a while with Debian Squeeze, trying to persuade it to run as a libvirt/kvm guest on my Ubuntu 12.04 host. After successful installation, the guests failed to boot the kernel with the messages:

Loading Linux 2.6.32-5-amd ...
unaligned pointer 0x8ebf0002
Aborted. Press any key to exit.


For my own reference, I record here the solution I just found after trying all sorts of things like installing different versions of grub into the virtual machine:

I had to choose cirrus as the video card model instead of my default vmvga. This way, I can run an unmodified Squeeze. In order to use the superior vmvga video card, I still have to replace grub…

January 26, 2013 01:06 AM

January 16, 2013

Rusty

Looking forward to linux.conf.au 2013

This year’s organizers took specific pains to attract deep content, and the schedule reflects that: there are very few slots where I’m not torn between two topics.  This will be great fun!

After a little introspection, I did not submit a talk this year.  My work in 2012 was with Linaro helping with KVM on ARM: that topic is better addressed by Christoffer Dall, so I convinced him to submit (unfortunately, he withdrew as January became an untenable time for him to travel).  My other coding work was incremental, not revolutionary: module signatures, CCAN nor ntdb shook the ground this year.  There just wasn’t anything I was excited about: a reliable litmus test.

See you at LCA!

January 16, 2013 01:36 AM

January 14, 2013

Andreas

cmocka – a unit testing framework for C

I’m a big fan of unit testing frameworks. When I developed csync, a bidirectional file synchronizer, I used check to write unit tests from the start. check was ok, but it were running valgrind on your testcases to find memleaks in your code the mode reports were about check. So I needed to add valgrind suppressions to get rid of them. When I started to work on libssh, a library implementing the SSH protocol, I wrote unit tests with check too. libssh is multi platform and also works on Windows and with Visual Studio. So we needed a new unit testing framework which is platform independent and has better code quality. I stumbled upon cmockery, a unit testing framework from Google. It was easy to use, the code looked good and it worked with Visual Studio. The build system sucked, so I added CMake support to produce a NSIS installer for Windows. I sent all my patches upstream but nothing happened. I fixed more bugs and added all patches people posted in their bug tracking system. I tried to talk with friends at Google, but in the end I needed to fork it.

cmocka is a fork and the successor of cmockery. I started to fix a lot of bugs, got all examples working and wrote API documentation with doxygen. The result is this first release version 0.2.0.

cmocka is a great unit testing framework with support for mock objects. Mock objects are simulated objects that mimic the behavior of real objects in a controlled way. Instead of calling the real objects, the tested object calls a mock object that merely asserts that the correct methods were called, with the expected parameters, in the correct order. It is really easy to write a unit test, take a look at the API an get started.

Example:

#include <stdarg.h>
#include <stddef.h>
#include <setjmp.h>
#include <cmocka.h>

/* A test case that does nothing and succeeds. */
static void null_test_success(void **state) {
(void) state; /* unused */
}
int main(void) {
const UnitTest tests[] = {
unit_test(null_test_success),
};
return run_tests(tests);
}

January 14, 2013 10:47 AM

December 29, 2012

Simo

What a great year!

This past year has been really great, too bad I found little time to update my blog :-)

A few things happened that made me cheer up while thinking about what has been going on this year.

Samba 4.0 finally happened. It has been an incredible, long ride, with highs and lows but amazingly we pulled it off!

FreeIPA 3.0 and 3.1 with AD cross-forest trust integration also were released this year. I am so proud of this project, it has achieved results I hardly hoped for when I started it a few years ago.

SSSD has seen multiple releases with the 1.8 Long Term Maintenance series and 1.9 series. SSSD is one of the most successful projects I started these past years and I used it every day myself with great pleasure.

Gss-Proxy is the last project I started, just this year, and has seen 2 initial no-fanfare releases. It is one of those plumbing things that are hardly seen (except when things break :-) but it was exciting to work so deep into GSSAPI code.

December 29, 2012 06:00 AM

December 24, 2012

Rusty

Fixed-length semi-lockless queues revisited

There were some great comments on my previous post, both in comments here and on the Google Plus post which points to it.  I’d like to address the point here, now I’ve had a few moments to do follow-up work.

One anonymous commenter, as well as Stephen Hemminger via email, point to the existing lockless queue code in liburcu.  I had actually waded through this before (I say waded, because it’s wrapped in a few layers which I find annoying; there’s a reason I write little CCAN modules).  It’s clever and genuinely lockless; my hat off to , but it only works in conjunction with RCU.  In particular, it’s an unlimited-length queue which uses a dummy element to avoid ever being empty, and the fact that it can safely traverse the ‘->next’ entry even as an element is being dequeued, because the rules say you can’t alter that field or free the object until later.

Stephen also pointed me to Kip Macy’s buf_ring from FreeBSD; it uses two producer counters, prod_head and prod_tail.  The consumer looks at prod_tail as usual, the producers compare and swap increment prod_head, then place their element, then wait for prod_tail to catch up with prod_head before incrementing prod_tail.  Reimplementing this in my code showed it to be slower than the lower-bit-to-lock case for my benchmarks, though not much (the main difference is in the single-producer-using-muliple-producer-safe-routines, which are the first three benchmarks).  I ignored the buf_ring consumer, which uses a similar two-counter scheme for consumers, which is only useful for debugging, and used the same consumer code as before.

Arjen van de Ven makes several excellent points.  Firstly, that transaction-style features may allow efficient lock-ellision in upcoming Intel CPUs (and, of course, PowerPC has announced transaction support for Power8), so we’ll have to revisit in a few years when that reaches me.

His more immediate point is thatuncontended locks are really cheap on recent CPUs; cheaper than cache-hot compare-and-swap operations.  All the benchmarks I did involve everyone banging on the queue all the time, so I’m only measuring the contended cases.  So I hacked my benchmarks to allow for “0 consumers” by having the producer discard all the queue contents every time it filled.  Similarly, filling the queue with junk when it’s empty for a “0 producers” benchmark.

Here we can see that the dumb, single lock comes into its own, being twice as fast as my optimal-when-contended version.  If we just consider the common case of a single writer and a single reader, the lockless implementation takes 24ns in the contended case, and 14ns in the uncontended cases, whereas the naive locked implementation takes 107ns in the contended case and 7ns in the uncontended case.  In other words, you’d have to be uncontended over 90% of the time to win.  That can’t happen in a naive implementation which wakes the consumer as soon as the first item has been inserted into the queue (and if you implement a batch version of queue_insert, the atomic exchange gets amortized, so it gets harder to beat).

For the moment, I’m sticking with the previous winner; there’s still much to do to turn it into a usable API.

December 24, 2012 04:33 AM

December 19, 2012

Jelmer

Bazaar: A retrospective

For the last 7 years I've been involved in the Bazaar project. Since I am slowly stepping down, I recently wrote a retrospective on the project as I experienced it for the last 7 years.

Thanks to a few kind people for proofreading earlier drafts; if you spot any errors, please let me know in the comments.

December 19, 2012 09:36 PM

December 17, 2012

Rusty

Fixed-length semi-lockless queues…

One of my vacation project was to look at a good queue implementation for ccan/antithread.  I read a few papers, which mainly deal with generic link-list-style queues (I got quite excited about one before I realized that it needed a 128-bit compare-and-swap for 64 bit machines).  I only really need a fixed-length queue of void *, so I set about implementing one.

You can find the cleaned-up version of my explorations on github.  For my implementation I use a tail counter, 32 void * entries, and a head counter, like so:

#define QUEUE_ELEMS 32
struct queue {
    unsigned int head;
    unsigned int prod_waiting;
    unsigned int lock;
    void *elems[QUEUE_ELEMS];
    unsigned int tail;
    unsigned int cons_waiting;
};

The head and tail counters are free running to avoid the empty-or-full problem, and the prod_waiting and cons_waiting are for a future implementation which actually does sleep and wakeup (I spin for my current tests).

The simplest implementation is for both producers and consumers to grab the lock, do their work, then drop the lock.  On my 32-bit x86 dual core 2 HT laptop, with 1 producer on cpu0 and 1 producer on cpu1 (ie. two hyperthreads of same core), it takes about 179 usec to enqueue and dequeue each element (but hugely variable, from 73 to 439 ns).  You can see that (as expected) the 2 and 3 producers cases are quite expensive, though not so bad if there are 2 producers and 2 consumers.

Lockless dequeue is quite easy:

1. Read tail counter, then read head counter (order matters!)
2. If it’s empty, wait until head changes).
3. Grab entry[tail % 32].
4. Try to compare and swap the tail to tail+1.  If not, we raced, so goto 1.

But lockless insert is harder, so I asked Paul McKenney who detailed a fairly complex scheme involving two offsets and some subtlety on both production and consumption side, and ended with “Now, are you -sure- locking is all that bad?  ;-)”.  So I went for a big lock around insertion to begin with.  It’s generally a little better, particularly for the common case of a single consumer and a single producer.

It’s worth noting that if you know you’re the only producer, you can skip the locks so I re-ran the benchmarks with a “queue_add_excl” implementation for the single-producer cases, as seen on the right.

You can similarly simplify the single consumer case, though it makes little difference in my tests.

However, you can do better than a straight naive lock: you can use the lower bit of the head counter to exclude other producers.  This means a production algorithm like so:

1. Read head.  If it’s odd, wait.
2. Read tail.
3. If queue is full, wait for tail to change, then goto 1.
4. Compare and swap head to head + 1; if it fails, go to 1.
5. Store the element.
6. Then increment the head.

For simplicity, I made the tail counter increment by 2 as well, and the consumer simply ignores the bottom bit of the head counter.  Avoiding a separate atomic operation on a “prod_lock” word seems to pay off quite well.

Finally, it’s worth noting that neither the exclusive producer nor exclusive consumer cases win much any more, so I can delete those altogether.

Before tying this into antithread, there are several things to do:

1. Re-audit to make sure the barriers are correct.
2. Test on PowerPC (always good for finding missing barriers).
3. Add in a decent notification mechanism, ie. futexes or falling back to pipes.

And that’s next…

December 17, 2012 03:38 AM

December 11, 2012

Andreas

Finally it is out, Samba 4.0.0!

We just released version 4.0.0 of Samba. About 10 years in development this is a huge milestone. It is the first version that can be a domain controller in an Active Directory domain!

December 11, 2012 06:13 PM

Jelmer

Samba 4.0.0, finally

This afternoon we released version 4.0.0 of Samba. This is a significant milestone, and I'm very proud of the result. Samba 4 is the first version that can be a domain controller in an Active Directory domain.

We embarked on this journey almost a decade ago - the first commit is from August 2003. It's been a long and bumpy ride. I hardly recognize the people in this team photo from 2003 (I'm second from the left).

A lot has happened in that time. We wrote a few million lines of code. We migrated from CVS to Subversion to Git. We've drifted apart and grown back together as a team.

In my youthful naivity I predicted a release "within 1 or 2 years" during a talk at the NLUUG in 2004. But Active Directory was a lot harder than we thought, and there were quite a few other distractions as well. I'm glad this release, which is by far the biggest and longest running software project I have ever worked on, has finally happened.

Some older RCs of Samba 4 have already been packaged for Debian and Ubuntu, in the samba4 source package. For Debian jessie, these will be integrated into the main samba source package. Please use experimental if you do want to try the existing packages, as it is most up to date.

December 11, 2012 05:00 PM

November 30, 2012

Jelmer

Documentation

From LWN's weekly edition:

Documentation is the sort of thing that will never be great unless someone from outside contributes it (since the developers can never remember which parts are hard to understand).

—Avery Pennarun

November 30, 2012 11:00 PM

Last updated: May 23, 2013 09:00 AM