Testing PostgresSQL

PostgreSQL is one of the leading Open Source databases. While it has a smaller installed base than MySQL it is usually regarded to be more robust with more features than any other Open Source Database.

Development

Source code is kept in CVS. See instructions on how to get it. There is a huge set of documentation on how to get involved in the development of PostgreSQL.

The team provides nightly code builds, though the last one was on the 10th August and today is the 21st. It seems they had a long night there.

Testing

Automated testing

PostgreSQL has a distributed build farm consisting, partially their own, partially volunteers

The build-farm itself is written in Perl, it has it own separate project page. The main page of the build-farm has a link to the documentation on how to setup a client. It looks frightening, but actually it is a lot simpler than what the text looks like. Actually before you do that you have to first login to the CVS server as described on the Source Code Repository page.

They require a bunch of Perl modules - most of them are standard - so they should not get people worried right at the beginning. They might also want to add instructions on how to install the missing modules.

The configuration files is a bit funny as it is a Perl data structure.

AFAIK they should move to something with less parentheses such as an INI file or YAML.

The smoke testing client can run the whole build/testing cycle. Each client has a single configuration and that is what it is testing

Before one can actually send the test result s/he needs to register. During registration I had to provide Operating System, OS Version, Compiler Compiler Version, Architecture, my name and e-mail. I gave the following values (Ubuntu, 7.04, gcc, 4.1.2, i686) answering the first five questions. I am still not sure if that's what they meant by architecture and why don't they collect this information during by the client. What if I upgrade my machine and forget to tell them? Will they notice it? What if I want to setup another client on another machine? Do I need to register again?

I like the fact that I need to provide a verifiable e-mail with my report. While this make reporting slightly more difficult at the beginning, if I happen to send bogus reports by mistake, they can easily contact me.

Using a username and password also reduces the risk of someone else sending bogus reports in my name.

While trying to setup the client, on the first run I got the following, quite cryptic error message: Stage Configure failed with status 77 First it seemed as an actual failure in the code base of PostgreSQL but after some research I found several log files in the HEAD/lastrun-logs/ that pointed me to my error. I have not installed ccache. I bumped into another problem till I got my smoke client up but eventually it worked. I was waiting for them to process my request to join the build farm for several days. That's a bit disappointing, though it is August. So it might be just because of the Holidays.

The status page of the PostgreSQL Build Farm is very nice. On the main page you can see the latest status of each machine. Success/failure and when was it last reported. They have cute flags to show what flags are on for that particular build machine. (I wonder why not let the same machine run several configurations one after the other.) Clicking on the "Config" link they provide a drill down to the many phases of the smoke testing. Starting by checking out from CVS, configuring, building and running several tests.

They also provide a history for the specific machine to see when did it run previously and what were the results.

BTW I found this article about the PostgreSQL BuildFarm OnLamp.

TODO

See how tests are written and how can one add a new test.