Building Software from Source Code OpenSSL encryption software Apache web server Configuration files cURL readline PHP PHP configuration Session directory MediaWiki Installation Upgrading Patches Webalizer PostgreSQL database server MySQL database server

Building Software from Source Code

Our servers now run a fairly up-to-date version of Ubuntu Linux, which includes most of the software we need. We prefer to use Ubuntu packages for software when we can, because MCS systems can then update it easily for us and we don't have to bother maintaining it ourself. There are, however, some software packages which we have had to build and install from source code. These are listed below, along with the reason we build from source and instructions on how to do it (usually just running a script). In general, you'll need to build things in the order listed here.

You can build this software anywhere, but in general Eric has done so under /usr/local/src. By tradition and Unix convention any software we build from source code is installed under /usr/local. Any system upgrade needs to preserve /usr/local. (And after an upgrade you may well need to re-build things to match the newer environment.)

Scripts to build some of this software are in SVN in trunk/tools/ and generally begin with build_.... If there is such a script, it is generally set up for you to run it from the top level directory in the source tree. So to build OpenSSL you would unpack the source tarball, cd into the source directory, and then say `sh ../build_openssl`./ If there is no build script for a software package then it should be sufficient to cd into the source directory and say `./configure; make; make install`.

OpenSSL encryption software

Our web server uses the OpenSSL libraries for secure HTTPS, and the database server will use it for encrypted TLS connections. Secure shell (ssh) may use the libraries as well. If you build OpenSSL from source code (which is recommended) then do so before you build anything else which would use it.

Historical note: A flaw was found in early 2008 in the way Debian packaged OpenSSL which basically made keys generated by the Debian distribution easily guessable. The solution to this was to upgrade to a newer version of OpenSSL (and SSH?) and regenerate host keys. While the host keys were regenerated for the ELabs cluster, it's not clear that any software was updated. And we'd like to avoid such things in the future. Hence we take reponsibility for building OpenSSL ourselves and keeping it up to date.

The proceedure is simple:

1. Download the latest source code package from http://www.openssl.org/source, and verify the integrity of the package by checking the MD5 or SHA1 hashes, or the PGP signature (or all of these).
2. Review the build_openssl script Eric wrote to configure and build OpenSSL. Run it from the top level source directory, though the script is likely to be found one level above that.
3. If the build and test complete with no errors, become the superuser and give the command `make install`. This will install under /usr/local, with working directory /usr/local/openssl (not the default, /usr/local/ssl).

Some of the software installed as part of the Ubuntu distribution checks the version of dynamic SSL libraries and complains if there is not a match. To avoid this, we just build a static libraries and link the software below to that, but the system software can use the dynamics libraries with the OS, which is probably safe at this point.

Apache web server

While the e-Labs are served to the web via Tomcat, we also use the Apache web server for Bluestone, the bug report form, MediaWiki, and the forums. We also use Apache httpd to proxy Tomcat (which is on port 8080) out via port 80, so that everything looks like it comes from one web site/server.

There is a version of Apache's web server, httpd, already installed on our servers, but it is not properly configured for our use. We therefore have to build and install our own httpd from source code.

The main problem with the web server that comes pre-installed on our servers (if indeed there is one pre-installed) is that it's built with the wrong multi-processing module (mpm). To effectivly run PHP we need to use separate processes, not threads, which means using the "prefork" mpm module rather than the threaded module (called "worker"). The PHP manual explains this further here. The web server that comes with Ubuntu and Debian is compiled to use the "worker" module. Hence we must build our own.

Note from experience: there was a problem with the configure script in httpd-2.2.9, but this was fixed in 2.2.10.

Configuration files

This section has been moved to Web Server Configuration

cURL

cURL is a tool for downloading files addressed via URL syntax. It is both a library and a command line tool. We need cURL to build PHP. You can obtain the latest source code from http://curl.haxx.se/download.html. We either don't have curl installed by default, or the version installed was not sufficient. It's very easy to build and install; just do `./configure; make; make install`.

readline

The readline library provides a set of functions that allow users to edit typed command lines. PHP needs this if we want to be able to type commands interactively. Eric's script configures PHP to not use this, since it is not required for server operations, and readline is not installed on some of our machines. If you want to use this feature, install readline and alter the configuration in the build_php script.

PHP

PHP is a server-side scripting language which Eric also uses for CLI functions the way many people use Perl. It's used by the LIGO analysis tool Bluestone, the BOINC forum software, MediaWiki, and Moodle. Bluestone uses the PECL httpd extension to perform HTTP operations from the server, acting as a client, to upload files to the JSP e-Lab pages, or to submit a task to Swift. The geneal bug reporting form also uses this extension to submit a copy of a bug report to a Help Desk forum, in addition to sending the report via e-mail. This extension is not available with the "stock' version of PHP installed on our servers, so we need to build PHP from newer source.

Building PHP requires the program apxs from Apache httpd, so you must build _and install_ the Apache web server before you try to build PHP if apxs is not already installed on the system.

PHP will also need the MySQL header files to allow it to act as a database client. We have those on our cluster machines, but if they are missing they will need to be installed for PHP to be able to use MySQL.

To build PHP with the http extension you need to do the following

1. Download the latest stable PHP code from http://www.php.net and unpack the tarball
2. Download the latest stable PECL package for pecl_http from http://pecl.php.net/package/pecl_http and unpack the tarball
3. Move the directory for the pecl_http package to the PHP source tree as php-5.2.X/ext/http. That is, it becomes the http subdirectory of the ext subdirectory of the PHP source tree.
4. Run the build_php script Eric wrote to configure and build PHP. Run it from the top of the PHP source tree, though the script is likely one level up from that.

If PHP builds successfully you will be encouraged to run the `=make test=` command. It's a good idea to do so. If any tests fail (and often some do) you'll be asked to allow the script to send a report back to headquarters. It's also a good idea to do this, as it gives the developers feedback which helps them improve PHP.

One of the tests is labeled HttpRequest GET/PUT (test HttpRequest_002), and it failed the last time I built PHP. I looked at what the test was doing, and the failure seems to be trivial and nothing to worry about.

PHP configuration

Once PHP is installed you need to install and/or tweak the configuration file, which will be /usr/local/etc/php.ini. The key changes you should make or verify are:

1. Turn off implicit global variables, which present a serious security problem in PHP:

   register_globals = Off
   

2. Adjust error reporting to turn off NOTICE errors, and to log errors but not show them to users:

   error_reporting = E_ALL & ~E_NOTICE
   ignore_repeated_errors = On
   track_errors = On
   html_errors = Off
   

3. Configure for uploads, including Increasing the allowed size for uploaded files:

   post_max_size = 20M
   upload_tmp_dir =/var/www/upload
   upload_max_filesize = 20M
   

   You'll need to create /var/www/upload and make it owned by user www-data
4. Treat URL's like regular files, both for fopen() and include():

   allow_url_fopen = On
   allow_url_include = On
   

A copy of a customized php.ini file is available in the CVS repository mentioned at the top of the page.

Session directory

PHP stores session information in files between page "hits", and so the directory where these "session" files are saved must exist and be writeable by the web server. The location of this directory is set in the php.ini file. Asssuming the default, you can create and configure the session directory with

#  mkdir -p /var/lib/php/session
# chmod 755 /var/lib/php
# chmod 770 /var/lib/php/session
# chgrp www-data /var/lib/php/session

(Use group "apache" on a Fedora/RedHat machine.)

Eric theorizes that if this directory is shared between hosts via NFS then they can both serve the same PHP session, which is a form of load balancing. But he hasn't actually tried it yet.

In addition to the session directory, you should create /var/lock/subsys so that the init script can write a lockfile to it, and create /var/www/upload and make it owned by www-data, for PHP file uploads.

MediaWiki

Installation

Installation of MediaWiki is very easy. Instructions are included, but the basic idea is you unpack the tarball and copy most everything into the directory from which you want to serve it. The first time you connect via a browser you'll have to fill out a form describing the configuration. A script will then create the configuration and generate the database.

Other things you need to know about to manage the wiki:

• Configuration: all local configuration is kept in the file LocalSettings.php.
• Upload Area: to be able to upload files (usually images, but also documents) you need create an upload subdirectory and set the permissions such that the web server user can write to that directory. (Usually user "apache" on Fedora/Red Hat machines, user "www-data" on Debian/Ubuntu machines).
• Extensions: extensions are unpacked under the extensions subdirectory of the top level directory. Some will be a single file, some may have their own directory.
• Skins: are in the skins subdirectory

Upgrading

You need to preserve the file LocalSettings.php and the upload subdirectory. And any custom skins or extensions you may have added. Otherwise, replace all the code with the newer version. One easy way to do this is to move the old wiki out of the way, create a new directory, unpack the new code there, and then move LocalSettings.php and the upload directory and other customizations back in place.

For our servers we have added the files kiwi.php and body.php to automatically trigger use of skins of the same name. These are just soft links to index.php so you just need to re-make those links (and have the file SkinByURL.php file in the extensions directory).

I think that databse updates are applied automatically (need to verify this).

(Any other tricky bits get put here when we find them.)

Patches

Occasional updates to the MediaWiki software are distributed as Unix patch diff files. These are easy to install from the command line. For security patches you should apply them as soon as possible. The procedure is:

1. Download the patch to a file named something like mediawiki-1.10.4.patch and put this in the top-level directory of the wiki
2. From the top-level directory of the wiki run the Unix patch command with input redirected to come from this file, and the -p0 flag. That is,

 patch -p0 <mediawiki-1.10.4.patch

Webalizer

Webalizer is a software package which skims through web server logs to produce useful reports, which can be viewed on the web. Webalizer uses the 'GD' graphics package to create bar and pie charts, so you'll need to install "GD" first.

Both can easily be built and installed from source with `=configure; make; make install=`. You should obviously build and install 'GD' first.

The configuration file is in /usr/local/etc/webalizer.conf, with copious comments. You will need to choose (and create) an output directory where the report is kept, and configure the apache web server to get to it. Since we use BOINC for administration a good choice html/ops/usage/. You will also need to choose (and possibly create) a directory to hold webalizer history and state information between runs. A common choice is /var/lib/webalizer.

You can run webalizer by hand to get started, but in operation you should arrange to have it run nightly from a cron job. An example entry is

00 3 * * * /usr/bin/webalizer -c /usr/local/etc/webalizer.conf

which runs it every night at 3:00 AM. Note that webalizer just needs to be able to read the web server logs and write to the output directory and storage directory, but it doesn't necessarily need to be run as root.

PostgreSQL database server

(It appears that Mihael or Tibi built this a while ago. Ask one of them to fill in this section, or fill it in when we update.)

MySQL database server

While the MySQL client is installed on data1.i2u2.org as part of the default system software, the server is not. Eric therefore built the MySQL server from source code. See the file build_mysql for instructions. The data directory on data1 is /data/mysql. and the configuration file is /etc/my.cnf Eric also built mysqld for www10 for the QuarkNet site using the same script. The data directory is the more traditional /var/lib/mysql, and the configuration file is /usr/local/mysql/etc/my.cnf

The daemon can read configuration information from a file named my.cnf in one of several places, such as /etc, or /usr/local/mysql, or (for clients) the file ~/.my.cnf. You can ask the server to tell you the order in which it searches for the file with the command

$  mysqld --verbose --help |more 

Note that mysqld reads ALL option files, if they exist, in the order listed; it does not just read the first it finds and stops. Options in latter files can therefor override those in the first file read.

A copy of a template my.cnf file is available in the CVS repository mentioned at the top of the page, but you must go through it to verify that the settings are correct for a particular host.

-- Main.EricMyers - 30 May 2008
-- Main.EricMyers - 11 Dec 2008