June 30th, 2014

SSL: different codebases, same IP Address, different sub-domain

So you want to serve different codebases on the same IP Address and port over SSL with different subdomains?

The Problem of different codebases on the same IP

You’re running more than one site over SSL/HTTPS, using a wildcard SSL certificate one the same subdomain, the same ip address and port – but you want to serve different codebases.

The complexity of SSL

You cannot use name-based virtual hosts to determine which host definition and therefore, which SSL certificate or codebase to serve.

This is because the SSL traffic is encrypted, and has to be de-crypted using the appropriate SSL Certificate.

The only way to know which SSL certificate is the correct one, is to use the IP address that the traffic is coming in on.

So far, so good.

But how do you have more than one DocumentRoot or codebase, served by that same, single host definition?

Solutions

Sub-directories and aliases

Yes sure, you can use different subdirectories like this;

	https://mysite.com/site-one

and

	https://mysite.com/site-two

Configured as Aliases:

	Alias /site-one /home/mysite.com/www-ssl/site-one

	Alias /site-two /home/mysite.com/www-ssl/site-two

But this isn’t pretty and customers may find it a little cumbersome.

Directory/Path translation with mod_rewrite and multiple sub-domains

The simpler looking method to serve more than one codebase over ssl depending upon the sub-domain, using mod_rewrite to rewrite the internal file/folder path.

So instead of using an aliased sub-directory, we use a sub-domain to indicate which codebase we wish to use, thus our users access our site(s) like this:

	https://one.mysite.com

and

	https://two.mysite.com

This looks much prettier, and means that we haven’t got to worry about users remembering to put the /subdirectory on the end of the web-address, something that is likely given that most users’ recollection of web-addresses end with “dot com” or similar.

To many users, anything after the TLD is just strange “computer mumbo-jumbo”.

So our solution would look something a little like this:

	# Turn on the rewrite engine:
	RewriteEngine On

	# Specify the condition that this rule is only relevant
	# if the site is being accessed via the URL "two.mysite.com",
	# and that it's Not Case-sensitive:
	RewriteCond %{HTTP_HOST} ^two\.mysite\.com$ [NC]

	# And therefore, re-write the filesystem path
	# to look at the relevant codebase, ignoring any subsequent rules:
	RewriteRule ^(.*)$ /home/mysite.com/www/two/$1 [L]

In practice, in a test scenario, this might look like this:

	#
	#    default document root -
	#    for url's such as www.mysite.com
	#
	DocumentRoot /home/mysite.com/www/default
	#
	#    one.mysite.com -
	#    rewrite to use a different codebase
	#
	<IfModule rewrite_module>
		RewriteEngine On
		RewriteCond %{HTTP_HOST} ^two\.mysite\.com$ [NC]
		RewriteRule ^(.*)$ /home/mysite.com/www/two/$1
	</IfModule>
	#
	#    one.mysite.com -
	#    directory settings
	#
	#    In this example, "/home/mysite.com/www/one"
	#    is actually a symlink to a codebase release elsewhere.
	#
	<Directory /home/mysite.com/www/one>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>
	#
	#    two.mysite.com - rewrite to use a different codebase
	#
	<IfModule rewrite_module>
		RewriteEngine On
		RewriteCond %{HTTP_HOST} ^two\.eaubl\.eu$ [NC]
		RewriteRule ^(.*)$ /home/mysite.com/www/two/$1
	</IfModule>
	#
	#   two.mysite.com    - directory settings
	#
	#    In this example, "/home/mysite.com/www/two"
	#    is actually a symlink to a codebase release elsewhere.
	#
	<Directory /home/mysite.com/www/two>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>
October 13th, 2012

How to configure a samba share with privileged and unprivileged permissions

A reddit user posted a request for help configuring a samba server to share content across their network on http://www.reddit.com/r/linux.

I offer the recipe/how to below as a possible solution.
It’s targeted at an Ubuntu/Debian installation and tested against Raspbian on my raspberry pi about 10 minutes ago.

The main features are

  • the share allows non-privileged users read only access
  • privileged users have read/write access
  • privileges are based upon being a member of the “multimedia” group
  • By default, the process below shares the home directory of a user called “multimedia”
  • this how-to is a start from scratch including installation of the samba server
  • share names/directories and groups are consistent for easy management
  • variables are used in the below commands for easy modification to suit your system/prefs
  • The commands are presented to allow a straight “copy and paste” implementation, hence writing to the samba config file is done using “tee” rather than instructing the user to open the smb.conf in their favourite editor, scroll to the bottom etc….
  • The below text is intended to all be run as terminal commands, by default I’m using BASH. I ran the commands via an ssh session; but they will work equally well pasted into a terminal window, directly on the serving machine itself.


# Most of the following actions require root level access, let's do it as root
sudo su -

# install samba
apt-get samba install

#
# now the hard part - you have to decide where to serve files from
# and how you wish to manage the files/folders and access to them
#
# Let's assume that your "/home" partition is your largest drive/partition
#
# Let's also assume you want to serve multimedia files to users in
# the (arbitrarily named) "multimedia" group - these people can have read/write
# access, all others can have readonly access. The group name is intended to match the use-case and username for consistency.
# We're going to force files written acrosss the share to have
# default permissions of ug=rw, o=r and group ownership "multimedia"
#
# we're going to use a consistent naming strategy; the default owner, group,
# shared directory and share name are all going to match up - it makes life
# easier like that.

# use a variable for the samba daemon name
SMB_DAEMON="samba";

# use a variable for looking at the log
SMB_LOG="/var/log/samba/log.smbd"

# use a variable for the location of the smb.conf
SMB_CONF="/etc/samba/smb.conf";

# use a variable for the share name/group/user etc...
SHARE_NAME="multimedia";

# use a variable for the "share location" - this can easily be changed if you
# want to share a different location
SHARE_PATH="/home/${SHARE_NAME}";

# add a multimedia user (and by default, multimedia group) and create a multimedia home
useradd -m "${SHARE_NAME}";

# how about some default directories?
mkdir -p "${SHARE_PATH}/music" "${SHARE_PATH}/photos" "${SHARE_PATH}/video/films" "${SHARE_PATH}/video/series"

# set the ownership on the new location
chown -Rf ${SHARE_NAME}:${SHARE_NAME} "${SHARE_PATH}";

# the user and group members should have read/write access, others read only
chmod -Rf ug=rwx,o=rx "${SHARE_PATH}";

# backup your existing samba config (just in case)
cp -a "${SMB_CONF}" "${SMB_CONF}.$( date +%Y%m%d_%H%M%S )";

# now we want to add the share to the bottom of your samba config file,
echo | tee -a ${SMB_CONF};
echo | tee -a ${SMB_CONF};
echo "[${SHARE_NAME}]" | tee -a ${SMB_CONF};
echo " comment = ${SHARE_NAME} share" | tee -a ${SMB_CONF};
echo " path = ${SHARE_PATH}" | tee -a ${SMB_CONF};
echo " public = yes" | tee -a ${SMB_CONF};
echo " writable = yes" | tee -a ${SMB_CONF};
echo " write list = +${SHARE_NAME}" | tee -a ${SMB_CONF};
echo " create mode = 664 #ug=rw,o=r" | tee -a ${SMB_CONF};
echo " directory mode = 775 #ug=rwx,o=rx" | tee -a ${SMB_CONF};
echo " browseable = yes" | tee -a ${SMB_CONF};
echo " force user = ${SHARE_NAME}" | tee -a ${SMB_CONF};
echo " force group = ${SHARE_NAME}" | tee -a ${SMB_CONF};
echo " valid users = @${SHARE_NAME}" | tee -a ${SMB_CONF};

# ask samba to reload the configuration to make it active
service ${SMB_DAEMON} reload

#stop being root
exit

# add yourself to the (in this case) multimedia group, which is the same as the SHARE_NAME, thus making you a privileged user allowed the full read and write access
sudo usermod -a -G${SHARE_NAME} $USER

# give yourself a samba user account/password
sudo smbpasswd -a $USER

# start a watch that monitors the samba process, dir and logs for testing
sudo watch "ps faux | grep -v grep | grep \"smb\|PID\"; echo; ls -lrh ${SHARE_PATH}; echo; tail ${SMB_LOG}"

now try browsing to the machine/share across your network; test it and watch the “monitor” whilst you make changes, such as saving files to one of your share directories.

What does the watch tell you?

Below is an example of the output of the watch statement, it shows you:

  • A list of the samba processes – you get to see how many child processes there are, the CPU usage etc
  • a recursive listing of your newly configured directories and contents
  • the tail of the samba log so that you can see whether there are any errors
  • In my case with the default installation there are some minor issues with CUPS, but nothing to worry about at the moment.
Every 2.0s: ps faux | grep -v grep | grep "smb\|PID"; echo; ls -lrh /home/multimedia; ech...  Sat Oct 13 11:02:14 2012

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      2977  0.0  1.3  18864  3272 ?        Ss   Oct12   0:00 /usr/sbin/smbd -D
root      2995  0.0  0.4  19380  1160 ?        S    Oct12   0:00  \_ /usr/sbin/smbd -D
root      4828  0.0  1.2  19152  2872 ?        S    10:38   0:00  \_ /usr/sbin/smbd -D
root      4836  0.0  1.2  19152  2872 ?        S    10:38   0:00  \_ /usr/sbin/smbd -D
root      4851  0.0  1.2  19152  3056 ?        S    10:38   0:00  \_ /usr/sbin/smbd -D
root      4881  0.0  1.4  19348  3356 ?        S    10:38   0:00  \_ /usr/sbin/smbd -D

total 16K
drwxrwxr-x 4 multimedia multimedia 4.0K Oct 13 10:13 video
drwxrwxr-x 2 multimedia multimedia 4.0K Oct 13 10:13 photos
drwxrwxr-x 2 multimedia multimedia 4.0K Oct 13 10:13 music

[2012/10/13 10:46:18.290102,  0] printing/print_cups.c:487(cups_async_callback)
  failed to retrieve printer list: NT_STATUS_UNSUCCESSFUL
[2012/10/13 10:59:18.995271,  0] printing/print_cups.c:110(cups_connect)
  Unable to connect to CUPS server localhost:631 - Connection refused
[2012/10/13 10:59:18.996708,  0] printing/print_cups.c:487(cups_async_callback)
  failed to retrieve printer list: NT_STATUS_UNSUCCESSFUL
[2012/10/13 11:02:19.779253,  0] printing/print_cups.c:110(cups_connect)
  Unable to connect to CUPS server localhost:631 - Connection refused
[2012/10/13 11:02:19.780782,  0] printing/print_cups.c:487(cups_async_callback)
  failed to retrieve printer list: NT_STATUS_UNSUCCESSFUL

Hit CTRL + c to terminate the “watch” monitor.
Enjoy!

Tags:
February 10th, 2012

Tunneling ILO over ssh

Do you want to access HP ILO via a SSH tunnel?

HP’s iLO (Integrated Lights Out) is great, especially if you wish to administer the server remotely for the installation of the OS, check the video output for a kernel panic when the OS is unresponsive, configure the BIOS, or just so that you can avoid standing in a cold server room or because you’re just not at the premises.

The servers that I administer come with iLO1, iLO2 or iLO3 and it’s great to see the interface developing at every release.

If the remote server is located on a remote LAN that does not expose the server to the outside world e.g. a DMZ you can tunnel a connection over ssh to provide access both to the remote iLO web interface over https, and the remote console that allows you to access the display of the remote machine via a browser based Java applet, as if you were using keyboard, mouse and monitor plugged into the machine itself.

So, how do you access iLO remote console over ssh?

To do so, you just need to forward a couple of ports via a handy ssh server somewhere on the remote network.

You may need to refer to some of the port settings within the iLO administration pages, I’ve included a screenshot of where to find them within iLO1 below:

hp ilo1 administration globalsettings remote console port

The below steps assume that you have a proven, working ssh server on the remote network and the necessary ports are forwarded on your remote router to allow you access to the remote network and therefore, the iLO on the remote server.

  1. find the necessary ports for java, Administration -> Access Settings -> Servicenote the port that your iLO is listening on for “remote console” – this is the port that the server will expect the Java remote console to be able to communicate over.
  2. Now you need to create the relevant ssh tunnel, I’ve broken this down to a command with some bash variables to make it easier to understand what’s what.
    1. the IP of iLO on your remote network
      iloaddress=192.168.1.10
    2. the local https port to forward
      localhttps=8443
    3. the https port the iLO is listening on
      remotehttps=443
    4. local port to forward for Remote Console (probably should be the same as the Remote Console Port within: Administration -> Access Settings -> Service)
      localconsole=17990
    5. the Remote Console Port within: Administration -> Access Settings -> Service
      remoteconsole=17990
    6. Your username for ssh-ing into a server on the remote network
      sshusername=<your-user-name>
    7. Your remote IP or domain name for the remote network
      sshserver=my-secret-router.dyndns.org
    8. the port on the remote router for forwarding to an ssh server on the remote network
      sshport=2205
  3. Now run the ssh command to bring up the tunnel using the above parameters:
    ssh -fnNL ${localhttps}:${iloaddress}:${remotehttps} -L ${localconsole}:${iloaddress}:${remoteconsole} ${sshusername}@${sshserver} -p ${sshport}
  4. Finally, if all is well, access the iLO via your browser, I’m launching it via the command line here to provide the correct port based upon the pre-populated variables:
    firefox https://127.0.0.1:${remoteconsole}
  5. You can check that the tunnel command is still running with the following command:
    ps faux | grep -v grep | grep "ssh -fnNL ${localhttps}:${iloaddress}:${remotehttps}\|USER"
  6. Handily, this also gives you the PID(Process id) of your tunnel, so you can kill your tunnel like this:
    kill <process id>

Let’s put the above together as a single block that you can paste into a text editor to make it into a bash script, all in one go along with bash comments to document it (once you’ve edited the parameters of course):

#!/bin/bash
# the IP of iLO on your remote network
iloaddress=192.168.1.10
# the local https port to forward
localhttps=8443
# the https port the iLO is listening on
remotehttps=443
# local port to forward for Remote Console
localconsole=17990
# the Remote Console Port
remoteconsole=17990
# Your username for ssh-ing into a server on the remote network
sshusername=<your-user-name>
# Your remote IP or domain name for the remote network
sshserver=my-secret-router.dyndns.org
# the port on the remote router for forwarding
sshport=2205
# launch tunnel
ssh -fnNL ${localhttps}:${iloaddress}:${remotehttps} -L ${localconsole}:${iloaddress}:${remoteconsole} ${sshusername}@${sshserver} -p ${sshport}
# check tunnel
ps faux | grep -v grep | grep "ssh -fnNL ${localhttps}:${iloaddress}:${remotehttps}\|USER"
# launch browser
firefox https://127.0.0.1:${remoteconsole} &

Remember to make the script executable once you’ve saved it with a chmod +x /path/to/script

November 5th, 2010

Permissions, permissions, permissions – part 1

Whenever “the guys” have a problem accessing some shared resource, it usually comes down to permissions, either:

  • file/directory permissions on the server
  • samba permissions
  • user permissions
  • permission masking on the users local machine for their mounts

Today I’d like to talk about file/directory permissions.

Your number’s up

The first time that I was introduced to *nix style file permissions, the listing of “permission types”

  • execute
  • write
  • read

were explained to me “numerically”.  The explanation of how to set the permission mode on a file using chmod “CHangeMODe” were demonstrated “Numerically”:

chmod 0654 somefile.txt

and do you know what? In all of these years, I *still* cannot remember what each number means.  I know that each permission has a numeric value, and that they can be combined using binary “and-ing”, but I just can’t recall that 1=execute, 2=write and 4=read.

It’s not big and it’s not clever

I remember programming as a kid, learning ASCII and coding in BASIC on a Tandy TRS 80, on a Toshiba MSX and various friends Sinclair Spectrums.  I learnt that you could check which key had been pressed by comparing the trapped key with it’s ASCII value, so to see if the “A” key had been pressed, you’d check it against it’s ASCII value “65″.

But what I later learnt is that proramming code littered with obscure codes does little to aid readability and maintainability, it does not lend itself to error-free code.

So I switched, now if I were to write some code to compare with say, the letter “A” I would do just that – compare with the letter “A”, not the ASCII value 65.

Keep it simple, stupid

So it is also with permissions.  I don’t “chmod” numerically, I do it the much simpler, easier and less error prone way.

chmod supports textual permissions specification, so that’s what I use, for example, to specify that the “user” of a file may read,write and execute it, that memebers of the files’ group may read and execute it, and that any others may only execute it (ok, a somewhat contrived example I’ll admit)  we can choose to specify the permissions either numerically, or textually:

  • chmod 751 somefile
  • chmod u=rwx,g=rw,o=x somefile

Ok, so which is more readable and explicit in it’s meaning? Yes I’ll admit the “textual” example is a little more verbose and perhaps doesn’t carry the “geek cred” that some might seek to attain, but come on, we’re supposed to be getting on with stuff, not pretending to have some mystical knowledge based upon some obscure numerical code.

Perhaps it’s a sign that I don’t use the numerical codes – but whilst putting the numeric example together above, I had to go back and make a correction to one of the permission codes as I got it wrong…

Suddenly, It all makes sense

The other thing about using the textual method to specify permissions, is that, if you take a look at a directory listing, it all comes together nicely, take a look at some example files:

---x------ 1 root root     0 Oct 31 10:05 0100.perm
--w------- 1 root root     0 Oct 31 10:05 0200.perm
--wx------ 1 root root     0 Oct 31 10:05 0300.perm
-r-------- 1 root root     0 Oct 31 10:05 0400.perm
-r-x------ 1 root root     0 Oct 31 10:06 0500.perm
-rw------- 1 root root     0 Oct 31 10:06 0600.perm
-rwx------ 1 root root     0 Oct 31 10:06 0700.perm

Yup, the permissions are expressed textually.  Because it’s more accessible, because it’s readable and it’s not so obscure.

But as you might have guessed from the filenames, what I’ve actually done here is effectively create a “crib sheet” for myself, – the filename of each file is actually the numerical, octal equivalent of the permissions assigned to that file.

Why would I do that?  Well, that’s simple, because when you’re editing config files for systems such as logrotate you still have to specify the permissions for newly created empty logfiles in octal.

Is there cake? Can I eat it?

As I can never remember them, I made the “crib sheet”.  In my opinion, it’d be “safer” and easier to specify the permissions in either the same style that we see in the directory listing: -rwx—— for example, or preferably, as you would with chmod: u=rwx,go=

But I guess you can’t have everything.

October 27th, 2010

SED – Edit in place

example of "in-place" editing with sed

example of "in-place" editing with sed

I use the GNU sed command (sed = Stream EDitor) to process text files on a regular basis.

Sed is brilliantly fast.  It’s perfect for tricks such as altering the content of a huge database dump, or a spreadsheet or csv file.

One of my colleagues routinely opens csv files in a spreadsheet application to edit some item of other, only to find that upon saving, the dates have get converted from ISO to UK format.  Using sed can avoid this frustration and speed up his workflow.

“S” is for “Substitute”

To my shame, I typically only use the ’s’ option – to substitute one piece of text for another.

But today I finally found another use, which I had been sure must surely exist; that of deleting a specific line from a file based upon it’s position.

The scenario is that I often ssh into boxes, when setting up servers for instance, that have the same IP address as a previous server, (because they initially use DHCP to acquire an address).

This means that when ssh compares the finger print to it’s known_hosts file, it balks, tells me the fingerprint has changed, what line the existing fingerprint is on but wont let me connect.  This usually means that I edit my known hosts file, remove the offending line and then I can get on.

Now that I’ve found out how to do this with sed however, it’s much easier.

“i” is for “in-place”

In the past when I’ve used sed, I’ve either piped the output to another command using the “|” pipe character, or redirected it to another file using “>” .
But this isn’t so convenient when you want to actually edit a file.  The magic switch is “-i” which means “edit in place” the command syntax is:

sed -i ‘<nth line>,<number of lines to apply action to><action>’ <filename>

e.g. sed -i ‘2,1d’

Find the 2nd line, then for 1 line, delete the line.

Testing

First I created a test file:

$ cat > test.txt
one
two
three
four
[CTRL + D]

We can then test using sed to modify the file (as sed will typically output the results to standard out – i.e. to the terminal)

$ sed -e '2,1d' ./test.txt

Let’s take a look at the file now:

$ cat test.txt
one
three
four

Result

Hurrah! the line has gone!

So now, when ssh balks because the host key of the remote system differs to line x of ~/.ssh/known_hosts changed (usually because I’ve re-installed the OS at the other end) instead of firing up an editor, such as nano or gedit to remove the offending line, it’s just a straight forward “sed -i ‘x,1d’ ~/.ssh/known_hosts” and the old key is gone.

August 5th, 2010

Guide to CentOS Network installation via HTTP

You may have seen my voicing of my frustrations at the apparent lack of guidance to installation of CentOS 5.4 via Network / HTTP.
This is my step-by-step guide for installing CentOS 5.4 via network over HTTP. Or FTP.

I was installing upon a virtual machine using Virtual Box OSE, but the steps below should pretty much apply to a real/physical installation.
This is a description of what *I* did, your mileage may vary. The usual warnings apply; do this at your own risk – I accept no responsibility for your actions, whether you follow my *guide* or not. It is after all, *my guide*, that is, it is a guide for my own future reference.

  1. Getting the Net installer ISO.
    1. Use your web-browser to navigate to the
      centos.org mirror list
    2. I’m going for the 64bit Install of CentOS 5.x: so I visited the following Urls to find a mirror in the UK:
  2. Mount the ISO onto your virtual machine
  3. Start the virtual machine
  4. Textual set-up (Pre GUI)
    1. When prompted, I pressed the [ENTER] key for a GUI based install
    2. I selected my language, “English”
    3. I selected my keyboard type, “uk”
    4. I selected my Installation Method, “HTTP”
    5. I used the default settings for the “Configure TCP/IP” screen
    6. HTTP Setup
      You need to find an online source for the installation files.
      I decided to use the same mirror for the Net/HTTP installation as I did to download the Net installer, so I did the following:

      1. navigated two levels up from http://anorien.csc.warwick.ac.uk/mirrors/centos/5.4/isos/x86_64/
        to the directory http://anorien.csc.warwick.ac.uk/mirrors/centos/5.4/
      2. navigated down via “os” -> “x86_64″ to http://anorien.csc.warwick.ac.uk/mirrors/centos/5.4/os/x86_64/
      3. I broke the url down into it’s constituent parts; server “Web site name” and the subfolders “CentOS directory” thus:
        Web site name: anorien.csc.warwick.ac.uk
        CentOS directory: mirrors/centos/5.4/os/x86_64/
      4. The Net installer will attempt to retrieve a file, “stage2.img” from the details provided in the previous step.If this fails, then either:
        • The url is wrong
        • the file doesn’t exist
        • there’s a problem with your networking
  5. GUI installer phase
    1. Eventually, after about 10 minutes, I was greeted with an X-session/GUI installer. I clicked [Next].
    2. This is a fresh (virtual) machine, so I selected the defaults for disk partitioning. I clicked [Next].
    3. I’m only using a small amount of ram, about 320MB for this VM. So I allowed the installer to activate a swap partition.
    4. I selected the nearest city representative of my timezone and location via the map;
      London, Englad, United Kingdom. [Next].
    5. This is going to be a cli based server, so I de-selected the “Desktop – Gnome” option and selected the “Server” option.
      I clicked [Next].
    6. I entered a nice, long, obscure password for my “root” user administration account.
  6. The installer was ready to begin installing the system. I clicked [Next]
  7. I watched the install whilst attending to other duties.
    Eventually I was greeted with the “Congratulations, the installation is complete” message. I clicked [Reboot].
  8. After un-mounting/ejecting the Net installer ISO, my machine booted.I was presented with some options via the “Setup Agent” – I went with the defaults, selecting the “Exit” option.
    Eventually I was presented with a nice, textual, login prompt.

Sometimes you just need a shell

One issue that I had whilst using the Graphical installer, occurred during a subsequent CentOS installation on a physical (rack) server.
A post-partitioning screen asks the user to specify upon which device to install the bootloader.
Unsure which device was the correct one, I tried an old fallback plan to get a shell (command line terminal); hitting CTRL + ALT + F1, which actually worked.
This yielded a prompt that allowed me to mount and list the contents of the various disk devices to determine which was the correct one.
Returning to the graphical installer was merely just a case of sequentially working through the F-keys in combination with CTRL + ALT until I found the right one.

May 13th, 2010

Frustrations with CentOS 5.4 via Net Installation

It’s raining outside, my girlfriend is in bed with a cold and I’ve retreated to the office with the cat for company.
Today I’m setting up a pair of web and database servers, using virtual box for replication and disaster recovery testing.
The OS of choice for todays experiments has naturally been chosen to match that of some of our real-world production web and database servers: CentOS 5.4

Despite being a self confessed Ubuntu/Debian derivative fan, I’ve found that CentOS does provide a comfortable environment within which to work, although there are a few niggles that bother me with the installation:

  1. You can’t install from the LiveCD.  Well, not unless you’re willing to jump through hoops to do so. (Some instructions expect a “roll-your-own ISO” approach)
  2. If you choose to go the “Net Install” route, you are not provided with a list of http or ftp mirrors from which to install – you are provided merely with a couple of entry boxes in which to specify the source ftp/http server and path.

Now come on folks, it isn’t hard to to provide a default list of mirrors within the installer, or at the very least, a facility within the installer to at least download a list of mirrors, or else a straight forward guide on the CentOS.org site with links leading to the appropriate urls.

The CentOS website appears to be based upon the basic assumption that users are interested in installation via the downloading of a CD or DVD .ISO image.  Were installation via the Live CD a straight forward matter, then this wouldn’t be such an issue, but it’s not a straight forward matter.

Sure, I could download the Installation DVD image – but that’s comparitively huge.  I don’t really want *ALL* of those packages for a server install do I?
No, the Net install .ISO is perfect at a mere 9.4MB.  Yup – that’s all it is.  I think that the target audience is that of sys-admins looking to roll out across multiple machines on a LAN. But it also works for single installations, across the internet.

The process is, simple.  You boot from the Net Install .ISO, answer some questions regarding language, keyboard, and installation method.  It’s perfect.

CentOS 5.4 Net Install dialogue - FTP

CentOS 5.4 Net Install dialogue - FTP

Not quite perfect

Perfect save for one problem – trying to get the URL details for those “precious” servers that are able to provide the packages and resources for the Net install to work.
Finding those details via the CentOS website just doesn’t seem that easy.

Now remember, Google is our friend, so we turn to the Big “G” next, surely this’ll find a mirror list and a guide on the CentOS site for us?

Er… No.

No, the links all seem to centre around requests for help from those stranded out at sea, waiting on some assumed bug or other (I think it’s just a problem of the wrong URL being used) to be fixed, or questions about how to set up an ftp server on their network to then proceed with the network installation.

No official concrete guide.

Come on people, we can do better than this.  A How-To and a mirror list that is tailored to Network installation via ftp or http is a must and should be immediately accessible on the CentOS installer and download pages.

Eventually, I found a url;

http://ftp.linux.ncsu.edu/mirror/mirror.centos.org/5.4/os/x86_64/images/boot.iso

Entering it into the dialogue box and hitting [Ok] caused the dialogue to disappear, and then re-appear – no error information, no user feedback at all.

after much url editing, tweaking and testing, I managed to get the details down to:

FTP site name: ftp.linux.ncsu.edu
CentOS directory: /mirror.centos.org/5.4/os/X86_64

After a short delay and some flashing from the virtual NIC, I was presented with the following message:

Unable to retrieve
ftp://ftp/linux.ncsu.edu//mirror.centos.org/5.4/os/X86_64/images/stage2.img.

Aside from everything else, (i.e. the relevant file not being present) it looks like that trailing forward slash isn’t necessary for the ftp server.

More googling yielded the following thread:

http://centos.org/modules/newbb/viewtopic.php?viewmode=thread&topic_id=24961&forum=37&post_id=102175

Armed with the info from the “not quite successful” stage2.img we can start inferring what info we really need, so it’s time to hit the mirror list at http://mirror.centos.org/centos/5/ to see whether we can find a good, local mirror and try a bit more URL tweaking to get us to where we want to be.

http://www.centos.org/ -> Downloads -> Mirrors leads us to:

CentOS Downloads -> Mirrors

http://www.centos.org/modules/tinycontent/index.php?id=15

Again, this page is structured around the assumption that we want to download an ISO. We don’t, but we do want 5.4, so lets follow that “CentOS-5 ISOs” link and see whether we can find something more to our liking in the tree

Interesting, lets climb the directory tree and see whether there are any sibling folders that we may make use of?
Ah! the “OS” directory – let’s explore that and see whether the files that we need can be found in there?

Bingo, we finally navigate to http://mirror.centos.org/centos/5/os/x86_64/images/ and find this, complete with “stage2.img”

Lets use that url, selecting http rather than ftp.

CentOS Net Installation HTTP Setup

CentOS Net Installation HTTP Setup

Bingo!

After about 10 minutes (with my connection, machine etc..) We get an X screen and a sexy GUI based installer.

Sexy CentOS 5.4 GUI Installation screen

See my “Guide to CentOS Network installation via HTTP” for the straight forward installation procedure.

May 3rd, 2010

Who, what, why. An introduction.

Who am I?

I’m Isaac, I’ve been working as a developer and IT specialist since the mid to late 1990’s.  Over the years I’ve been employed, contracted and self-employed.  I’ve handled hardware, networking, developing and administering.

I’ve always said that the WORST thing about being in I.T. is that you can sometimes spend half your working day on the floor, crawling around under someones desk, but that the *BEST* thing about being in I.T. is that you get to spend half-your working day on the floor, crawling around under somebodies desk.

What do I do?

I’ve worked as an application developer in MS Access, VB and C. As a web developer in ASP 2 and 3 with IIS, MS Access and MS Sql server, Macromedia Director, Java and all of the usual technologies; html, css, javascript.

Sometime during the “noughties” I switched into Apache, Mysql, PHP,  PostgreSQL and embraced the open source route.

These days my time is split between development with PHP/PostgreSQL and being a Linux sys admin, managing web and database server clusters, lan servers and desktops, using distributions such as Ubuntu, CentOS, RHEL, Fedora and occasionally Gentoo.

*NIX

I first discovered that I liked the “Unix way” whilst studying software development in C, using a Xenix based multi-user system.  Something just seemed “right” about it.  I enquired of my instructor how I could find a version of Xenix for home – he mentioned the existence of Minix.  I never got into Minix, but then I stumbled upon GNU/Linux.

Over the following years I tinkered with Caldera, DSL, DSL-N, Eeebuntu, Kanotix,  Knoppix, Linux Mint, Mandrake, Mandriva, MuLinux, the old Red Hat releases, Slackware, SUSE and probably flirted with a few other Linux distributions, before finally falling in love with Ubuntu.

I do have to admit that I favour the Debian derived distributions.  I think that it’s something to do with the well thought out documentation and file system layout that I prefer.

But that’s not to say that I think that the Debian way is necessarily “better” than say the Red Hat Enterprise Linux / CentOS way – I just find the Debian way a little more comfortable, in the same way that I prefer my old diesel saloon to my newer petrol estate car.

Why this blog?

This blog is intended as my repository for the things that I learn and wish to share, to provide a resource for myself for those moments when I catch myself staring at a screen whispering the words – “now how did I do this last time?”

If anybody else benefits from these notes then so much the better!