Setting Up a FreeBSD Lab

by Oscar Bonilla

Say you need to set up a lot of workstations running FreeBSD that will have the same local software installed and have exactly the same hardware configuration. What would you do? Arm yourself with patience and install the OS and applications in each? What about configuration? Wouldn't that take a lot of your time? In the spirit of Unix, you don't want to do repetitive work, you want to automate everything as much as possible. This is what this article is about, how to clone the disk of a pre-set workstation into all of the others.

Prerequisites

Basically you need the server already setup and a sample workstation to clone from. In the lab I set up this way, I used NIS for authentication and authorization, and NFS for file sharing. I was able to setup NIS using the NIS Startup that John Kenagy wrote. NFS has a chapter in The FreeBSD Handbook and is so easy to set up I won't go into detail.

There are two things I would like to go into detail however. One of them is the importance of time. If you haven't set up a time server on your network, this is the time to do so. In particular, if you have NFS, you need a time server. Conveniently, FreeBSD comes with xntpd, which is both a client and a server for the NTP (Network Time Protocol). You can find a nice guide on how to set up a time server on The FreeBSD Diary, so I won't go into detail here - just do it.

The other thing I'd like to talk about is the importance of central logging. When you set up a lab of workstations that are going to be used mainly by students, guess how many times you'll look at the log files of each individual machine. You guessed right, never. If you let your machines log quietly to /var and forget all about it, you'll never know if something bad has happened to one of them. You'll also miss the warning signs that a dying disk might be sending to /var/messages. So it's essential to have a central logging facility (and read the logs too). Once again, FreeBSD is capable of client/server logging using the standard syslog facility. All you have to do is modify /etc/syslog.conf to point to the central logging machine (in my case it's the same NIS/NFS/NTP server) and tell the server to accept logs from the clients. Read the man page for syslogd; it's really simple.

I have DHCP set up in my lab. While not strictly necessary, this really simplifies the administration of the workstations since you can change DNS servers or default gateways, and not have to go to each and every machine to make the change effective. Since I didn't have to install a DHCP server I won't talk about it here, but if anyone has good links to information on how to set up one please email me.

Speaking of DNS, don't forget to be sure that your DNS server knows about the workstations both in the forward and reverse maps. If you find later on the procedure that you're getting refused logins on the server or that the NFS server won't let you mount file systems, chances are that your DNS is not properly configured.

Okay, enough preliminaries. I will assume from now on that you have set up the sample client and it's running smoothly. It's authenticating correctly, mounts file systems, has all the software it needs, is synchronizing with a time server, and logs to a central server. Let's talk about cloning.

Taking a Dump

Now you need to take a dump of the file systems on your client and store them in the server. Let's say you have three file systems on the client.

/dev/wd0s1a mounted on /
/dev/wd0s1e mounted on /var
/dev/wd0s1f mounted on /usr
      

Log in the client as root and add a file called .rhosts to root's home directory. The contents of the file should be just the name of the server. Try to execute the following command as root on the server:

# rsh <client> ls -l
      

Where <client> is the name of your client machine. If you get a listing of the client's root home directory it means you have successfully configured your DNS. DNS? Yes, DNS. If you get a prompt asking you for a password or nothing at all, the first suspect would be the Domain Name Service. Check that and try again.

Ok, now to the actual dumps. On the server, as root, type the following commands:

# rsh <client> dump 0f - / > root.dmp
# rsh <client> dump 0f - /var > var.dmp
# rsh <client> dump 0f - /usr > usr.dmp
      

This will put the dumps in the files you've specified above.

The next step is to have the information about the disk layout in a couple of files to automate the slicing and partitioning of the workstation file systems. I'm assuming that all the workstations will have the same layout and that all the disks are of the same geometry as the sample workstation.

Once again, in the sample workstation, as root type the following command:

$ fdisk

This will tell you all the info about the slices (called partitions in the MSDOS world, just to confuse things). Basically it will give you something like this:

      ******* Working on device /dev/rwd0 *******
parameters extracted from in-core disklabel are:
cylinders=559 heads=240 sectors/track=63 (15120 blks/cyl)

parameters to be used for BIOS calculations are:
cylinders=559 heads=240 sectors/track=63 (15120 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
    start 63, size 8225217 (4016 Meg), flag 80 (active)
        beg: cyl 0/ sector 1/ head 1;
        end: cyl 543/ sector 63/ head 239
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
sysid 160,(unknown)
    start 8225280, size 211680 (103 Meg), flag 0
        beg: cyl 544/ sector 1/ head 0;
        end: cyl 557/ sector 63/ head 239
      

Notice the size field above. This is the size of the disk in cylinders. In this case the whole disk is dedicated to FreeBSD, if you had a Windows partition you would see another partition utilized. Now read the man page for fdisk and pay special attention to the -f flag. Did you read it? Well, if you did you won't need to read this next paragraph.

Create a file (I called mine fdisk.conf) which will have the information on how to partition (slice?) the disk. Here's an example:

g       c8894   h15     s63
p       1       165     0       8404830
p       2       0       0       0
p       3       0       0       0
p       4       0       0       0
a       1
      

You still need another configuration file. Namely the output of the disklabel command. All you have to do is go to the client workstation and run:

$ disklabel -r wd0 > disklabel.conf
      

Cloning the Sample Client

This is the really interesting part, how to automatically replicate the disk of the machine you've already configured to all the other workstations on the network.

I used PicoBSD. PicoBSD is basically FreeBSD on a single floppy. The base distribution has a router, a dial-up server, and other stuff. We'll be creating our own custom PicoBSD installer. You'll need a machine with full sources installed (not X, just the OS) to be able to accomplish this.

Got the machine? Ok, let's go to /usr/src/release/picobsd and take a look at the directories there. You'll see a directory named "net". Copy that directory to your home directory or some other place of your preference. In my case I copied it in ~obonilla/inst/pico.

Now do the following (all directories are below ~obonilla/inst/pico in my machine):

1. Check the Kernel: in conf/ you'll find a file named PICOBSD. That's the kernel config file for the floppy. Needless to say you should delete everything you don't need. The less you have the more space you'll leave on the floppy for more stuff later. Take into account that you'll only be using the floppy to install the OS and therefore you don't really need the parallel port controller, serial port controller, etc. You do need the networking facilities though. Don't even think about deleting those.
2. What commands do you need? in crunch1/ you'll find a file named crunch.conf that contains a list of software to add to your repertoire on the floppy. PicoBSD uses a technique called crunched binaries to squeeze as many programs as it can onto the floppy. What you'll end up with will be a bunch of symbolic links that point to the same binary file (which is comprised of all the programs "crunched" into one large binary file). Read the PicoBSD documentation for more info. Notice the srcdirs section? That lists where the sources for the programs are. You can usually leave this alone. Just below the srcdirs lines there are a bunch of lines that begin with "progs". These lines tell the build system which programs you want included in your floppy. I deleted everything I didn't need (telnetd for example) and added the following programs:
   • disklabel
   • fdisk
   • newfs
   • restore
   • dhclient
   As you can see, all the tools necessary to prepare a new system.
3. Devices: PicoBSD doesn't really care about disk drives. It's usually used on a system without a hard disk and therefore it might not include the device driver for the disk you have. Also, once you have booted with the PicoBSD floppy you won't have /dev/MAKEDEV. What this means is that you have to create all of the device files a priori, i.e. while making the boot floppy and not from the boot floppy.

   You do this by modifying the Makefile in mfs.tree/. If you look at the last few lines you'll see the list of devices that will be created. It should look something like this:

   devnodes: tree
           (cd ${DESTDIR}/dev; \
           ln -s /dev/MAKEDEV; \
           ./MAKEDEV wd0 bpf0 std tun2 cuaa0 cuaa1 cuaa2 vty10 fd0 wd0s1h pty0; \
           rm MAKEDEV )
   	  

   The original Makefile did not have the wd0 and bpf0 devices, I added those in order to support IDE Disks and the Berkeley Packet Filter needed for DHCP configuration.
4. Misc Files: Remember those two files you created on the sample client? namely fdisk.conf and disklabel.conf? You'll have to put those somewhere on the floppy to be able to fdisk and disklabel the new disks. I chose to put them in /etc (on the floppy) and the way you do that is simply to copy them to the floppy.tree/etc/ directory. You should also copy to this directory the files /boot/boot1 and /boot/boot2. These files are needed to make the disk bootable once FreeBSD is installed. If you will be using DHCP, copy /sbin/dhclient-script here too. This script is needed to configure the interfaces and won't be installed by default.
5. /etc files: Say, did you change files such as rc.conf and resolv.conf on the floppy.tree/ directory? Were you surprised it didn't work? Well, PicoBSD has multilingual support! You have to change those files in the lang/ directory and they will be stored as rc.conf.[en,pl] (en stands for English and pl stands for Polish, the author's native language). Unless you know Polish you should change the files ending with .en.

   What to change in those files you ask? Well, remember that the rc.en file is the one that init is going to run as soon as the system is booted, so you can basically automate everything you want done from this file. Here's what I used:

   #!/bin/sh
   # $Id: fbsdlab,v 1.1.1.1 2000/01/25 07:39:12 dn Exp $
   ############################################
   ### Special setup for one floppy PICOBSD ###
   ### THIS IS NOT THE NORMAL /etc/rc !!!!! ###
   ############################################
   mount -a -t nonfs
   rm -f /var/run/*
   dev_mkdb
   
   # dhclient wants dhclient-script in /sbin
   cp /etc/dhclient-script /sbin
   
   # and it has to be an executable
   chmod 500 /sbin/dhclient-script
   
   # get the network configuration via DHCP
   dhclient
   
   # fdisk wants the file /boot/mbr to be there... I wonder why?
   mkdir /boot
   cp /etc/boot1 /boot/mbr
   
   # slice the disk
   fdisk -f /etc/fdisk.conf
   
   # partition the disk 
   # WARNING: don't use the -R flag to disklabel as this will overwrite
   # the stuff you wrote with fdisk
   disklabel -B -b /etc/boot1 -s /etc/boot2 wd0 /etc/disklabel.conf auto
   
   # create the new file systems
   newfs /dev/rwd0s1a
   newfs /dev/rwd0s1e
   newfs /dev/rwd0s1f
   
   # create the mount points
   mkdir /mnt/root
   mkdir /mnt/var
   mkdir /mnt/usr
   
   # This is for NFS mounting the dumps, it's not needed for rsh
   mkdir /mnt/src
   
   # mount the new file systems
   mount /dev/wd0s1a /mnt/root
   mount /dev/wd0s1e /mnt/var
   mount /dev/wd0s1f /mnt/usr
   
   # mount the file system where the dumps are
   # neptuno is my server's name
   mount -t nfs neptuno:/export/dumps /mnt/src
   
   # restore the root file system
   ( cd /mnt/root ; restore rvf /mnt/src/root.dmp )
   rm /mnt/root/restoresymtable
   
   # restore creates stuff in /tmp and the MFS on the floppy is
   # usually too small, so use the /tmp of the root file system
   # we've just restored
   rm -rf /tmp
   
   ln -s /mnt/root/tmp /tmp
   
   # restore /var
   ( cd /mnt/var ; restore rvf /mnt/src/var.dmp )
   rm /mnt/var/restoresymtable
   
   # restore /usr (this takes a long time)
   ( cd /mnt/usr ; restore rvf /mnt/src/usr.dmp ) 
   rm /mtn/usr/restoresymtable
   
   # unmount the file systems
   umount /mnt/root
   umount /mnt/var
   umount /mnt/usr
   umount /mnt/src
   
   # Warn the user :)
   echo 'Would you like to check it or do you feel lucky?'
   
   # finally allow login to go through
   exit 0 
   
   	  

   Note that by modifying rc.en, we've automated the whole procedure. And if rc fails for any reason (we could check return values of the commands) we still have a login prompt we can use. Incidentally the root password that PicoBSD has by default is setup.
6. Create the floppies: Now go to the directory /usr/src/release/picobsd/build and run the build script there. Change the type to custom and point it to the directory where you have all your stuff. If you need more space enlarge the MFS using option 2 (I used 2500). And type n to build the floppy image. At the end of the build process the script will ask you if you want to create the floppies. Insert a new floppy on the drive and answer yes. You can create as many floppies as you like, but once /etc has been copied from the floppy to the MFS, you can safely remove the floppy and use it on another machine.

Don't worry if you don't have a DHCP server on your network. You can just omit dhclient from the programs above and have an interactive rc.en that asks you what IP address, default gateway, etc. to use. Then you would have to boot up each client, type the information necessary for configuring the network, remove the floppy and proceed to the next client.