I recently needed to install FreeBSD on an ALIX 2D13 board. I decided to try PXE booting the installer, then installing to the compact flash card. There are other ways to accomplish this, but I'd been looking for an excuse to play with PXE.
There are a few main components to getting PXE booting working. The target board needs to use DHCP to get some specific information about where to find the PXE bootloader and root filesystem it needs. Then it uses TFTP to get the PXE bootloader and NFS to mount the filesystem.
The system serving TFTP and NFS can be the same one running the DHCP server, or not. In my case, I used my laptop for everything because I can't customize the DHCP configuration on my DD-WRT router. I'll replace it with an OpenBSD router soon.
These are the detailed instructions for exactly how I made this work. I pieced them together from multiple incomplete or outdated sources.
NFS
-
Get a FreeBSD image.
fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/i386/ISO-IMAGES/10.1/FreeBSD10.1-RELEASE-i386-disc1.iso
I'm using the full install image rather than the bootonly version because the ALIX board isn't going to have internet access while I'm doing this.
-
Create a directory for the installation files. We'll be serving this over NFS later.
mkdir /usr/public/fbsd10
-
Mount the ISO image you just downloaded and copy the files into this directory.
mdconfig -a -t vnode -f FreeBSD-10.1-RELEASE-i386-disc1.iso mount_cd9669 /dev/md0 /mnt cp -R /mnt/ /usr/public/fbsd10 umount /mnt mdconfig -d -u 0
-
Comment out the CD line in
/usr/public/fbsd10/etc/fstab
#/dev/iso9660/10_1_RELEASE_I386_CD / cd9660 ro 0 0
-
Export the installation directory over NFS in
/etc/exports
by adding this line./usr/public/fbsd10 -ro
This gives anyone who can connect read-only access.
-
At this point, NFS is ready to go. Start it up.
service nfsd onestart
It will start
mountd
andrpcbind
as dependencies. If you make any changes to/etc/exports
then you must restartmountd
TFTP
The target board will use TFTP to fetch the pxeboot binary.
-
Create a directory to hold the TFTP files.
mkdir /usr/public/tftp
-
Enable TFTP in
/etc/inetd.conf
tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /usr/public/tftp
This line should already exist, you just need to uncomment it and adjust the path on the end.
-
Copy the pxeboot executable from the installation files to the tftp directory.
cp /usr/public/fbsd10/boot/pxeboot /usr/public/tftp
-
Start inetd.
service inetd onestart
Now TFTP is all set.
DHCPD
-
Install isc-dhcpd43-server.
pkg install isc-dhcpd43-server
-
Add something like the following configuration to
/usr/local/etc/dhcpd.conf
subnet 10.0.0.0 netmask 255.255.255.0 { range 10.0.0.2 10.0.0.3; option subnet-mask 255.255.255.0; option routers 10.0.0.1; option broadcast-address 10.0.0.255; option domain-name-servers 10.0.0.1; # TFTP server next-server 10.0.0.1; # PXE boot file filename "pxeboot"; # root filesystem option root-path "/usr/public/fbsd10"; }
-
Add the following lines to
/etc/rc/conf
ifconfig_re0="inet 10.0.0.1 netmask 0xffffff00" dhcpd_ifaces="re0"
This sets our ethernet port (re0) with a static IP and tells dhcpd to use it.
-
Start dhcpd.
service isc-dhcpd onestart
Testing
-
Make sure you can grab pxeboot over tftp.
tftp localhost _> get pxeboot
Look in
/var/log/xferlog
if something is wrong. -
Make sure you can mount the NFS share.
mount -t nfs 127.0.0.1:/usr/public/fbsd10 /mnt umount /mnt
Look in
/var/log/messages
if something is wrong. If you have any trouble, double check your/etc/exports
syntax and restartmountd
again. To list all your shares, use:showmount -e localhost
PXE Booting
-
Time to PXE boot the ALIX board. Connect the laptop to the board with ethernet. Connect your null modem FTDI serial cable, run
screen /dev/ttyU0 38400
, and power it on. Presss
while the BIOS is starting up to get the BIOS menu. Change the baud rate to 9600 and enable PXE boot. Save and exit. -
Restart screen without the baud rate argument and watch it (hopefully) PXE boot. If you see gibberish, you don't have the baud rate right. If it can't find the DHCP server, you're probably using the wrong ethernet port.
Installing
-
Go through the FreeBSD installer like normal, installing to the CF card. Choose 'Guided UFS'.
-
Make sure you don't install the ports tree or the system source. We want to spare the CF card as much as possible.
-
Make sure the hostname is a FQDN -- I used alixbuild.lan. If you don't do this, it will try to determine it during boot, which will cause it to hang for a minute.
-
Add a user if you like. Invite the user to
wheel
andoperator
groups. Only members ofwheel
can su to root.
After the installer runs, ask for a shell in the chroot. There are several changes we need to make.
-
-
Edit
/etc/fstab
and add the optionnoatime
to the root filesystem. It should look like this:# Device Mountpoint FStype Options Dump Pass# /dev/ada0p2 / ufs rw,noatime 1 1 /dev/ada0p3 none swap sw 0 0
If you don't disable atime, it will write to the CF card every time a file is accessed, which will significantly shorten its life.
-
Edit
/etc/ttys
and specifyoff
for eachttyv
line. Find thettyu0
line and make sure it looks like this:ttyu0 "/usr/libexec/getty std.9600" vt100 onifconsole secure
The startup script
/etc/rc
reads this file during boot to determine where to listen for logins with getty and how many virtual consoles to spawn. Because this board has no VGA output and we won't be connecting a keyboard to it, we don't need the normal virtual terminals.We also need to make sure that the first serial terminal is set properly. The getty setting should match the baud rate we're using. In this case, it's
std.9600
-
Add the following lines to
/boot/loader.conf
autoboot_delay="3" beastie_disable="YES"
This disables the ACSII Beastie display and shortens the delay to something more appropriate for an embedded device.
-
Edit
/etc/ssh/sshd_config
and setPermitRootLogin
toyes
. This will allow us to ssh in as root, which is disabled by default. -
Add the following lines to
/etc/rc.conf
ifconfig_vr1="DHCP" sshd_enable="YES"
This tells it to connect to a router on the
vr1
interface and enables ssh. This way, if the serial port doesn't work for some reason, we may be able to ssh in and recover.
Booting Up
Reboot the system and disable PXE boot in the bootloader. It should successfully boot from the CF card.
At this point, you can connect the vr1
interface to your router (or the
DHCP server still running on your laptop) and ssh in. You can also talk
to it over the serial port if you like seeing text print at 9600 baud :)
Update your system with freebsd-update fetch install
and reboot again.
Next Steps
Now that your system is up, you may need to build ports if you need custom flags set. I recommend creating a new NFS share on your laptop to store the ALIX board's ports tree. Then mount it from the board and grab a ports tree.
mkdir /usr/ports
mount -t nfs 10.0.0.1:/usr/public/ports /usr/ports
portsnap fetch extract
Now when you build something, the file storage happens on your laptop over NFS, which will make the CF card happy.
Conclusion
There were a lot of things to get right because I combined PXE booting with running FreeBSD on an embedded device, but it worked well. I got hung up with NFS refusing to mount for a while because I forgot to restart mountd
or had the syntax in /etc/exports
wrong. But it was pretty straightforward.