SPIE.org : SPIE Newsroom : Ultrashort-pulse laser ablation: insights from molecular-dynamics simulation

SPIE.org : SPIE Newsroom : Ultrashort-pulse laser ablation: insights from molecular-dynamics simulation

When PXE saves the day...

It was so shocking last night when one of our filers crashed. It would have not so terrible if the mainboard is , say, normal.  This system is my self built NAS-station, having 6 SATA ports. Yes, it is not normal, since all the ports are used by the harddisk, and I used "internal" USB slot to boot. I had no choice. The weird thing is that no IDE connector exist, but the old-useless-ugly-space-wasting-floppy-disk-connector is there! I never understand brain damaged companies that keep retaining this connector. To my opinion IDE is much more useful like for plugging a flash-disk.

The headache is, no direct access to the server room, no available replacement of the USB stick, and no port to plug a harddisk. It was time to think of a strategy. The only hope is network boot, pxe, as the boot server is alway running in the head. Thank to IPMI, I could just do everything remote although the speed is somehow irritating. Well the strategy is, boot to network, use ramdisk (tmpfs). This should be easy, I thought.

I worked on automating node installation using pxe, tftp, http, etc, a couple of years ago. So I dig again a bunch of scripts I wrote. It functioned, and it should still function this time. This is actually a brilliant idea of mine, if I'm allowed to say so, that to an unregistered node or any alien computer is given an installation boot scheme. One need only to create pxe boot configuration named before the mac address of the ip given by the dhcp server in hexadecimal form. Deleting the file would send a node to the default configuration, which is the installation. This default scheme is created by modifying initrd to allow loading installation script from the http server on the head. It's quite efficient and practical, since a user may change the script according to his need, not only for installation purpose. I called this a respond script, because it is used as a respond to the request sent by a booting node. The node send information of it's configuration (minimalist) such as mac addresses, information about hardisks in the request which is named by the first mac address.

The respond script will be executed by the node. To install a node it may have a straight forward step.

1. Format hardisk
2. Create disk image, maybe using debootstrap, etc.
3. Load harddisk content from the server and put it in the harddisk. I used wget to take a targz-ed disk image. Quite efficent.
4. Make a correct PXE boot configuration and let the node reboot.

This time I failed to use the scheme for installation. The reason is, that mdadm and lvm2 can not be installed in the debootstraped directory. So I used it just to diagnose the broken NAS.  I only need to put 'sh' in the respond script to put me in shell command line.  So I installed first a lazy computer (it did not do any work) and make sure that everything works fine and able to load mdraid and lvm volumes, and take the targz-ed image from it.

Then It came to my mind, it should be easier just to put the disk image inside the initrd. Yes, the size explodes but who cares. One needs only about 250MB unzipped space for NAS. I modified initrd so it mount a tmpfs as root, and untargzs the attached disk image there.  Now, actually I did not need the scheme I told you before, but I don't feel guilty telling you that because this idea came after meshing around with it. So I need only two files, the kernel and the fat initrd.

In init script inside initrd file, there are two options to load root file system, local and nfs. I could have added new options, say ramdisk, but I decided to do it in a rude way: force the system to use ramdisk.
A simple inclusion in scripts directory, called 'ramdisk' is sufficient:

# ramdisk

mountroot ()
{
  wait_for_udev 10
  mount -t tmpfs none ${rootmnt} -o size=2048M
  tar xzvf /disk.tgz -C ${rootmnt}
  rm /disk.tgz
}

and set $BOOT variable in init script to ramdisk. Actually I did it hard way, call directly scripts/ramdisk before 'maybe_break mountroot'.

Boots... and smiled.... I then did ssh to the node and all the raids and lvms are there. Mount them, export to nfs, done. At the end I copied the kernel and initrd also to a USB disk. Now it's better because the disk is only used for loading the image in boot time.

Notes:

• debian squeeze is used.
• unpacking initrd: gzip -dc path/to/initrd-file|cpio -id
• repacking initrd: find .|cpio -H newc -o|gzip > ../initrd-ramdisk , from the unpacked directory.
• Services on server:
• dhcp/bootp
• Required packages for NAS:
• mdadm
• lvm2
• rsync
• ssh
• modification needed in the disk image:
• /etc/fstab
• /etc//udev/rules.d/70-persistent-net.rules
• /etc/exports
• cleaning of unnecessary file under /usr and /var

Using ipmitool @SuperMicro server

It' about time using IPMI instead of KVM-switch. In fact this capability is all the time there in our servers. Don't ask why I did not do this from the beginning.
Anyway, this is how to redirect console output to serial over lan (SOL), and controlling power via IPMI.
In fact a SuperMicro server has already embedded IPMI in the mainboard, which is accessible using http connection (Luckily its true for me). It has as well a KVM over ip facility, using java's iKVM. But, this java dependency is not so "flexible" for my need, regarding the fact that java plugin does not (currently) always work (particularly in x64). I think console application is still the perfect tool. So, I put my eyes to impiconsole and ipmitool.

Before using remote management, first you have to do some BIOS settings. This figures explain those settings, quite clearly:

Remote access settings 

IPMI IP settings

Notice that COM3 must be used to make SOL works properly.

After all the settings are correct, you may direct a browser to the specified ip address, and the server can be fully controlled using browser. Console redirection will work if java plugin is configured correctly (under "Remote Control" menu).

Now install ipmitool and freeipmi-tools on the managing server (client, or whatsoever):

apt-get install ipmitool freeipmi-tools

Ipmitool is used for controlling/managing the server. It has a complete set of instructions. In fact, only "power on" and "power off" command are the most useful ones. Example:

ipmitool -H 192.168.0.100 -U ADMIN -P ADMIN power on

to power on the unit. ADMIN and ADMIN is the default user name and password.

To enable access using SOL, the following configuration must be done:

1. @/boot/grub/grub.cfg add this:

serial --unit=2 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
and add "serial console=ttyS2,115200n8" to the "linux" line.

Don't use splash!
(Yes, grub2 configuration file looks a bit weird...)

2. @/etc/inittab add this:

s0:2345:respawn:/sbin/agetty ttyS2 115200

3. Make sure that ttyS2 is listed in /etc/securetty

Thats it. Now call:

ipmiconsole -h 192.168.0.100 -u ADMIN -p ADMIN

and boot the server. This is more or less the console output from the server:

For installation purpose, ipmiconsole is good if the installation media is able to redirect its output to the serial port. If not, the iKVM konsole (http access) is better, since it does not need any COM port at all.
This ipmitool command has the same function as ipmiconsole:

ipmitool -I lanplus -H 192.168.0.100 -U ADMIN -P ADMIN sol activate

See manual page of ipmitool and ipmiconsole for more complete reference.

One good thing from this server that IPMI may use a network wire together with the first interface (eth0), so I can use one less cable. Indeed, a separate RJ45 connector is also provided. In the client side I used multiple address in one of interfaces. Embarrassingly, I found it out recently that this is so easy to configure in Linux.

Say, in the admin computer (actually the main server) you have eth0 as your main interface, and is connected to the real ip address in the network. Since the network cable in the server (node) is used together by IPMI and the main interface, both are plugged into the same hub connection. Normally we need two interfaces plugged to the same hub at admin side.  Uneconomical indeed.
The better way is using multiple address for a single interface. After bringing eth0 up, bring another interface, eth0:0, as example:

ifconfig eth0:0 192.168.0.1 netmask 255.255.0.0

In debian system, one can put in /etc/network/interfaces:

auto eth0:0

iface eth0:0 inet static
address 192.168.0.1
netmask 255.255.0.0

Now the first interface can be used both to access the internet with its original address (eth0) and also for connecting IPMI devices in 192.168.0.0 network.
For mac user, multiple address in a single interface can be easily configured in "Network Preferences".

Imperfect notebook

In fact this notebook is not bad at all. I bought it couple of weeks ago for my sons birthday present: a tiny acer notebook with Intel Atom in it. From the size, it fits well to children's fingers. It is imperfect in the sense that, I made a wrong choice. I took SSD 8Gb version with only 512M memory. Well, I thought it was OK for children. From the hardware point of view I have no objections. I thought I could just put more memory later, and a harddisk as well.

I'm not going to tell the full story about my impression, its boring anyway. Someone can just put "acer one" in google search-bar, then he will be dropped to a bunch of articles about that. I think this notebook is getting more popular now. The interesting thing here is how I struggled with the operating system, since it came with a very ugly Linux settings. I would say terrible.

Firstly, I decided _not_ to change the OS (Linpus), since it might be using a special drivers, or whatever. Then I tried to restore it to the usable Linux form. I found: strange xorg configuration, no runlevel, stupid functionality to restore the users own settings, silly graphical user interface. It is also hard-wired to xfce window manager. If you want to use another lighter window manager, forget it! I made a brute change to its configurations. I deleted files which Acer provided (hacked?), for instance xfce-desktop2.

I'm not going to tell this "restoring" work either, cause it turns out that this wont work. Well it does work, but very ugly since you will not find Linux system like you expected. I was starting to think, that those guys who put this OS, wanted to make bad impression of Linux, or to show that it is not, at all, the better OS than Windows. The perfect example is when you delete some package using yum (Linpus comes from fedora), may be you will stuck to the black screen without being able to login, since xfce is responsible for everything. This system calls the init-scripts directly from xfce's autostart function.

The notebook was running for a couple of weeks with the "rehack" OS. I put the additional memory in it as well. Since I got many complains from my son, I started to realize that what I did was stupid. You can not expect good functionality from an ugly base. So I decided to switch to a "real" operating system. I Installed Debian in it following this wiki. The installing took hours, since the debian install cd image I used has a bug. Don't check the laptop option in the package selection menu! Everything worked perfectly afterward and it seems that all the hardware were detected well. Yes I said everything.

One thing to consider is to avoid using swap space. May be suspend of hibernate will not function, but I don't care. The last thing and maybe the most important thing I did is moving all the actively used directories to the ram space using ramdisk (tmpfs). This is very easy. What we have to do is the followings:

1) use this as /etc/fstab

proc /proc proc defaults 0 0 /dev/sda1 / ext2 defaults,noatime,nodiratime,errors=remount-ro 0 1 none /var/log tmpfs defaults,size=16M 0 0 none /tmp tmpfs defaults,size=128M 0 0 none /var/tmp tmpfs defaults,size=32M 0 0 /dev/mmcblk0p1 /media/home ext2 defaults,noatime,nodiratime 0 0

2) and make a small script to provide some missing directories as /etc/init.d/prepare-var.sh. Call this script on boot time. What I did is creating a link under rc2.d directory.

#!/bin/bash #### # Acer Aspire One: # some directory @ var are moved to tmpfs # prepare directory below /var # do_start() { /bin/mkdir -p /var/log/apt /bin/mkdir -p /var/log/gdm /bin/touch /var/log/dmesg } case "$1" in start|"") do_start ;; restart|reload|force-reload) echo "Error: argument '$1' not supported" >&2 exit 3 ;; stop) # No-op ;; *) echo "Usage: prepare-var.sh [start|stop]" >&2 exit 3 ;; esac

This makes the tiny notebook much faster and also saves the life time of the SSD. The best imperfect thing is, now it is a normal usable computer. This trick is also useful in other system that uses Compact Flash as its boot device. I did also the similar thing to my shuttle pc. I put 4GB CF-disk in it and use it as the boot device. It starts speeding!!

GlusterFS: The end of (our) NFS?

In fact this is not the first GlusterFS installation in our cluster. /G has installed it a couple times ago, but It was not used. Now I have to reactivate it and rearrange the settings in the better way. Now I am forced to do it, just because the P5 system is not responding up to now.

The situation is, we have nodes with rather big storages each. These storages were not used optimally before. GlusterFS seems to be ideal, since it can gather all the free spaces for all nodes. I configured the harddisks using LVM last time, so the file system size spans in two storages. The total capacity is now 265 per node. Actually we can have around 3.5Gigs, but I decided to make it redundant.

Preparing GlusterFS

GlusterFS can be downloaded from this link. We used version 1.3.7. This file system uses FUSE to be mounted on client. So FUSE must also be installed prior to GlusterFS. The patched version can be found here.

Do a normal installation, configure, make and make install to those downloaded source codes. Please refer to their installation instruction. Try to solve all dependencies. If you still have trouble and have no idea to install them, maybe you need to find another job :D.

Configuring GlusterFS

I have, in front my eyes, 14 nodes to configure. The first node is used as the head node. So it is excluded from the main storage brick. Instead its free space is combined with node-12 for a scratch space. The nodes are enumerated from 00 to 12, so when redundant storage is preferred, only node-00 to node-11 can be used.

All nodes, except node-12, share their space in the same way. node-00 and node01 are selected as the head storage. A head node has a special shared space, called the name-space. So basically these two nodes are the same with others. all nodes mount the shared partition below /gls directory. The fstab entry is as follow:

/dev/scratch-vol/scr /gls ext3 defaults 0 0

The volume configurations are stored inside the directory /etc/glusterfs. Followings are the content of these files:

1) server with name-space (node-00 & node-01), gfs-server-ns.vol

volume wapblock type storage/posix option directory /gls/block end-volume volume wapblock-ns type storage/posix option directory /gls/block-ns end-volume volume server type protocol/server option transport-type tcp/server subvolumes wapblock wapblock-ns option auth.ip.wapblock.allow * option auth.ip.wapblock-ns.allow * end-volume

2) server without name-space (node-02 to node-11), gfs-server.vol

volume wapblock type storage/posix option directory /gls/block end-volume volume server type protocol/server option transport-type tcp/server subvolumes wapblock option auth.ip.wapblock.allow * end-volume

3) scratch server (node-head & node-12), gfs-server-scr.vol

volume wapblock type storage/posix option directory /gls/block end-volume volume wapblock-ns type storage/posix option directory /gls/block-ns end-volume volume server type protocol/server option transport-type tcp/server subvolumes wapblock wapblock-ns option auth.ip.wapblock.allow 192.168.100.* option auth.ip.wapblock-ns.allow 192.168.100.* end-volume

I set the allowed address here explicitly, because head node is world accessible. I would think to do the same for others, but maybe later. Those are the required configuration files for brick/block servers. The followings are the client configuration files.

1) main working space, gfs-client.vol

volume gls-node00 type protocol/client option transport-type tcp/client option remote-host node-00 option remote-subvolume wapblock end-volume #..... the same code blocks take place here, except the volume name and remote-host name #..... for node-01 to node-11 # now namespaces. Two of them are needed. volume gls-node00-ns type protocol/client option transport-type tcp/client option remote-host node-00 option remote-subvolume wapblock-ns end-volume volume gls-node01-ns type protocol/client option transport-type tcp/client option remote-host node-01 option remote-subvolume wapblock-ns end-volume # Automatic file replication volume mirror01 type cluster/afr option replicate *:2 subvolumes gls-node00 gls-node01 end-volume # the same code blocks continue. # Pairing: gls-node02 - gls-node03 .... until gls-node10 - gls-node11 # namespace mirror volume mirror-ns type cluster/afr subvolumes gls-node00-ns gls-node01-ns end-volume # combine all volume unify type cluster/unify option namespace mirror-ns option scheduler rr option rr.limits.min-free-disk 5% option rr.refresh-interval 10 subvolumes mirror01 mirror02 mirror03 mirror04 mirror05 mirror06 end-volume # performance volume readahead type performance/read-ahead option page-size 256 # 256KB is the default option option page-count 4 # 2 is default option option force-atime-update off # default is off subvolumes unify end-volume volume writebehind type performance/write-behind option aggregate-size 1MB # default is 0bytes option flush-behind on # default is 'off' subvolumes unify end-volume

2) scratch space, scratch.vol

Scratch space uses no redundant blocks.

volume scr-node1 type protocol/client option transport-type tcp/client option remote-host node-head option remote-subvolume wapblock end-volume volume scr-node2 type protocol/client option transport-type tcp/client option remote-host node-12 option remote-subvolume wapblock end-volume volume scr-ns type protocol/client option transport-type tcp/client option remote-host node-head option remote-subvolume wapblock-ns end-volume volume unify type cluster/unify option scheduler rr option namespace scr-ns subvolumes scr-node1 scr-node2 end-volume

After all the configuration files are prepared, the files are copied to the nodes. I used rgang to do the copy. I prepared first files below /etc/clusternodes for rgang, according to the definitions mentioned above: glshead, glsserv, glsscr, and world.

$ rgang -c glshead gfs-server-ns.vol /etc/glusterfs/gfs-server.vol $ rgang -c glsserv gfs-server.vol /etc/glusterfs/gfs-server.vol $ rgang -c glsscr gfs-server-scr.vol /etc/glusterfs/gfs-server.vol $ rgang -c world gfs-client.vol /etc/glusterfs $ rgang -c world scratch.vol /etc/glusterfs

Now we need the script to activate them. I wrote the following script, and put it below /etc/init.d. All the nodes have the same script, glus-load.

#!/bin/bash # Loading GlusterFS # Rosandi 2008 #loading glusterfs daemon @ server load-srv() { glusterfsd -f /etc/glusterfs/gfs-server.vol || echo failed >> /var/log/glus-srv.log } #mounting client load-cl() { echo "/usrshome" glusterfs -f /etc/glusterfs/gfs-client.vol /usershome echo "/scr" glusterfs -f /etc/glusterfs/scratch.vol /scr } unload-cl() { umount /usershome umount /scr } case "$1" in start) echo "Loading GlusterFS daemon: " load-srv sleep 5 echo "mounting GlusterFS: " load-cl ;; stop) unload-cl killall glusterfsd ;; restart) $0 stop $0 start ;; *) echo "Usage: glus-load {start|stop|restart}" exit 1 esac exit 0

Now since everything is ready, we only need rgang again to activate them.

$ rgang world /etc/init.d/glus-load start

I would suggest to activate the new filesystem only from the head node, after checking the availability of all required node. This can take place in rc.local. Of course in Debian system.

I read also that someone has provided a Debian repository for GlusterFS. So things will became less complicated.

So this looks like the end of our _imperfect_ NFS server. Unless if I can fix our AIX station. To be honest, I don't like AIX. Its only expensive. I am thinking of installing Linux in it, maybe, share the filesystem also with GlusterFS?

Now look what we have after mounting the new filesystem,

Filesystem Size Used Avail Use% Mounted on /dev/sda2 133G 6.8G 119G 6% / tmpfs 3.9G 0 3.9G 0% /lib/init/rw udev 10M 44K 10M 1% /dev tmpfs 3.9G 0 3.9G 0% /dev/shm /dev/sdb1 147G 188M 140G 1% /gls glusterfs 1.6T 1.1G 1.5T 1% /usershome glusterfs 411G 376M 390G 1% /scr

Changing layout

I changed the layout of this blog. Now everything looks like a mess! Man, I don't care.

Moving the cluster

Last Thursday I moved the cluster to computer center room. Of course with the help of my two nice friends, Gerolf and Christian. Reasons: our previous server room is too small and the cluster became too hot. That server room turned out as a sauna room :D. It was reasonable why the cluster became hot. The usage was increasing very highly. Almost all the time 100% of processors are used.

Look at the mess we have created!

The server room is pretty large, since it stores almost all the servers of the university. I put the cluster in the corner, beside our new-larger-cluster.

Everybody on the floor!!

This cluster is 10 times more difficult to assembly. The screws are not matched each other, the door is heavy, the rack is terribly terrible. But I made it! It is imperfect of course, but it is prefect in giving me some exercises. Now here it is, after assembling.

Putting them together

Of course beside our black cluster! Do you see those penguins?

Standing side by side with our 'nigol' cluster

New node plug and pray

Since our new cluster has just come. Partly. I want to make them automagically booting without further per-node configuration. We (me and Gerolf) have made the scenario like this: All nodes boot via pxe. For this we need to install a proper dchp, nfs, tftp, and linux-pxe. apt-get will help us. Notes: - do not use dhcp3, it fails to give pxe files. Dunno why. - use tftp-hpa The problem is, we don't know the addresses of interfaces of the nodes. The installation would be rather annoying, if we had to plug one-by-one a monitor to each node, just to see they hardware addresses. No. I don't want to do like that. This is the trick: Node side 1) steal initrd from Debian, and change its init script, just to detect network interfaces (eth*). 2) use tftp to stor the address at servers directory. Put it in the provided slots at the server. 3) wait until the server provide everything needed to boot, by waiting for an indicator. I chose as indicator a string 'ready' in the slot file. 4) reboot, and fetch the right kernel, initrd, and nfsroot. Server side 1) configure dhcpd to serve request from unknown interfaces. To do this I give a range declaration inside the subnet block for dynamic ips fron 10.255.0.1 to 10.255.0.255. 2) prepare request directory, containing slot files called req-(number). (number) is only sequence number to identify the slot. 3) check the slot files. If they contains hardware address then process it and put the corresponding entries in /etc/dhcpd.conf. 4) write indicator (ready string) inside each configured nodes. 5) call refresh script, written by Gerolf, to refresh pxe boot files, nfs exports, dhcp, and everything which is needed. Consult documentation about setting up pxe-boot somewhere in the net. If I put that details here, it would only give noises. If there is nothing wrong, the node will boot and getting online soon. This is the init script inside initrd,

----------------------------------------------------------- #!/bin/sh echo "Loading, please wait..." [ -d /dev ] || mkdir -m 0755 /dev [ -d /root ] || mkdir --mode=0700 /root [ -d /sys ] || mkdir /sys [ -d /proc ] || mkdir /proc [ -d /tmp ] || mkdir /tmp mkdir -p /var/lock mount -t sysfs none /sys mount -t proc none /proc tmpfs_size="10M" if [ -e /etc/udev/udev.conf ]; then . /etc/udev/udev.conf fi mount -t tmpfs -o size=$tmpfs_size,mode=0755 udev /dev [ -e /dev/console ] || mknod /dev/console c 5 1 [ -e /dev/null ] || mknod /dev/null c 1 3 > /dev/.initramfs-tools mkdir /dev/.initramfs export DPKG_ARCH= . /conf/arch.conf export ROOT= . /conf/initramfs.conf for i in conf/conf.d/*; do [ -f ${i} ] && . ${i} done . /scripts/functions export break= export init=/sbin/init export quiet=n export readonly=y export rootmnt=/root export debug= export cryptopts=${CRYPTOPTS} export ROOTDELAY= export panic= SERVER= DEV=eth0 for x in $(cat /proc/cmdline); do case $x in server=*) SERVER=${x#server=} ;; dev=*) DEV=${x#dev=} ;; break=*) break=${x#break=} ;; break) break=premount ;; esac done depmod -a maybe_break top run_scripts /scripts/init-top load_modules run_scripts /scripts/init-premount maybe_break request [ -z "$SERVER" ] && panic "Server name must be defined in \ kernel option: server=" echo "#### getting interface address of $DEV ####" ipconfig -c dhcp -d $DEV ping -c 2 $SERVER || panic "Cannot connect server $SERVER" ##### # request file in format req-(number) must exist and writable (mode 666) # in request directory @ server # req=1 reqfile="" while [ ${req} -le 100 ]; do tftp -g -l req -r request/req-${req} $SERVER if [ -s req ]; then req=$(( ${req} + 1 )) else reqfile=req-${req} break fi done [ -z "${reqfile}" ] && panic "No free request slot.. this is impossible" # slot available for i in /sys/class/net/eth*; do cat $i/address >> req done tftp -p -l req -r request/${reqfile} $SERVER ready= while [ -z "$ready" ]; do echo "<<<< waiting for valid boot files >>>>" sleep 10 tftp -g -l req -r request/${reqfile} $SERVER grep ready req && ready=yes done echo "<<<< Booting files are ready.... REBOOT... >>>>" reboot --------------------------------

And to handle the request, I put a _secret_ tag:

#--> insertion point bla bla bla

somewhere in a right place inside dhcpd.conf. All the host entry will be written in _one line_. I put also a lease range 10.255.0.1 to 10.255.0.255 in the subnet block. Here is the handling script,

---------------------------------- DIR=/srv/tftp die() { echo $@ exit } cd $DIR/request echo "searching request" rm -f /tmp/newnode handled= for i in req-*; do [[ -s $i ]] || continue grep 'ready' $i && continue mainif=`awk 'NR==1{print}' $i` auxif=$(awk 'NR==2{print}' $i ) pxef=$(echo $mainif|awk '{gsub(/:/,"-");print}') [[ -f ../pxelinux/$pxef ]] && continue echo "Handling request for $mainif" handled="$handled $i" # find node number awk 'BEGIN{n=0} /^[[:space:]]*#/{next} /^[[:space:]]*host node/{gsub(/[[:alpha:]_-]/,"",$2);if(n<$2)n=$2} END{print "host node"n+1" \ {option host-name \"node"n+1"\";\ hardware ethernet '$mainif';fixed-address 10.10.1."n+1";}" if("'$auxif'") print "host node"n+1"a \ {hardware ethernet '$auxif';fixed-address 10.10.2."n+1";}" }' /etc/dhcpd.conf >> /tmp/newnode done [[ -z $handled ]] && die "no new node request" awk 'BEGIN{fl=0} FNR==1{fl++} fl==1{x[FNR]=$0;next} /^#--> insertion/{for(i in x)print x[i]} {print}' /tmp/newnode /etc/dhcpd.conf > /tmp/dhcpd-tmp mv /tmp/dhcpd-tmp /etc/dhcpd.conf rm -f /tmp/newnode /root/admin-scripts/refresh # to reboot the node: put 'ready' in req-* file in request directory for i in $(echo $handled);do echo ready > $i; done ----------------------------------------------------------------------

Imperfect NAS

After hacking our REAL nas system last time, we now built one by our own. Sixteen hard disks in one case is big enough. We will have more or less 6Tb inside. Of course mr.Guru will fill it up again immediately. What we installed inside is simply LVM. Since we use hardware raid 16 disks sata controller, no need to setup MD device. It is great and easy. Ups... first test there was an accident. The panel cable was burning!! In that cable there was 3 vcc, and 3 ground lines. These lines are too small for power line. We should have been plugging additional power line. But anyway, this is one thing that makes our new nas imperfect. See how this device fits to my baby cluster!! Instead of stacking around with LVM documentation, this is the sequence of command we need to run in order to set it up:

#!/bin/bash INCHD="/dev/sd?? and so on" VOLNAME=scratch-vol LNAME=scr MNTDIR=/scr DEV=/dev/$VOLNAME/$LNAME apt-get install lvm-common lvm2 pvcreate $INCHD vgcreate $VOLNAME $INCHD vgchange -a y $VOLNAME TSIZ=`vgdisplay $VOLNAME | grep "Total PE" | awk '{print $3}'` lvcreate -l $TSIZ $VOLNAME -n $LNAME mkfs.ext3 $DEV mkdir -p $MNTDIR echo "$DEV $MNTDIR ext3 defaults 0 1" >> /etc/fstab mount $MNTDIR df -h $MNTDIR

Awkfull slider

My awkfull slide creator is now more or less usable. I am thinking of publicly sharing this tool. But as I said, this tool is awkfull. I wrote almost everything using awk. This language is pretty simple, and rarely used as a serious programming language. One can see a lot of awk one-liners inside a shell script, but complete program... Slider uses awk as a tool to translate plain text to an swfc script. This plain text is straight forward, containing tags and contents of slides. Here is the example:

$slide
This is my first slide
~First item
~~and first sub item

Pretty simple, right? At least it is very readable to me. Wait how about figures? Not only figures, you can also put directly movies inside it. Of course, swf format is more preferable. Look at this:

$slide
How to put figure...
$figure myfigure.jpg x=100 y=100 scale=50% alpha=50%
$figure movie.swf y=100 y=250 scale=50%

thats it. I will comeback to this post to tell the repository where I put slider program. I need to make also some examples for user. This is really an Awkfull program.

Imperfect bootable linux in usb-flash disk

I believe a bootable linux system should not be too difficult to build. I don't need a perfect system anyway. The point is making it able to boot, and has stand alone tools just to make some recoveries when something goes wrong with our computer system. In fact we have almost everything inside initrd file. A busybox package is almost perfect. Simply copying it along with its kernel version is enough to give you a 'panic' shell interface. On Debian system, one may try also passing break=mount, to break the booting process right before initrd mounting filesystems. Now, to make a bootable flashdisk we can simply create a partition inside it, format it as ext2 filesystem, and then install grub on its MBR. We do this from our running system without downloading anything from the net. I use this variable in this process:

rootflash=/mnt/buildboot drive=/dev/sdb1

1) partinion the drive (/dev/sdb1, 83, bootable)

fdisk /dev/sdb

2) format as ext2:

mkfs.ext2 $drive e2label $drive ROSBOOT

We need to give a label to our disk, because we are working with a removable disk. It is unsure in which device node the usb storage will be referred. Using label we can easily point to as /dev/disk/by-label/ROSBOOT. We need a small change on init script in order to make it possible.
3) mount usb drive and copy kernel

mount $drive $rootflash mkdir $rootflash/boot cp /boot/vmlinuz-2.6.18-4-686 $rootflash/boot/vmlinuz cp /boot/initrd.img-2.6.18-4-686 $rootflash/boot/initrd

4) install grub on the mbr of our usb storage and create grub menu.

grub-install --root-directory $rootflash /dev/sdb cat > $rootflash/boot/grub/menu.lst << endt title Debian Tumb root (hd0,0) kernel /boot/vmlinuz root=/dev/ram0 vga=794 initrd /initrd boot

Until this point the usb drive can boot and gives busybox. What else do you want? It is now a totally imperfect booting system. But at least busybox gives us a lot of useful basic command. We can now mount something from nfs to boot if we want. The better idea is maybe creating Debian bootstrap:

debootstrap --arch i386 etch $rootflash

and change one corresponding line in menu.lst to

kernel /boot/vmlinuz root=LABEL=ROSBOOT ROOTDELAY=5 vga=791

Than we will use at least 150Mb for nothing. I haven't tried that, but anyway, I don't want to. My first idea is to have an imperfect rescue system. For which I only need to add some administration applications and change init script to make it imperfect.
5) Modify initrd
We can extract our working initrd and put it somewhere. Here are the commands:

cd /tmp; mkdir work-initrd; cd work-initrd gzip -dc /boot/initrd.img-2.6.18-4-686 | cpio -id

Now we put some missing good applications, modules and libraries.

install /sbin/fdisk sbin/ install /lib/libext2fs.so.2* lib/ install /lib/modules/2.6.18-4-686/kernel/drivers/usb/input/usbkbd.ko lib/modules/2.6.18-4-686/kernel/drivers/usb/input/ install /lib/modules/2.6.18-4-686/kernel/drivers/usb/input/usbmouse.ko lib/modules/2.6.18-4-686/kernel/drivers/usb/input/

Now it is time for us to make modifications on init file. It is a normal shell script. My hand feels at home.

joe init

and do our blablablas there! I put my imperfect init script as a tail to this blog. We need to modify the modules file as well:

cat > conf/modules << endt unix usbcore usbhid usb-storage ext2 i8042 atkbd usbhid usbkbd usbmouse

Finished with modifications? now repack initrd!

find ./ | cpio -H newc -o > /tmp/initrd.cpio gzip /tmp/initrd.cpio mv /tmp/initrd.cpio.gz $rootflash/boot/initrd

Now for me everything is OK. It boots well in my notebook, but an old computer in the corner can not boot from it, even though it has capability for it. Maybe the bios is imperfect! Don't care. The size of my booting flashdisk system is now 7.6Mb. Maybe I can have it smaller by recompiling the kernel and throw out some unused drivers. Next idea is to put directories inside the flash disk and make a link to it inside the initram like usr directory, etc, so we don't have to change the initrd afterward.
6) Tail: imperfect init script
I modified the init script from initrd of Debian etch. With this modification I intend to call /disk/etc/rc.d file in the real usb drive storage. We do not need to touch this init rd anymore, and control everything there.

#!/bin/sh echo "Loading, please wait..." [ -d /dev ] || mkdir -m 0755 /dev [ -d /root ] || mkdir --mode=0700 /root [ -d /sys ] || mkdir /sys [ -d /proc ] || mkdir /proc [ -d /tmp ] || mkdir /tmp mkdir -p /var/lock mount -t sysfs none /sys mount -t proc none /proc ## increase tmpfs size to 20M ### tmpfs_size="20M" ################################# if [ -e /etc/udev/udev.conf ]; then . /etc/udev/udev.conf fi mount -t tmpfs -o size=$tmpfs_size,mode=0755 udev /dev [ -e /dev/console ] || mknod /dev/console c 5 1 [ -e /dev/null ] || mknod /dev/null c 1 3 > /dev/.initramfs-tools mkdir /dev/.initramfs export DPKG_ARCH= . /conf/arch.conf export ROOT= . /conf/initramfs.conf for i in conf/conf.d/*; do [ -f ${i} ] && . ${i} done . /scripts/functions export break= export init=/sbin/init export quiet=n export readonly=y export rootmnt=/root export debug= export cryptopts=${CRYPTOPTS} export panic= ## give default ROOTDELAY to 5 seconds # # this gives time to udev to make usb disk online # export ROOTDELAY=5 #################################################### # no need to parse complete command line options for x in $(cat /proc/cmdline); do case $x in break=*) break=${x#break=} ;; break) break=premount ;; esac done if [ -z "${NORESUME}" ]; then export resume=${RESUME} fi depmod -a maybe_break top run_scripts /scripts/init-top maybe_break modules log_begin_msg "Loading essential drivers..." load_modules log_end_msg maybe_break premount [ "$quiet" != "y" ] && log_begin_msg "Running /scripts/init-premount" run_scripts /scripts/init-premount [ "$quiet" != "y" ] && log_end_msg ####### cut debian's init script here ######### echo "mounting flash filesystem" echo "if this fails, we run without it" mkdir /disk mount -t ext2 /dev/disk/by-label/ROSBOOT /disk RCRET=255 # if exist /disk/etc/rc.d then give control to it if [ -x /disk/usr/rc.d ]; then /disk/usr/rc.d RCRET=$? fi # if rc.d returns 255 run shell [ $RCRET -eq 255 ] && PS1='ROSBOOT$ ' /bin/sh -i /dev/console 2>&1 echo "Unmounting file systems...." umount /disk sleep 5 echo "Quits rosboot.... REBOOTING...." sleep 1 reboot

Remember! imperfectness is perfectness it self.

The Windows Effect

As I said.... I am totally imperfect! Or.. should I say stupidly imperfect. Now I've got trouble with a very nice copy-center workers. And I would have to pay a lot of money!!! No... wait thats not my failure! I lost my "card" with PIN number sticked in it. I confess, that was my fault... totally my fault. But I asked them to block it, immediately after I realized it. I believe It was before 10.9! Because I had holiday, on my birthday. OK, she told me to call back before that day, which I did not do. But that was just ridiculous! I came to their office just to ask them to block it, why should I call back! And they did not block it just because I did not call back! This is stupid! ----------------------- I believe that this is exactly The Windows Vista Effect! When I wanted to block my card because I'd lost it, I would be asked, with a very nice window: DO YOU REALLY WANT TO BLOCK IT? I clicked YES...., later came another window.... ARE YOU SURE YOU WANT TO BLOCK IT?????? I answered.. Damn it... YES! Another nice windows showed up..... PLEASE THINK IT OVER AGAIN AND CALL US BACK BEFORE THIS DATE IF YOU THINK YOU ARE SURE THAT YOU WANT TO BLOCK IT! This is idiot! Now after everything was too late, they were laughing and asked for more money! This is exactly how they got rich so fast! I just can not accept this... totally ridiculous...

Imperfect OS on your computer

Well, All I want is to have a very simple working system in front of me. I was satisfied with SuSE a couple of years ago, but then it turns out that it is not any longer convenient to use since it was "captured" by Novel. Open SuSE is not bad also, but it broke my hart already. To be honest I don't like to get contact with a free commercial OSes. They are ridiculous. Then Slackware was the next choice, and I was very happy, to make it running on my Samsung X-20. It also was my very favorite distribution long time ago, and it was just working. I do not remember why should I change my mind to try other distribution after that. May be I only felt to rigid. I had everything which I did not use or only seldom. Slackware had (or may be officially still has) no support to 64bit procs. I had to think about good distro to be installed in my server. After Slamd64, which was not working properly, I tried Debian on it. With it, I almost have no problem until now. Apt system is very powerful. It was just running fine. Maybe this was the reason why I wanted to install Debian also in my notebook. It was running very good, except the network setting which I didn't like. But of course it is not the fault of Debian. For this I would rather writing my own script. My X-20 was running under it, using only about 3Gigs of space, until now. It is perfect! Time is taken away, and I feel my X-20 is getting older. Besides I had a better reason: My son occupied it. I bought another notebook for me, an Amilo Si-1520. I directly deleted Vista on it, and installed Debian (Etch). It was a very clean installation. The problem was only wlan ipw3945, but everything is available on the net. It was not a big deal to compile the module and activate the network. Network devices was disabled, except lo, and wlan was activated using my self-written script. Another problem was card reader, but I don't care, and don't use it until now. After a couple of months The system was swollen. I did too many apt-get install on it. I installed almost every garbages! I was starting to hate it, especially Debian's bad habit to delete everything when one deletes an important "unimportant-program", such as exim, a mail server. Why should a laptop has a mail server in it? It is just ridiculous! Out of curiosity, I tried Arch-Linux a couple of mounts after it. It worked also fine at the beginning. But then I found out that pacman is not as good as apt. It broke other program when I installed a software from repository. There was also an annoying problem: silly postscript problem. It is more than ridiculous! It is crazy! Arch-Linux is good when you want to learn how to make an OS work, but not to use it for your real work. Well, I gave up, and decided to go back to Debian. I tried Lenny, but it has problem with sound card. Yes, it is again not the fault of Debian, but alsa. Finally the conclusion is go back to Etch. It is the best for my notebook now, at least for the time being. But I am happy enough with my 1.9Gigs OS, 35 seconds booting time, installed in my notebook. No KDE, no complete gnome. Icewm is my best choice, light and simple. No perfect OS available, but it just perfect because I can try something on it!