mhoenicka

Import spreadsheet data into DocBook

Sat, 19 Mar 2011 17:19:21 +0100

DocBook is a versatile XML vocabulary for technical documents, including grant proposals and theses in life sciences. Typing tabular data into an XML editor like Emacs + nxml is pretty cumbersome as there is a bad signal-to-noise ratio in terms of markup vs. data. In contrast, typing the same data into a spreadsheet like OpenOffice Calc is painless. Also, I'm used to keep my data in spreadsheets anyway, so I had to find a way to import these into DocBook files. Apparently there is no such tool on the market, so I dusted off my Perl skills and wrote a little script for this purpose.

Being a lazy person, I didn't build this script on Text::CSV but I rather used a lightweight approach to parse the input data. Best results are obtained if you save your spreadsheet data using tabs as column separators. As long as you don't insist on using tabs in your data as well, things will turn out fine.

See here for some more information including the download link. Maybe you find this tool as useful as I do while typing my "habil" thesis.

How to configure FreeBSD on an Acer TravelMate 8371

Sun, 21 Nov 2010 21:05:35 +0100

In a previous post I've explained how to install FreeBSD 8.1-RELEASE on an Acer TravelMate 8371 and still be able to boot it. Now I'd like to share the details on the configuration which makes the OS usable in the first place. This is going to be pretty technical and a bit lengthy, and I'll make sure to include and explain all config files that I've created or edited. The reason why I'm elaborating on this stuff here is that The FreeBSD Laptop Compatibility List wouldn't let me create an entry.I've got the following to work:

- graphics
- touchpad
- USB
- SD card reader
- ethernet
- wifi
- sound
- play DVDs and audio CDs from an external optical drive
- check battery status

Things that don't work as expected include:
- set the screen brightness via keys
- set the volume via keys
- use any of the acpi sleep modes

Things that I'd expect to work (according to dmesg) but can't test due to the lack of equipment or applications accessing the stuff:
- bluetooth
- webcam

At first, let's review the components of my box. The 8371 is sold in a couple of configurations, so please don't just blindly follow the instructions before making sure your box has the same bits and pieces inside as mine does. The full name of the laptop is Acer TravelMate TM8371-944G50n. According to a sticker on the palmrest (which I removed right after typing this), there is an Intel Core 2 Duo processor SU9400, an Intel GMA4500MHD graphics chip, 4GB of memory, a 500GB HDD, an Acer Nplify 802.11a/b/g/n WLAN chip, and Bluetooth 2.1+EDR, whatever that is. We'll see shortly that the WLAN is nothing to be afraid of, as pciconf knows it better than the sticker.

First one hint about the installation procedure which I forgot to mention last time. The current version (8.1-RELEASE) creates too small a / partition if you use the auto-layout feature of the partition editor. Make sure to have at least 1GB here, as you'll run out disk space with the default (500MB) as soon as you build a custom kernel - and you will have to. I've filed a bug report, but apparently this problem has already been fixed for the next release.

The first file to edit is /boot/loader.conf. This is the place to stick in things that should be loaded or set right at boot time.

--8<---
verbose_loading="YES" # print verbose loader output during boot
wlan_scan_ap_load="YES" # load wlan stuff
wlan_scan_sta_load="YES" # load wlan stuff
coretemp_load="YES" # intel temp sensor support
hw.psm.synaptics_support="1" # improved touchpad support
--8<---

Next is /etc/rc.conf, the system configuration file. This file contains two configurations for Ethernet (which I use most of the time) and WLAN (which I enable whenever I need it).

moused must be told which mouse to use. I set it to an external USB mouse as I was not able to use the touchpad under X if moused grabs it as a sysmouse here. Downside is that I don't have a console mouse if there is no USB mouse attached, but this not a high price to pay.

powerd is told to use hiadaptive mode when plugged in and adaptive mode when running on battery. This is to extend battery life as much as possible without sacrificing much speed.

devfs support needs to be added in order to support using an external USB optical drive. The ruleset mentioned here is defined in /etc/devfs.rules (see below)

linux support must be enabled for flashplugin - who wouldn't like to watch paint dry once in a while :-)

--8<---
# -- sysinstall generated deltas -- # Sat Nov 13 20:42:02 2010
# Created: Sat Nov 13 20:42:02 2010
# Enable network daemons for user convenience.
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/rc.conf.
hostname="wombat.mininet"
ifconfig_re0="DHCP"
#wlans_iwn0="wlan0"
#ifconfig_wlan0="DHCP"
moused_enable="YES"
moused_port="/dev/ums0"
nfs_client_enable="YES"
nfs_server_enable="YES"
rpcbind_enable="YES"
sshd_enable="YES"
powerd_enable="YES" # Run powerd to lower our power usage.
powerd_flags="-a hiadaptive -b adaptive -i 90" # Flags to powerd (if enabled).
devfs_system_ruleset="localrules"
linux_enable="YES"
--8<---

/etc/devfs.rules is modified to allow access to an external USB optical drive for everyone. The name of the ruleset must match the one used in rc.conf.

--8<---
# devfs(8) rules for devices not available at boot time
[localrules=10]
add path 'cd*' mode 0666
add path 'xpt*' mode 0666
add path 'pass*' mode 0666
--8<---

/etc/fstab needs an additional entry for the flashplugin to work.
--8<---
[...]
linproc /usr/compat/linux/proc linprocfs rw 0 0
--8<---

One of my favourite pastimes is to build a custom kernel. Here it is actually necessary in order to support some of the hardware. As always, you should familiarize yourself with building custom kernels by reading the appropriate chapter in the FreeBSD handbook. I've removed a few things which aren't required obviously, although I wasn't really zealous. I'm sure you could dump even more. However, the lazy among you can get away with copying the GENERIC kernel configuration and adding the following lines:

--8<---
# Intel Wireless support
device iwn
device iwnfw

# SMBus support
device smbus
device smb
device ichsmb

# sound
device sound
device snd_hda
--8<---

The first entry is an interesting one. As mentioned previously, the palmrest sticker claimed there is an Acer Nplify WLAN on board. However, pciconf -lv reports this:

iwn0@pci0:1:0:0: class=0x028000 card=0x13018086 chip=0x42328086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = 'Carte Intel WiFi Link 5100 AGN (Intel WiFi Link 5100)'
class = network
re0@pci0:2:0:0: class=0x020000 card=0x02831025 chip=0x816810ec rev=0x02 hdr=0x00
vendor = 'Realtek Semiconductor'
device = 'Gigabit Ethernet NIC(NDIS 6.0) (RTL8168/8111/8111c)'
class = network
subclass = ethernet

So there's a plain ol' Intel chip. This is good for us as the Intel chip is supported, the Acer one apparently not (unless they're the same under the hood).

Finally, graphics need to be configured as well. Although xorg does a great job at just working out of the box these days, I needed to tweak a few things. I started by creating an xorg.conf from the xorg defaults (run this as root):

Xorg -configure

This will generate /root/xorg.conf which you should copy to /etc/X11/.

I'll list only those sections that were added or changed:

--8<---
Section "ServerFlags"
Option "AllowEmptyInput" "off"
EndSection

Section "ServerLayout"
Identifier "X.org Configured"
Screen 0 "Screen0" 0 0
InputDevice "Mouse0" "CorePointer"
InputDevice "Synaptics_Touchpad" "AlwaysCore"
InputDevice "Keyboard0" "CoreKeyboard"
EndSection

Section "Module"
Load "extmod"
Load "record"
Load "dbe"
Load "glx"
Load "dri"
Load "dri2"
Load "synaptics"
EndSection

Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
Option "XkbRules" "xorg"
Option "XkbModel" "pc105"
Option "XkbLayout" "us,de"
Option "XkbOptions" "grp:lwin_toggle,grp_led:scroll"
Option "XkbVariant" "nodeadkeys"
EndSection

Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/sysmouse"
Option "CorePointer"
Option "ZAxisMapping" "4 5 6 7"
EndSection

Section "InputDevice"
Identifier "Synaptics_Touchpad"
Driver "Synaptics"
Option "Protocol" "psm"
Option "Device" "/dev/psm0"
Option "SendCoreEvents" "true"
Option "ZAxisMapping" "4 5 6 7"
Option "FingerLow" "26"
Option "FingerHigh" "31"
Option "HorizEdgeScroll" "1"
Option "MinSpeed" "0.1"
Option "MaxSpeed" "0.15"
Option "RTCornerButton" "2"
Option "RBCornerButton" "3"
Option "TapButton2" "2"
Option "TapButton3" "3"
EndSection
--8<---

The ServerFlags section must be inserted in order to get mouse and keyboard input to work at all. It is still a mystery to me why xorg defaults to not letting the user interact with it by means of a keyboard or a mouse. Does it expect to be yelled at?

The ServerLayout section receives an additional line that includes the touchpad. Make sure to use the "AlwaysCore" option, not a second "CorePointer" option as with the mouse. Xorg will be upset if you do this.

We'll have to load the synaptics module in order to get improved touchpad support. I tested the builtin touchpad support of the mouse driver as well, but I found it buggy as it inserted random additional mouse events like scroll or left click whenever I used the right touchpad button.

The keyboard is configured for two layouts. I prefer the US layout for programming tasks, but I need to keep the DE layout around for German texts and mails. As you can see, I finally found a way to make use of that apparently superfluous key with that weird Windoze logo on it: now it toggles my keyboard layouts :-)

The final sections set up the touchpad. I've arrived at the settings empirically, using the synclient(1) tool to change them on the fly. From top to bottom, I've made the touchpad slightly less sensitive, switched on horizontal scrolling, and limited the maximum speed to a value that avoids nausea. The last four lines add a middle mouse button at the top right corner and a right mouse button to the bottom right corner. This is convenient e.g. for opening new tabs in Firefox without having to move to the mechanical touchpad buttons.

That's about it. From here, everything else should work just as with your regular desktop in terms of configuration.

How to install FreeBSD on an Acer TravelMate 8371 ... and not brick it

Thu, 18 Nov 2010 00:53:52 +0100

After about 5 years of almost daily use my laptop more or less fell apart. The built-in speakers have been mute for years, the battery plug was worn out, the battery itself barely lasted for 20 min when I unplugged the box, and the backlight no longer allowed to use any setting other than the lowest, making the box a nice companion for dark nights. This finally convinced me that it is about time to get a new one.

After a couple of weekends spent on surveying the market, I purchased an Acer TravelMate 8371 that seemed to suit my needs. Needless to say, it ships with an OS from Redmond that I simply don't care for. When I tried to install a far better replacement (aka FreeBSD), I almost bricked the box. Read on to see how to avoid this.According to the specs, installing FreeBSD on the 8371 should not be too challenging. I expected some difficulties with the external CD drive which I had purchased separately in lieu of an internal optical drive, but most components like graphics and network were supposed to work out of the box. I first used a Knoppix Linux live CD to test the basics. Knoppix would boot just fine and do almost everything right, including graphics, sound, and network. It didn't handle the touchpad well (maybe by design), but I knew that this could easily be fixed by using some xorg tweaks. This is where I decided to give FreeBSD a try.

In went the FreeBSD 8.1-RELEASE netinstall boot CD. As the box has 4GB of memory, I used the amd64 branch to make best use of it. First I had to partition the hard drive (create slices in BSD lingo). The partition editor displayed a plethora of partitions and unassigned areas on the drive, only two of which were NTFS partitions. I'd expect one to be the restore image, the other the real Windows 7 installation. I boldly decided to use the entire disk as a single slice and went ahead to partition that slice (BSD partitions reside in one or more slices, so you can have all of your BSD partitions in a single PC partition). The remainder of the installation was uneventful, as I decided to install only the basics for the time being. bash was all I wanted to see work in the first place.

Then the moment of truth arrived, and I rebooted the box. Nothing happened. The box got stuck in the initial Acer splash screen, and neither F2 (edit BIOS) nor F12 (boot device menu) would work for me. All I could do is press Ctrl-Alt-Del to powercycle the brick. This didn't exactly come up to my expectations.

I was prepared to return the laptop, as it didn't seem to like non-Windoze OSes on its HDD at all. The freebsd-mobile mailing list came to the rescue, as others had noticed this problem before on other Acer notebooks. Turns out that Acer uses a weird BIOS setting for the internal HDD which apparently works well only with Windows. Now I was in a catch 22 as I wasn't able to edit the BIOS once FreeBSD was installed. The BIOS apparently got stuck in an endless loop trying to access the drive in an improper mode. Someone suggested to remove the HDD temporarily to avoid this loop, and it indeed allowed me to edit the BIOS and alter this setting. After restoring the drive, FreeBSD would boot just fine. Phew.

If you'd like to avoid this kind of trouble, here's how:

1) during the initial splash screen, hit F2 to enter BIOS setup

2) enable the boot menu

3) set the HDD mode from AHCI to IDE

4) reboot with a Clonezilla stick or CD to clone the HDD contents, in
case you need to go back to square one

5) reboot with your favourite FreeBSD install medium

And this is the proper way if you missed step 3):

1) turn the laptop upside down

2) loosen the two obvious screws of the HDD lid. They are secured by
retainers, so they won't come off entirely

3) use gentle force to lift up the lid, e.g. by means of a plastic
credit card. The lid is clipped in on three sides.

4) remove the HDD

5) switch on the box and hit F2 to enter BIOS setup

6) set the HDD mode from AHCI to IDE

7) switch off the box

8) put the HDD back into place and close the lid

9) switch on the box and, lo and behold, it is going to boot again

Malaysians

Mon, 16 Aug 2010 10:58:58 +0200

From a recruiting poster of the Japanese fast food chain MOS Burger, seen in Singapore:

"Housewives, students, retirees, and Malaysians welcome"

It is interesting though that the latter are missing in their job opportunities page on the web:

http://www.mosburger.com.sg/employment.php

How to run NTEmacs and Cygwin Emacs on the same box

Wed, 24 Mar 2010 15:12:27 +0100

I've decided to switch to Cygwin Emacs from NTEmacs lately. As I had previously seen that the Cygwin X server may refuse to work at times, I wasn't ready to deinstall NTEmacs altogether (the X problems were certainly caused by my lack of understanding, or lack of maintenance, or both. Cygwin X in general is said to run smoothly). I rather figured it should be possible to run both Emacsen in parallel, without duplicating all efforts which go into maintaining your hand-crafted .emacs file. So I tried to come up with a way to share my existing .emacs with both versions. The major problem is that NTEmacs requires quite a lot of tweaks to make it cooperate with Cygwin bash, which is a far superior shell compared to Windows "cmd", whereas Cygwin Emacs requires at least as many tweaks to make it cooperate with native Windows tools like web browser or proprietary Windows programs. This is how I solved the problem.The first question is how to reliably tell apart which version of Emacs is trying to load our .emacs. I settled for checking the value of "window-system". This is "w32" for NTEmacs, whereas it is nil for Cygwin Emacs running in a console window and "x" for the same program running as a GUI app under X. Therefore you get away with checking for "w32" to handle NTEmacs in the if body and to handle Cygwin Emacs in the else body.

I have maintained an "emacsen" directory for lisp code outside of my NTEmacs installation directory. This considerably simplifies upgrading, and it also comes in handy if you want to run two different versions of Emacs. The following (simplified) example code includes some of these lisp directories into the load path, taking care of the different path notations of NTEmacs and Cygwin Emacs:

(if (equal window-system 'w32)
(setq load-path
(nconc load-path (list "C:/Programme/emacsen/site-lisp"
"C:/Programme/emacsen/site-lisp/planner-latest"
"C:/Programme/emacsen/site-lisp/muse-latest/lisp"
)))
(setq load-path
(nconc load-path (list "/cygdrive/c/Programme/emacsen/site-lisp"
"/cygdrive/c/Programme/emacsen/site-lisp/planner-latest"
"/cygdrive/c/Programme/emacsen/site-lisp/muse-latest/lisp"
))))

Similar checks are required whenever paths are part of the code.

The next problem is to improve browse-url support for local files on Cygwin Emacs. My planner-mode based project management makes extensive use of links to local OpenOffice, SigmaPlot, SigmaStat, or Acrobat files. This is extremely convenient as your data or graphs or statistics are just a mouseclick away while you read the experimental protocol. This worked out of the box on NTEmacs. Cygwin Emacs requires a replacement for w32-shell-execute, which I borrowed from Ken Brown, and another helper function for browse-url:

(if (not (equal window-system 'w32))
(progn
;; Minimal replacement for w32-shell-execute under Cygwin.
(defun w32-shell-execute (operation document &optional parameters show-flag)
(if (string-equal operation "open")
(shell-command (concat "cygstart " (shell-quote-argument document)))))

;; This is a helper to open arbitrary files specified by a file:// kind of URL via browse-url in their default Win32 app
(defun w32-display-file (document)
(w32-shell-execute "open" document))
))

Now we need to customize (Options->Customize Emacs) browse-url. The variable "muse-url-protocols" should use our function "w32-display-file" whenever browse-url is called with an URL containing the "file://" protocol. The URL is essentially handed to the cygstart utility by our custom functions. cygstart uses conventional Windows magic to open files with their associated application.

One problem remains to be solved: If customizations involve paths, we need to use separate "custom-set-variables" blocks for NTEmacs and Cygwin Emacs.
;; save customizations into separate files depending on the Emacs version we're running. The custom file needs to be loaded manually
(if (equal window-system 'w32)
(setq custom-file "~/.emacs-nt-custom.el")
(setq custom-file "~/.emacs-cygwin-custom.el"))

(load custom-file)

This is all it takes to run both flavours of Emacs on Windows (and it could certainly be extended to cover XEmacs as well).

Speed comparison of NTEmacs and Cygwin Emacs

Wed, 24 Mar 2010 14:32:43 +0100

I may have mentioned previously that I'm using a Windows XP box as a Netware client at work. As I prefer the Unix way of doing things otherwise, I've been installing lots of Unix software on this box. The two main components are Cygwin, which is essentially a Unix-like environment including all essential GNU tools, and the native GNU Emacs port. The latter is built with MinGW, the "Minimalist GNU for Windows" tools. The native port has been around for years, I must have been using it for more than a decade now. However, Cygwin has also provided an Emacs port for a while. This one either runs in your terminal (like MinTTY), or as a GUI app if you use Cygwin's X server. A speed comparison made me think twice about my previous choice.I'm using planner-mode to organize my lab projects, and I usually start every work day by creating my HTML task list which doubles as my browser homepage. As my projects continue to grow, creating the task list took an increasing amount of time. As Emacs isn't multithreaded, Emacs would be locked up while I wasn't able to get much done. And even if it were multithreaded, it would still be unusable as the CPU load goes up to 100% due to hardware limitations. A week ago or so I was approaching the 10 minute barrier, and I decided that something must happen (don't ask why I was so patient until then). With a little help from Jim Ottaway I was able to do some profiling, and it turned out that NTEmacs spent half a second per page (in a 1000 page project!) to decide whether or not to publish this page to HTML. Publishing of the page was rather fast compared to that. I was interested if this was a Windows or NTEmacs peculiarity. To find out, I cloned the project to my FreeBSD laptop. Publishing took approx. 20 seconds, which is about 30 times faster. The hardware is more recent, but I wouldn't expect more than twice the speed based on the hardware differences.

Now the interesting test was to find out if Cygwin Emacs was closer to Unix or closer to Windows. I upgraded my Cygwin installation from 1.5.x to 1.7.1, as the X server of my older installation wouldn't work properly. The new version run ok, and off went the tests. Turned out that Cygwin Emacs was even faster than the FreeBSD version. So the planner-mode speed problem was solely due to a speed problem in the native Emacs port.

Here is a comparison of two function calls. The first number indicates the times the function is called, the second one is the total time in seconds spent in the function. The first function is close to the overall processing time, whereas the second one roughly indicates the processing time of individual planner pages.

NTEmacs
muse-publish-markup-region 4 880.836
muse-project-publish-file-default 979 443.916

Cygwin Emacs
muse-publish-markup-region 4 11.225999999
muse-project-publish-file-default 979 12.124000000

It is pretty obvious that NTEmacs has serious speed issues here. I'm well aware that this is not a serious benchmark or anything, but it is definitely something which I have to consider in my workflow. With that in mind I've decided to switch to Cygwin Emacs, the lack of file drag&drop support notwithstanding.

Emacs Photo Database now available

Wed, 14 May 2008 10:54:07 +0200

This is to announce that Emacs Photo Database is now available as a SourceForge-hosted project. As outlined previously, this is a database holding film, negative, and print information for photographers who use real darkrooms instead of digital cameras and a "lightroom".

Photo database front-end, 2nd try

Wed, 23 Apr 2008 22:44:06 +0200

I've reported previously that my attempts to create a simple photo database to manage my negatives and prints failed miserably with OpenOffice Base. I thought that installing a web frontend for SQLite might simplify adding and retrieving the data to a SQL database, but the results were not entirely to my taste. The tool simplified the database creation somewhat, and it is fairly easy to check the rows in the tables. However, there are two limitations:- if I need all information about a print - the print info, the negative info, and the film info - I have to join three tables. You can do this in the web interface as well, but only by typing the SQL query by hand - this isn't any simpler than doing it on the command line
- as the data are relational (each print belongs to a negative, and this in turn to a film), inserting data with the web interface isn't straightforward. You can easily fill in a form with the data of one table at a time, but you have to run separate queries to find the foreign keys that associate prints, negatives, and films.

I figured my photo database should be easier to manage than this. I recalled the ease with which I can retrieve an address or a phone number with my computer: switch to Emacs (it is open by default anyway), run "Alt-x rolo-grep", and there you go. Why shouldn't "Alt-x photo-find-negatives" provide the same convenience for my photo database?

As photo-mode would just be another front-end for a command-line application (sqlite3 in this case), I figured I could borrow some code from refdb-mode to get started real quick. And presto, within one-and-a-half days I had a feature-complete prototype running. Beat that with any tool or language other than Emacs Lisp.

Before describing how the system works, I'll have to recap how the data are organized. A film is either a 35mm/roll film or a set of sheet films which are developed together in the same tray using the same conditions. All development-related information, such as film type, developer type, developing conditions, goes into the t_film table. Each film may contain one to many negatives (the individual images). The exposure information, as well as a description and a foreign key that associates the negative with its film, go into the t_negative table. Finally I can make one or more prints from each negative. The paper type, exposure tricks, development conditions, and a foreign key which associates the print with a negative, go into the t_print table. I won't have to enter each individual print into this table, but rather different papers, exposure and development conditions that I try out while approaching the "perfect" print.

I settled for a unified naming scheme for each of the items which also simplifies data entry as will be described shortly. Film names look like "YYYYMMDD-XY" using the development date and a numerical suffix if I happen to develop more than one film per day. Each negative receives another two-digit number, i.e. "YYYYMMDD-XY-ZQ". The number ZQ corresponds to the position of the negative on the film roll or in the negative preserver (for sheet film). Each print (or rather print condition) receives yet another two-digit number, as in "YYYYMMDD-XY-ZQ-RS". This number on the back of each print is all it takes to find the relevant information in the database.

The database structure is defined as a SQL script. Run it through the sqlite3 command line client to generate an empty database.

As expected there are three sets of functions to enter new data, one for each table. In order to simplify entering several items at a time, each set has one function which creates an empty template (photo-create-film, photo-create-negative, photo-create-print), and another function that actually sends the data to the database (photo-add-film, photo-add-negative, photo-add-print). After creating and filling in a template, you can reuse it any number of times, so you'll have to change only the variant fields for similar datasets. photo-mode automatically inserts the appropriate foreign keys - all you need to specify is the negative (for print entry) or film (for negative entry) name - these are simple to remember due to the strict naming scheme.

Once the data are in the database, we'll need several functions to get at the data again. These functions are listed below:

- photo-find-negatives: searches the negative table for datasets that contain your search term in the location or description field. This is useful to find negatives if you know where they were taken, or what they show.
- photo-list-prints-of-negative, photo-list-prints-of-film: shows all prints that were made from a particular negative or film, respectively
- photo-print-info, photo-negative-info, photo-film-info: shows all information related to a particular print, negative, or film, respectively. The print query also displays the associated negative and film info. Accordingly, the negative query also displays the associated film info.
- photo-list-prints-of-film, photo-list-prints-of-negative: display a list of prints along with the negative description that were made from a particular film or negative, respectively.
- the preceding two categories also have equivalent functions that use a range of marked text, instead of text that you enter at the prompt, to retrieve the information. These functions are convenient to reuse matches returned by the *-find-* or *-list-* family of functions for further queries.

All this is coded in roughly 800 lines of Emacs Lisp. After some cleanup and writing brief instructions, I'll make the package available to the public, i.e. to all three photographers that are also fond of Emacs :-).

Installing SQLiteManager 1.2.0 on FreeBSD

Wed, 16 Apr 2008 01:59:47 +0200

I've been into b/w photography for at least 15 years now, and I've got a serious collection of negatives and prints. A rule of thumb says: the larger the format, the fewer images. As I'm using my 4x5 as much as possible, there's only a handful of negatives to add each year. Still, I'd like to keep track of the negatives and the prints in an easier way than I used to do so far: I scribbled the data onto legal pads and waded through them if I needed to look up something. This calls for some sort of database.First of all, conventional digital image databases are not up to the task. They are geared towards digital camera shots, but they cannot map a print to the corresponding negative, or the negative to the film (or batch of sheet films). However, I need just this in order to find the print exposure/development data, the film development data, and the negative exposure data associated with a given print.

I thought OpenOffice Base (2.3.1) would do the trick. Base is something like a MS Access clone, i.e. a front end with graphical tools to build tables, define relationships, and to create queries, forms, and reports. I might even be able to link the data to the scans of the negatives and prints which I keep on the computer anyway, and display them in a search form. I had to find out the hard way that Base is not even close to being usable, at least on FreeBSD. Whenever you do something interesting, it crashes reproducibly. I came across half a dozen ways to crash Base before being able to enter even a single complete dataset. As much as I like and rely upon the other OO tools, Base is ridiculous. I had to go back to a tried-and-true SQL database.

Although I usually don't shy away from command lines, a frontend would simplify data entry considerably. Think of issuing INSERT commands in tables containing a dozen fields or so. This is of course doable, but inconvenient. I came across SQLiteManager-1.2.0 which seemed to offer a simple and clean web interface to my tables.

The installation was far from smooth, although SQLiteManager is part of the FreeBSD ports collection (i.e. my box has the instructions to build and install the software from the original sources). SQLiteManager is a PHP5 web application which means a few additional hurdles on our way to eternal happiness. But lets go through the installation procedure step by step.

After installing the databases/sqlitemanager, databases/sqlite3, and /databases/php-sqlite3 ports (I intended to use version 3 databases right away), I had to edit the Apache config file manually:

Alias /sqlitemanager/ "/usr/local/www/sqlitemanager/"

AllowOverride None
Order allow,deny
Allow from all
AddType application/x-httpd-php .php .phtml

Running "apachectl restart" as root caused Apache to pick up the changes. Now I pointed my browser to "http://localhost/sqlitemanager/index.php" - and was greeted by a PHP error message saying something about missing session support. I recalled that FreeBSD needs an extra port (www/php5-session) to enable PHP sessions.

After restarting Apache and reloading the page, I got one step further, just to find out that there was no support for SQLite3 yet. I don't speak French, but the French installation instructions were clear enough to make me edit /usr/local/www/sqlitemanager/include/user_defined.inc.php to add the line

define("SQLITE3", true);

However, this was not sufficient. I happened to find the INSTALL file in the SQLiteManager installation directory which told me to install two additional database-related PHP ports: databases/php5-pdo and databases/php5-pdo_sqlite. With these in place (and Apache restarted), SQLiteManager offered SQLite3 support as well (watch out for the version numbers which are displayed on top of the start screen).

Now there was still a problem with file permissions. SQLiteManager keeps internal information in a SQLite database of its own. The INSTALL file already mentioned to set the permissions appropriately. On FreeBSD, Apache (and thus PHP scripts as part of a web interface) runs as user:group www:www. I changed the group of /usr/local/www/sqlitemanager/include/config.db (and of config3.db, although the installation instructions didn't mention it) to "www" and added write permissions for group members. However, I still got errors saying the config database is not writable. I recalled that in order to write to a file, you'll also have to write to the directory that contains the file in order to update the timestamp and the size information. Therefore I had to change the group and change the permissions of /usr/local/www/sqlitemanager/include as well.

It wasn't immediately apparent how to create a new database from scratch. The interface won't tell, and the (French) manual was of limited use as well. I finally figured that you have to enter a short database name (which will then be displayed on the left hand side of the web interface) in the upper field and a full path to the database file in the lower field. This file does not necessarily have to carry the same name as the display name. SQLiteManager of course needs write permissions in the target directory in order to create databases. To this end, I created /usr/local/var/db/sqlitemanager, changed the group to "www" and added write permissions for group members. I'd then use this directory to hold all database files created with SQLiteManager.

Finally, with less than 2 h of hairpulling, I was able to add my first table and start some serious work. I'll probably get back with further comments once I have gained some experience in handling my data with this interface.

How to connect your liver to your heart with a toothpick

Wed, 19 Sep 2007 14:20:10 +0200

Here's another one in our ongoing series of how to harm yourself by eating toothpicks.

A 55-year old man reported to the hospital with chest pain and a difficulty to breathe when not at rest. The pain was also noticeable when he was lying on his right side. The patient also suffered from a loss of appetite, accompanied by a loss of weight, and did not feel well in general. He was diagnosed with a massive pericardial effusion (a loss of blood from the heart to the surrounding tissue) and a constrictive pericarditis (an inflammation of the sac surrounding the heart). The patient was taken to a computer tomograph which confirmed the pericardial effusion and additionally detected an abscess of the left side of the liver. The scan also showed a tubular structure connecting the liver and the heart which was obviously of foreign origin. A laparotomy (surgical opening of the belly) revealed a wooden toothpick that somehow made its way out of the bowel, through the liver, and partially into the heart. The toothpick was removed and the patient recovered within a week.

The paper also contains a brief literature search which reveals that only 12% of the patients treated for symptoms of a toothpick gone the wrong way remember swallowing that toothpick. That is, only one in about 10 people eating toothpicks are aware of the fact. On the other hand, 80-90% of the cases where people remember swallowing a toothpick are uneventful. Clearly, there are situations that make you less aware of what you put in your mouth (think alcohol), but it still makes me wonder how absentminded you have to be in order not to notice. But then - shit happens.