"In het verleden behaalde resultaten bieden geen garanties voor de toekomst"

These are the ramblings of Matthijs Kooijman, concerning the software he hacks on, hobbies he has and occasionally his personal life.

Questions? Praise? Blame? Feel free to contact me.

My old blog (pre-2006) is also still available.

Sun Mon Tue Wed Thu Fri Sat

5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31
Tag Cloud
&
(With plugins: config, extensionless, hide, tagging, Markdown, macros, breadcrumbs, calendar, directorybrowse, entries_index, feedback, flavourdir, include, interpolate_fancy, listplugins, menu, pagetype, preview, seemore, storynum, storytitle, writeback_recent, moreentries)
Valid XHTML 1.0 Strict & CSS
Automatically restarting my serial console on Arduino uploads

When working with an Arduino, you often want the serial console to stay open, for debugging. However, while you have the serial console open, uploading will not work (because the upload relies on the DTR pin going from high to low, which happens when opening up the serial port, but not if it's already open). The official IDE includes a serial console, which automatically closes when you start an upload (and once this pullrequest is merged, automatically reopens it again).

However, of course I'm not using the GUI serial console in the IDE, but minicom, a text-only serial console I can run inside my screen. Since the IDE (which I do use for compiling uploading, by calling it on the commandline using a Makefile - I still use vim for editing) does not know about my running minicom, uploading breaks.

I fixed this using some clever shell scripting and signal-passing. I have an arduinoconsole script (that you can pass the port number to open - pass 0 for /dev/ttyACM0) that opens up the serial console, and when the console terminates, it is restarted when you press enter, or a proper signal is received.

The other side of this is the Makefile I'm using, which kills the serial console before uploading and sends the restart signal after uploading. This means that usually the serial console is already open again before I switch to it (or, I can switch to it while still uploading and I'll know uploading is done because my serial console opens again).

For convenience, I pushed my scripts to a github repository, which makes it easy to keep them up-to-date too:

Introducing Tika

(This post has been lying around as a draft for a few years, thought I'd finish it up and publish it now that Tika has finally been put into production)

A few months years back, I purchased a new server together with some friends, which we've named "Tika" (daughter of "Tita Tovenaar", both wizards from a Dutch television series from the 70's). This name combine's Daenney's "wizards and magicians" naming scheme with my "Television shows from my youth" naming schemes quite neatly. :-)

It's a Supermicro 5015A rack server sporting an Atom D510 dual core processor, 4GB ram, 500GB of HD storage and recently added 128G of SSD storage. It is intended to replace Drsnuggles, my current HP DL360G2 (which has been very robust and loyal so far, but just draws too much power) as well as Daenney's Zeratul, an Apple Xserve. Both of our current machines draw around 180W, versus just around 20-30W for Tika. :-D You've got to love the Atom processor (and it probably outperforms our current hardware anyway, just by being over 5 years newer...).

Over the past three years, I've been working together with Daenney and Bas on setting up the software stack on Tika, which proved a bit more work than expected. We wanted to have a lot of cool things, like LXC containers, privilege separation for webapplications, a custom LDAP schema and a custom web frontend for user (self-)management, etc. Me being the perfectionist I am, it took quite some effort to get things done, also producing quite a number of bug reports, patches and custom scripts in the process.

Last week, we've finally put Tika into production. My previous server, drsnuggles had a hardware breakdown, which forced me wrap up Tika's configuration into something usable (which still took me a week, since I seem to be unable to compromise on perfection...). So now my e-mail, websites and IRC are working as expected on Tika, with the stuff from Bas and Daenney still needing to be migrated.

I also still have some draft postings lying around about Maroesja, the custom LDAP schema / user management setup we are using. I'll try to wrap those up in case others are interested. The user management frontend we envisioned hasn't been written yet, but we'll soon tire of manual LDAP modification and get to that, I expect :-)

JTAG and SPI headers for the Pinoccio Scout

The Pinoccio Scout is a wonderful Arduino-like microcontroller board that has builtin mesh networking, a small form factor and a ton of resources (at least in Arduino terms: 32K of SRAM and 256K of flash).

However, flashing a new program into the scout happens through a serial port at 115200 baud. That's perfectly fine when you only have 32K of flash or for occasional uploads. But when you upload a 100k+ program dozens of times per day, it turns out that that's actually really slow! Uploading and verifying a 104KiB sketch takes over 30 seconds, just too long to actually wait for it (so you do something else, get distracted, and gone is the productivity).

See more ...

Using a JTAGICE3 programmer under Linux: Setting up permissions

Last week, I got a fancy new JTAGICE3 programmer / debugger. I wanted to achieve two things in my Pinoccio work: Faster uploading of programs (Having 256k of flash space is nice, but flashing so much code through a 115200 baud serial connection is slow...) and doing in-circuit debugging (stepping through code and dumping variables should turn out easier than adding serial prints and re-uploading every time).

In any case, the JTAGICE3 device is well-supported by avrdude, the opensource uploader for AVR boards. However, unlike devices like the STK500 development board, the AVR dragon programmer/debugger and the Arduino bootloader, which use an (emulated) serial port to communicate, the JTAGICE3 uses a native USB protocol. The upside is that the data transfer rate is higher, but the downside is that the kernel doesn't know how to talk to the device, so it doesn't expose something like /dev/ttyUSB0 as for the other devices.

avrdude solves this by using libusb, which can talk to USB devices directly, through files in /dev/usb/. However, by default these device files are writable only by root, since the kernel has no idea what kind of devices they are and whom to give permissions.

To solve this, we'll have to configure the udev daemon to create the files in /dev/usb with the right permissions. I created a file called /etc/udev/rules.d/99-local-jtagice3.rules, containg just this line:

SUBSYSTEM=="usb", ATTRS{idVendor}=="03eb", ATTRS{idProduct}=="2110", GROUP="dialout"


This matches the JTAGICE3 specifically using it's USB vidpid (03eb:2110, use lsusb to find the id of a given device) and changes the group for the device file to dialout (which is also used for serial devices on Debian Linux), but you might want to use another group (don't forget to add your own user to that group and log in again, in any case).

Current measurement helper board

Recently, I needed to do battery current draw measurements on my Pinoccio boards. Since the battery is connected using this tinywiny JST connector, I couldn't just use some jumper wires to redirect the current flow through my multimeter. I ended up using jumper wires, combined with my Bus Pirate fanout cable, which has female connectors just small enough, to wire everything up. The result was a bit of a mess:

Admittedly, once I cleaned up all the other stuff around it from my desk for this picture, it was less messy than I thought, but still, jamming in jumper wires into battery connectors like this is bound to wear them out.

So, I ordered up some JST FSH connectors (as used by the battery) and some banana sockets and built a simple board that allows connecting a power source and a load, keeping the ground pins permanently connected, but feeding the positive pins through a pair of banana sockets where a current meter can plug in. For extra flexibility, I added a few other connections, like 2.54mm header pins and sockets, a barrel jack plug and more banana sockets for the power source and load. I just realized I should also add USB connectors, so I can easily measure current used by an USB device.

The board also features a switch (after digging in my stash, I found one old three-way switch, which is probably the first component to die in this setup. The switch allows switching between "on", "off" and "redirect through measurement pins" modes. I tried visualizing the behaviour of the pins on the top of the PCB, but I'm not too happy with the result. Oh well, as long as I know what does :-)

All I need is a pretty case to put under the PCB and a μCurrent to measure small currents accurately and I'm all set!

Update: The board was expanded by adding an USB-A and USB-B plug to interrupt USB power, with some twisted wire to keep the data lines connected, which seems to work (not shown in the image).

Updating the Xprotolab portable firmware on Linux

I recently got myself an Xprotolab Portable, which is essentially a tiny, portable 1Msps scope (in hindsight I might have better gotten the XMinilab Portable which is essentially the same, but slightly bigger, more expensive and with a bigger display. Given the size of the cables and carrying case, the extra size of the device itself is negligable, while the extra screen size is significant).

In any case, I wanted to update the firmware of the device, but the instructions refer only to a Windows-only GUI utility from Atmel, called "FLIP". I remembered seeing a flip.c file inside the avrdude sources though, so I hoped I could also flash this device using avrdude on Linux. And it worked! Turns out it's fairly simple.

1. Activate the device's bootloader, by powering off, then press K1 and keep it pressed while turning the device back on with the menu key. The red led should light up, the screen will stay blank.
2. Get the appropriate firmware hex files from the Xprotolab Portable page. You can find them at the "Hex" link in the top row of icons.
3. Run avrdude, for both the application and EEPROM contents:

sudo avrdude -c flip2 -p x32a4u -U application:w:xprotolab-p.hex:i
sudo avrdude -c flip2 -p x32a4u -U eep:w:xprotolab-p.eep:i


I'm running under sudo, since this needs raw USB access to the USB device. Alternatively, you can set up udev to offer access to your regular user (like I did for the JTAGICE3), but that's probably too much effort just for a one-off firmware update.

4. Done!

Note that you have to use avrdude version 6.1 or above, older versions don't support the FLIP protocol.

Dynamic memory allocation debugging

While trying to track down a reset bug in the Pinoccio firmware, I suspected something was going wrong in the dynamic memory management (e.g., double free, or buffer overflow). For this, I wrote some code to log all malloc, realloc and free calls, as wel as a python script to analyze the output.

This didn't catch my bug, but perhaps it will be useful to someone else.

In addition to all function calls, it also logs the free memory after the call and shows the return address (e.g. where the malloc is called from) to help debugging.

It uses the linker's --wrap, which allows replacing arbitrary functions with wrappers at link time. To use it with Arduino, you'll have to modify platform.txt to change the linker options (I hope to improve this on the Arduino side at some point, but right now this seems to be the only way to do this).

DIY desk cabinet and power switch panels

Since ages, I've been using switched power bars to connect my computer equipment. Originally, I used a normal outlet to connect my PC and a switched bar to connect my monitor, speakers, printer, etc. This allows me to actually completely switch off everything when I'm not using it, saving a couple of Watts of leakage power and removing the need to switch all of my equipment off separately.

Since then, the amount of electrical stuff on my desk has increased a bit, now also including quite some stuff unrelated to my computer. I now count: a laptop adapter, an USB hub, speakers, a printer/scanner, an external hard disk, a battery charger, a phone charger, a charger for my shaver, some soldering equipment, a pile of wireless access points (my work for Fon).

Plugging in all this stuff in a single power bar wasn't very helpful: I would have the bar turned on most of the day when working on my laptop, so all the other devices would be leaking power as well.

It thought I found the perfect solution when I found this power bar with five individually switched outlets from Brennenstuhl. However, because both the switches and the outlets need some space, this power bar is pretty huge (±60 cm long), making it pretty impossible to give it a useful place on my desk (without taking up too much desk space), so I would have to think of something else.

I have been thinking about building a nice cabinet to put on top of my desk to put my laptop, speakers and printer on top of and to put various paperwork and other misc items in (to replace the cardboard boxes that kept my laptop at an ergonomical height until then). In the design of that cabinet, I realized that I could just as well include some power switch panels in the cabinet.

After a few iterations of the design, I ended up with the following result:

As you can see, there's two switch panels with eight switches each in the lower corners. Each switch controls one or two outlets, most of which are hidden behind the panels. Since most of the devices don't move around much, it's perfect if the outlets and plugs are hidden inside the cabinet, since that means less cluttering of my desk.

However, I also have some devices (mostly wireless access points, settopboxes and monitors that I use for my work) that are not fixed all the time and regularly change. If I would need to get behind the switch panels everytime I wanted to change a device, I would get tired of that real quick. I could of course have used a completely separate outlet bar for those devices, but I still like to make those devices individually switched (since most of them have power adapters, which are prone to leak power, and I hardly ever use all of the devices at the same time).

So, I ended up making five of the switches in the right panel control outlets in a separate power bar, which I put on my (second) desk instead of behind the panel. To be able to do this, I had to get a length of thick power cable containing 12 separate leads (two for each outlet, plus one for the earth terminal). Also, since I didn't want the cable to be fixed onto the switch box, I got a humongous power plug, containing 10 contacts (plus chassis earth).

I designed these boxes for a current up to 10A. All components should be able to handle 16A, except for the power inlets, which contain a 10A fuse. Each switch panel contains 8 switches, for controlling 9 outlets (theres one switch in each box that controls two outlets).

I used two-pole switches, which can switch both the line and neutral terminals of each outlets. In theory, it should be sufficient to only switch the line terminals and leave the neutral terminals always connected, but since the boxes are connected to a wall socket using a normal power plug, there is no way to tell which incoming power wire is the line terminal and which is the neutral terminal, so you have to switch both of them.

One thing I had not accounted for in advance, is that the switch boxes themselves also draw a bit of power. The power inlets I used contain some filtering circuitry, which should improve the stability of the power delivered. However, these inlets also draw about 50-100mW (hard to measure since it's so little and has a low power factor) each, when there is nothing else connected. On yearly basis, this would amount to about one kWh, so that's not really a problem.

In addition, the lights in the switches also draw around 300mW, but since that only happens when a device is switched on, that power draw is negligible compared to the power usage of the connected device.

I started out building the cabinet, which took me just under a day (I just screwed all the panels together, not bothering with fine dovetail joints or other fine woodworking details, since I mostly wanted this cabinet to be functional without costing too much time. Then I started on the switch boxes, which took me three or four days in total, which seems really out of proportion :-)

## Front panels

I started out preparing the front panel, using an oldschool fretsaw. Fortunately, the switches have a small front panel that covers up a few milimeters of the hole, so I didn't have to get perfect cuts (though a few mm of space is still not that much room for error).

The switches are mounted in the panel by just snapping them into the holes cut out of the panel.

## Outlets

For the outlets, I was considering to use these power sockets or standard wall sockets, with the big downside of them being fairly expensive (± €8, I would need 20 of them). Fortunately, I found a box of old wall outlets at van Altena, local second-hand hardware store that has all kinds of old and new tools and equipment at cheap prieces. I ended up paying €0,75 per outlet, which reduced the total cost of the project quite a bit (though it still cost me around €175 in materials).

To save a bit of space and make everything fit better, I ended up cutting off the top of the outlets, so the outer mantle would come off and the outlets could be placed closer togeter. After doing the first one with a saw, I ended up doing all the others using a belt sander, fixed upside down in my vice creating an ad-hoc stationary belt sander.

## Box construction

Next, I created a base for the boxes and constructed the boxes on top. Apparently I didn't take any pictures of the process (I remember doing so, but perhaps I somehow lost the pictures, not sure). The base plate is thick (18mm) MDF, on which I've used my router to route out a 10mm deep "trench" in the wood through which the cabling can be put (so it can go underneath the outlets). You can see one of the trenches on the left of the right box below.

Most of the rest of the construction is 15mm plywood, except for the top plate (with holes for the outlets), I used 6mm MDF for that.

## Distribution wires

To connect the incoming power lines to each of the switches, I had a bit of a challenge. The switches have blade connectors, onto which blade receptacle connectors can be attached. These blade receptacles are "crimped" onto a cable. I tried to find some component that I could use to connect all a dozen of blade receptacles onto, but I couldn't find any of those. There are special blade connectors that allow daisy chaining each switch onto the next, but I didn't like the idea that the last switch would be connected through two dozen of crimp connections (and I did not have those daisy chaining connectors when I was working on this).

I also could not use a Twist-on wire connector, since that only works with solid cable (and the crimped blade connectors need stranded cable).

In the end, I just settled for soldering all these cables together. It's not very elegant, but I'm confident that this soldering connection should be capable of easily handling 16A of current, which is the most important in the end. I used two layers of heat-shrink tubing to insulate the big lump of soldering tin.

The last picture shows the end result: four cable octopusses that connect with one wire to the power entry point, and with eight wires to each of the switches. There's four of these in total, for both the line (with the brown marking tape) and neutral (with the white marking tape) connections in both of the boxes.

## Connecting the outlets

The line and neutral connections of each outlet need to be connected to a single switch. I first connected wires to all of the outlets, with a blade connector crimped onto the other side. I used wires cut from the multicable I bought, which are conveniently numbered. Since the line and neutral connections are interchangeable in the European outlet system ("Schuko"), I didn't need to add labels to distinguish both cables coming from the same outlet.

The earth terminals do not need to pass the switch, these need to be always connected. In the bigger box, I used an earthing terminal block (which is intended to be used in a breaker box) to connect the outlets in a "star" topology. For the smaller box, I couldn't find a smaller earthing terminal block, so I just daisy chained the outlets together. Since there's only a few outlets and there should never be much current through the earth terminals, this shouldn't be a problem here.

I also connected the multicable connector in the same way, but those wires are already covered by a MDF plate in the pictures.

The end result is 32 cables sticking out from each box, four cables for each switch.

## Connecting the switches

Connecting the switches was a matter of connecting all the 32 wires I prepared to the switches in the panel. After connecting the switches, I glued the front panel onto the box (so there would not be any visible screws on the front panel).

## Assembled boxes

Now the boxes are assembled and ready for use, time to build the external outlet bar.

Note that the outlets seem a bit randomly placed, but I've tried to leave as much room available for power adapters to stick out in different directions, without blocking other outlets.

## External outlets

For the external outlets, I got an old powerbar (again at van Altena), which has each outlet individually connected internally. I connected the big multicable to all of the outlets on side and to the big connector on the other side of the cable

The old powerbar also had a single powerswitch to control all of the outlets. Since I couldn't find any use for this switch, I just removed it. To cover up the hole that it left, I used two rectangles cut out of an old plastic cable conduit. The smaller rectangle exactly fits in the hole, the larger rectangle just serves to glue the smaller one in place. The result is a cleanly covered switch hole.

## Fixing the front panels

After installing the boxes into the cabinet, it turned out the front panels weren't glued on properly, the left one came off. Since I was already doubting if they would hold, I settled for a less elegant, but more secure solution: I just added some screws in an angle from the back, making them just long enough to not stick out at the front.

## Finished

So, here's the finished thing. I really like the contrast between the green buttons and the red mahogany finish I used, it gives the thing a very classic look.

To be able to plug in devices, I can pull out the boxes towards the front. This is not very quick to do, but since I do not need to switch devices very often, that works out just fine.

I also added labels to the various switches, so I won't have to remember where I plugged in every device.

## Design drawings

And just for historical reference, here are the design drawings I created during the process. If they look fuzzy to you, that's probably because I erased every part of the drawings probably at least once...

Booting an old Sparc Ultra1 with dead NVRAM

To be able to debug a bug in OpenTTD that only occured on SPARC machines, I needed an old SPARC machine so I could reproduce and hopefully fix the bug. After some inquiring at our local hackerspace, Bitlair (which had a few SPARC32 machines lying around, but I needed SPARC64), I got in touch with The_Niz from NURDSpace, the hackerspace in Wageningen. Surprisingly, it turned out I actually knew The_Niz already through work :-)

In any case, The_Niz was kind enough to lend me a SPARC Ultra1 workstation and Sun keyboard, and r3boot gave me a Sun monitor I could use (of course Sun hardware doesn't use a regular VGA connector...).

There was one caveat, though: The NVRAM battery in the Ultra1 was dead. The NVRAM chip stores the boot settings (like the BIOS settings in a regular PC), but also the serial number and MAC address of the machine (called the IDPROM info). Without those settings, you'll have to manually select the boot device on every boot (by typing commands at a prompt) and netbooting does not work, for lack of a MAC address (I presume regular networking, e.g., from within Linux, does not work either, but I haven't tested that yet).

Sun has taken an interesting approach to their NVRAM chip. Where most machines just use a piece of EEPROM (Flash) memory, which does not need power to remember its contents, Sun has used a piece of RAM memory (which needs power to remember its contents) combined with a small rechargeable battery.

This means that when the machine is not used for a long time or is very old, the battery will eventually die, causing the machine to spit out messages like "The IDPROM contents are invalid and "Internal loopback test -- Did not receive expected loopback packet".

I needed to install Debian on this Ultra1, so I needed some way to boot the installer. Since netbooting did not work, my first attempt was to ignore the NVRAM problem and just get the machine to boot off a Debian boot cd. This did not quite work out: the cdrom player in the Ultra1 (a 4x burner connected through SCSI) didn't like any of the CD-Rs and CD-RWs I threw at it. It spat out errors like "Illegal Instruction", "Program Terminated" or "SProgram Terminated" (where the first "S" is the start of "SILO", the SPARC bootloader).

As we all remember from a long time ago, the early generation CD-ROM players were quite picky with burnt CDs, so also this one. I found some advice online to burn my CDs at a lower speed (apparently the drive was rumoured to break on discs burned at 4x or higher), but my drive or CD-Rs didn't support writing slower than 8x... I could write my CD-RW at 4x and at one occasion I managed to boot the installer from a CD-RW and succesfully (but very, very slowly) scan the CD-RW contents, but then it broke with read errors when trying to actually load files from the CD-RW.

So, having no usable CD-ROM drive to boot from, I really had to get netboot going. Apparently it is possible (and even easy and not so expensive) to replace the NVRAM chip, but I didn't feel like waiting for one to be shipped. There is a FAQ available online with extensive documentation about the NVRAM chip and instructions on how to reprogram the IDPROM part with valid contents.

So, I reckoned that the battery was only needed when the machine was powered down, so I should be able to reprogram the IDPROM info and then just not poweroff the system, right?

Turns out that works perfectly. I reprogrammed the IDPROM using the MAC address I read off the sticker on the NVRAM chip and made up a dummy serial number. For some reason, the mkpl command did not work, I had to use the more verbose mkp command. Afterwards, I gave the reset command and the "Internal loopback test" error was gone and the machine started netbooting, using RARP and TFTP.

By now, I've managed to install Debian, get Xorg working and reproduce the bug in OpenTTD, so time for fixing it :-D

Thinkpad X201 mute button breaking speaker output

Recently, I was having some problems with the internal speakers on my Lenovo Thinkpad X201. Three times now, the internal speakers just stopped producing sound. The headphone jack worked, it's just the speakers which were silent. Nothing helped: fiddling with volume controls, reloading alsa modules, rebooting my laptop, nothing fixed the sound...

When trying to see if the speakers weren't physically broken, I discovered that booting into Windows actually fixed the problem and restored the sound from the speakers. It's of course a bit of a defeat to accept Windows a fix for my problem, but I was busy with other things, so it sufficed for a while.

When migrating my laptop to my new Intel SSD, I broke my Windows installation, so when the problem occured again, I had no choice but to actualy investigate it.

I'll skip right to the conclusion here: I had broken my sound by pressing the mute button on my keyboard... Now, before you think I'm stupid, I had of course checked my volume controls and the device really was unmuted! But it turns out the mute button in Thinkpads combined with Linux is a bit weird...

This is how you would expect a mute button to be implemented: You press the mute button, it sends a keypress to the operating system, which then tells the audio driver to mute.

This is how it works on my Thinkpad: You press the mute button, causing the EC (embedded controller) in the thinkpad to directly mute the speakers. This is not visible from the normal volume controls in the software, since it happens on a very low level (though the thinkpad_acpi kernel module can be used to expose this special mute state through a /proc interface and special audio device).

In addition to muting the speakers, it also sends a MUTE acpi keypress to the operating system. This keypress then causes the audio driver to mute the audio stream (actually, it's pulseaudio that does that).

Now, here's the fun part: If you now unmute the audio stream through the software volume controls, everything looks like it should work, but the hardware is still muted! It never occured to me to press the mute button again, since the volume wasn't muted (or at least didn't look like it).

I originally thought that the mute button handling was even more complex, when I found some register polling code that faked keypresses, but it seems that's only for older Thinkpads (phew!).

In any case, the bottom line is: If you have a Thinkpad whose speakers suddely stop working, try pressing the mute button!