"In het verleden behaalde resultaten bieden geen garanties voor de toekomst"
About this blog

These are the ramblings of Matthijs Kooijman, concerning the software he hacks on, hobbies he has and occasionally his personal life.

Most content on this site is licensed under the WTFPL, version 2 (details).

Questions? Praise? Blame? Feel free to contact me.

My old blog (pre-2006) is also still available.

Sun Mon Tue Wed Thu Fri Sat
28 29 30
Powered by Blosxom &Perl onion
(With plugins: config, extensionless, hide, tagging, Markdown, macros, breadcrumbs, calendar, directorybrowse, entries_index, feedback, flavourdir, include, interpolate_fancy, listplugins, menu, pagetype, preview, seemore, storynum, storytitle, writeback_recent, moreentries)
Valid XHTML 1.0 Strict & CSS
Script to generate pinout listings for STM32 MCUs

STM32 Chip

Recently, I've been working with STM32 chips for a few different projects and customers. These chips are quite flexible in their pin assignments, usually most peripherals (i.e. an SPI or UART block) can be mapped onto two or often even more pins. This gives great flexibility (both during board design for single-purpose boards and later for a more general purpose board), but also makes it harder to decide and document the pinout of a design.

ST offers STM32CubeMX, a software tool that helps designing around an STM32 MCU, including deciding on pinouts, and generating relevant code for the system as well. It is probably a powerful tool, but it is a bit heavy to install and AFAICS does not really support general purpose boards (where you would choose between different supported pinouts at runtime or compiletime) well.

So in the past, I've used a trusted tool to support this process: A spreadsheet that lists all pins and all their supported functions, where you can easily annotate each pin with all the data you want and use colors and formatting to mark functions as needed to create some structure in the complexity.

However, generating such a pinout spreadsheet wasn't particularly easy. The tables from the datasheet cannot be easily copy-pasted (and the datasheet has the alternate and additional functions in two separate tables), and the STM32CubeMX software can only seem to export a pinout table with alternate functions, not additional functions. So we previously ended up using the CubeMX-generated table and then adding the additional functions manually, which is annoying and error-prone.

So I dug around in the CubeMX data files a bit, and found that it has an XML file for each STM32 chip that lists all pins with all their functions (both alternate and additional). So I wrote a quick Python script that parses such an XML file and generates a CSV script. The script just needs Python3 and has no additional dependencies.

To run this script, you will need the XML file for the MCU you are interested in from inside the CubeMX installation. Currently, these only seem to be distributed by ST as part of CubeMX. I did find one third-party github repo with the same data, but that wasn't updated in nearly two years). However, once you generate the pin listing and publish it (e.g. in a spreadsheet), others can of course work with it without needing CubeMX or this script anymore.

For example, you can run this script as follows:

$ ./ /usr/local/cubemx/db/mcu/STM32F103CBUx.xml
(... more output truncated ...)

The script is not perfect yet (it does not tell you which functions correspond to which AF numbers and the ordering of functions could be improved, see TODO comments in the code), but it gets the basic job done well.

You can find the script in my "scripts" repository on github.

0 comments -:- permalink -:- 13:35
/ Blog / Blog
Using MathJax math expressions in Markdown

For this blog, I wanted to include some nicely-formatted formulas. An easy way to do so, is to use MathJax, a javascript-based math processor where you can write formulas using (among others) the often-used Tex math syntax.

However, I use Markdown to write my blogposts and including formulas directly in the text can be problematic because Markdown might interpret part of my math expressions as Markdown and transform them before MathJax has had a chance to look at them. In this post, I present a customized MathJax configuration that solves this problem in a reasonable elegant way.

See more ...

4 comments -:- permalink -:- 17:03
Making an old paint-mixing terminal keyboard work with Linux

Lacour paint terminal

Or: Forcing Linux to use the USB HID driver for a non-standards-compliant USB keyboard.

For an interactive art installation by the Spullenmannen, a friend asked me to have a look at an old paint mixing terminal that he wanted to use. The terminal is essentially a small computer, in a nice industrial-looking sealed casing, with a (touch?) screen, keyboard and touchpad. It was by "Lacour" and I think has been used to control paint mixing machines.

They had already gotten Linux running on the system, but could not get the keyboard to work and asked me if I could have a look.

The keyboard did work in the BIOS and grub (which also uses the BIOS), so we know it worked. Also, the BIOS seemed pretty standard, so it was unlikely that it used some very standard protocol or driver and I guessed that this was a matter of telling Linux which driver to use and/or where to find the device.

Inside the machine, it seemed the keyboard and touchpad were separate devices, controlled by some off-the-shelf microcontroller chip (probably with some custom software inside). These devices were connected to the main motherboard using a standard 10-pin expansion header intended for external USB ports, so it seemed likely that these devices were USB ports.

See more ...

0 comments -:- permalink -:- 18:52
Reliable long-distance Arduino communication: RS485 & MODBUS?

Arduino connected to other things

For a customer, I've been looking at RS-485 and MODBUS, two related protocols for transmitting data over longer distances, and the various Arduino libraries that exist to work with them.

They have been working on a project consisting of multiple Arduino boards that have to talk to each other to synchronize their state. Until now, they have been using I²C, but found that this protocol is quite susceptible to noise when used over longer distances (1-2m here). Combined with some limitations in the AVR hardware and a lack of error handling in the Arduino library that can cause the software to lock up in the face of noise (see also this issue report), makes I²C a bad choice in such environments.

So, I needed something more reliable. This should be a solved problem, right?

See more ...

2 comments -:- permalink -:- 12:42
Recovering data from a failing hard disk with HFS+

Recently, a customer asked me te have a look at an external hard disk he was using with his Macbook. It would show up a file listing just fine, but when trying to open actual files, it would start failing. Of course there was no backup, but the files were very precious...

This started out as a small question, but ended up in an adventure that spanned a few days and took me deep into the ddrescue recovery tool, through the HFS+ filesystem and past USB power port control. I learned a lot, discovered some interesting things and produced a pile of scripts that might be helpful to others. Since the journey seems interesting as well as the end result, I will describe the steps I took here, "ter leering ende vermaeck".

See more ...

0 comments -:- permalink -:- 14:53
Modifying a LED strip DMX dimmer for incandescent bulbs

DMX PWM dimmer module

For a theatre performance, I needed to make the tail lights of an old car controllable through the DMX protocol, which the most used protocol used to control stage lighting. Since these are just small incandescent lightbulbs running on 12V, I essentially needed a DMX-controllable 12V dimmer. I knew that there existed ready-made modules for this to control LED-strips, which also run at 12V, so I went ahead and tried using one of those for my tail lights instead.

I looked around ebay for a module to use, and found this one. It seems the same design is available from dozens of different vendors on ebay, so that's probably clones, or a single manufacturer supplying each.

DMX module details

This module has a DMX input and output using XLR or a modular connector, and screw terminals for 12V power input, 4 output channels and one common connection. The common connection is 12V, so the output channels sink current (e.g. "Common anode"), which is relevant for LEDs. For incandescent bulbs, current can flow either way, so this does not really matter.

Dimmer module PCB

Opening up the module, it seems fairly simple. There's a microcontroller (or dedicated DMX decoder chip? I couldn't find a datasheet) inside, along with two RS-422 transceivers for DMX, four AP60T03GH MOSFETS for driving the channels, and one linear regulator to generate a logic supply voltage.

On the DMX side, this means that the module has a separate input and output signals (instead of just connecting them together). It also means that the DMX signal is not isolated, which violates the recommendations of the DMX specification AFAIU (and might be problematic if there is more than a few volts of ground difference). On the output side, it seems there are just MOSFETs to toggle the output, without any additional protection.

See more ...

0 comments -:- permalink -:- 16:56
Running an existing Windows 7 partition under QEMU/KVM/virt-manager

I was previously running an ancient Windows XP install under Virtualbox for the occasional time I needed Windows for something. However, since Debian Stretch, virtualbox is no longer supplied, due to security policy problems, I've been experimenting with QEMU, KVM and virt-manager. Migrating my existing VirtualBox XP installation to virt-manager didn't work (it simply wouldn't boot), and I do not have any spare Windows keys lying around, but I do have a Windows 7 installed alongside my Linux on a different partition, so I decided to see if I could get that to boot inside QEMU/KVM.

An obvious problem is the huge change in hardware between the real and virtual environment, but apparently recent Windows versions don't really mind this in terms of drivers, but the activation process could be a problem, especially when booting both virtually and natively. So far I have not seen any complications with either drivers or activation, not even after switching to virtio drivers (see below). I am using an OEM (preactivated?) version of Windows, so that might help in this area.

Update: When booting Windows in the VM a few weeks later, it started bugging me that my Windows was not genuine, and it seems no longer activated. Clicking the "resolve now" link gives a broken webpage, and going through system properties suggests to contact Lenovo (my laptop provider) to resolve this (or buy a new license). I'm not yet sure if this is really problematic, though. This happened shortly after replacing my hard disk, though I'm not sure if that's actually related.

Rebooting into Windows natively shows it is activated (again or still), but booting it virtually directly after that still shows as not activated...

Creating the VM

Booting the installation was actually quite painless: I just used the wizard inside virt-manager, entered /dev/sda (my primary hard disk) as the storage device, pressed start, selected to boot Windows in my bootloader and it booted Windows just fine.

Booting is not really fast, but once it runs, things are just a bit sluggish but acceptable.

One caveat is that this adds the entire disk, not just the Windows partition. This also means the normal bootloader (grub in my case) will be used inside the VM, which will happily boot the normal default operating system. Protip: Don't boot your Linux installation inside a VM inside that same Linux installation, both instances will end up fighting in your filesystem. Thanks for fsck, which seems to have fixed the resulting garbage so far...

To prevent this, make sure to actually select your Windows installation in the bootloader. See below for a more permanent solution.

See more ...

0 comments -:- permalink -:- 18:13
Calculating a constant path basename at compiletime in C++

In some Arduino / C++ project, I was using a custom assert() macro, that, if the assertion would fail show an error message, along with the current filename and line number. The filename was automatically retrieved using the __FILE__ macro. However, this macro returns a full path, while we only had little room to show it, so we wanted to show the filename only.

Until now, we've been storing the full filename, and when an assert was triggered we would use the strrchr function to chop off all but the last part of the filename (commonly called the "basename") and display only that. This works just fine, but it is a waste of flash memory, storing all these (mostly identical) paths. Additionally, when an assertion fails, you want to get a message out ASAP, since who knows what state your program is in.

Neither of these is really a showstopper for this particular project, but I suspected there would be some way to use C++ constexpr functions and templates to force the compiler to handle this at compiletime, and only store the basename instead of the full path. This week, I took up the challenge and made something that works, though it is not completely pretty yet.

Working out where the path ends and the basename starts is fairly easy using something like strrchr. Of course, that's a runtime version, but it is easy to do a constexpr version by implementing it recursively, which allows the compiler to evaluate these functions at compiletime.

For example, here are constexpr versions of strrchrnul(), basename() and strlen():

 * Return the last occurence of c in the given string, or a pointer to
 * the trailing '\0' if the character does not occur. This should behave
 * just like the regular strrchrnul function.
constexpr const char *static_strrchrnul(const char *s, char c) {
  /* C++14 version
    if (*s == '\0')
      return s;
    const char *rest = static_strrchr(s + 1, c);
    if (*rest == '\0' && *s == c)
      return s;
    return rest;

  // Note that we cannot implement this while returning nullptr when the
  // char is not found, since looking at (possibly offsetted) pointer
  // values is not allowed in constexpr (not even to check for
  // null/non-null).
  return *s == '\0'
      ? s
      : (*static_strrchrnul(s + 1, c) == '\0' && *s == c)
        ? s
        : static_strrchrnul(s + 1, c);

 * Return one past the last separator in the given path, or the start of
 * the path if it contains no separator.
 * Unlike the regular basename, this does not handle trailing separators
 * specially (so it returns an empty string if the path ends in a
 * separator).
constexpr const char *static_basename(const char *path) {
  return (*static_strrchrnul(path, '/') != '\0'
      ? static_strrchrnul(path, '/') + 1
      : path

/** Return the length of the given string */
constexpr size_t static_strlen(const char *str) {
  return *str == '\0' ? 0 : static_strlen(str + 1) + 1;

So, to get the basename of the current filename, you can now write:

constexpr const char *b = static_basename(__FILE__);

However, that just gives us a pointer halfway into the full string literal. In practice, this means the full string literal will be included in the link, even though only a part of it is referenced, which voids the space savings we're hoping for (confirmed on avr-gcc 4.9.2, but I do not expect newer compiler version to be smarter about this, since the linker is involved).

To solve that, we need to create a new char array variable that contains just the part of the string that we really need. As happens more often when I look into complex C++ problems, I came across a post by Andrzej Krzemieński, which shows a technique to concatenate two constexpr strings at compiletime (his blog has a lot of great posts on similar advanced C++ topics, a recommended read!). For this, he has a similar problem: He needs to define a new variable that contains the concatenation of two constexpr strings.

For this, he uses some smart tricks using parameter packs (variadic template arguments), which allows to declare an array and set its initial value using pointer references (e.g. char foo[] = {ptr[0], ptr[1], ...}). One caveat is that the length of the resulting string is part of its type, so must be specified using a template argument. In the concatenation case, this can be easily derived from the types of the strings to concat, so that gives nice and clean code.

In my case, the length of the resulting string depends on the contents of the string itself, which is more tricky. There is no way (that I'm aware of, suggestions are welcome!) to deduce a template variable based on the value of an non-template argument automatically. What you can do, is use constexpr functions to calculate the length of the resulting string, and explicitly pass that length as a template argument. Since you also need to pass the contents of the new string as a normal argument (since template parameters cannot be arbitrary pointer-to-strings, only addresses of variables with external linkage), this introduces a bit of duplication.

Applied to this example, this would look like this:

constexpr char *basename_ptr = static_basename(__FILE__);
constexpr auto basename = array_string<static_strlen(basename_ptr)>(basename_ptr); \

This uses the static_string library published along with the above blogpost. For this example to work, you will need some changes to the static_string class (to make it accept regular char* as well), see this pull request for the version I used.

The resulting basename variable is an array_string object, which contains just a char array containing the resulting string. You can use array indexing on it directly to access variables, implicitly convert to const char* or explicitly convert using basename.c_str().

So, this solves my requirement pretty neatly (saving a lot of flash space!). It would be even nicer if I did not need to repeat the basename_ptr above, or could move the duplication into a helper class or function, but that does not seem to be possible.

0 comments -:- permalink -:- 21:33
Automatically remotely attaching tmux and forwarding things

GnuPG logo

I recently upgraded my systems to Debian Stretch, which caused GnuPG to stop working within Mutt. I'm not exactly sure what was wrong, but I discovered that GnuPG version 2 changed quite some things and relies more heavily on the gpg-agent, and I discovered that recent SSH version can forward unix domain socket instead of just TCP sockets, which allows forwarding a gpg-agent connection over SSH.

Until now, I had my GPG private keys stored on my server, Tika, where my Mutt mail client also runs. However, storing private keys, even with a passphrase, on permanentely connected multi-user system never felt quite right. So this seemed like a good opportunity to set up proper forwarding for my gpg agent, and keep my private keys confined to my laptop.

I already had some small scripts in place to easily connect to my server through SSH, attach to the remote tmux session (or start it), set up some port forwards (in particular a reverse port forward for SSH so my mail client and IRC client could open links in my browser), and quickly reconnect when the connection fails. However, once annoyance was that when the connection fails, the server might not immediately notice, so reconnecting usually left me with failed port forwards (since the remote listening port was still taken by the old session). This seemed like a good occasion to fix that as wel.

The end result is a reasonably complex script, that is probably worth sharing here. The script can be found in my scripts git repository. On the server, it calls an attach script, but that's not much more than attaching to tmux, or starting a new session with some windows if no session is running yet.

The script is reasonably well-commented, including an introduction on what it can do, so I will not repeat that here.

For the GPG forwarding, I based upon this blogpost. There, they suggest configuring an extra-socket in gpg-agent.conf, but I've found that gpg-agent already created an extra socket (whose path I could query with gpgconf --list-dirs), so I didn't use that extra-socket configuration line. They also talk about setting StreamLocalBindUnlink to clean up a lingering socket when creating a new one, but that is already handled by my script instead.

Furthermore, to prevent a gpg-agent from being autostarted by gnupg serverside (in case the forwarding fails, or when I would connect without this script, etc.), I added no-autostart to ~/.gnupg/gpg.conf. I'm not running systemd user session on my server, but if you are you might need to disable or mask some ssh-agent sockets and/or services to prevent systemd from creating sockets for ssh-agent and starting it on-demand.

My next step is to let gpg-agent also be my ssh-agent (or perhaps just use plain ssh-agent) to enforce confirming each SSH authentication request. I'm currently using gnome-keyring / seahorse as my SSH agent, but that just silently approves everything, which doesn't really feel secure.

0 comments -:- permalink -:- 16:46
Running Ruby on Rails using Systemd socket activation

Ruby on Rails logo

On a small embedded system, I wanted to run a simple Rails application and have it automatically start up at system boot. The system is running systemd, so a systemd service file seemed appropriate to start the rails service.

Normally, when you run the ruby-on-rails standalone server, it binds on port 3000. Binding on port 80 normally requires root (or a special capability enabled for all of ruby), but I don't want to run the rails server as root. AFAIU, normal deployments using something like Nginx to open port 80 and let it forward requests to the rails server, but I wanted a minimal setup, with just the rails server.

An elegant way to binding port 80 without running as root is to use systemd's socket activation feature. Using socket activation, systemd (running as root) opens up a network port before starting the daemon. It then starts the daemon, which inherits the open network socket file descriptor, with some environment variables to indicate this. Apart from allowing privileged ports without root, this has other advantages such as on-demand starting, easier parallel startup and seamless restarts and upgrades (none of which is really important for my usecase, but it is still nice :-p).

See more ...

1 comment -:- permalink -:- 10:35
Showing 1 - 10 of 168 posts
Copyright by Matthijs Kooijman