murks about stuff

• The Tools
• Let's Paint!
• Let the ARSE do its Magick!

Well, it's not ARSE anymore, but ARSS, but I prefer the maiden name. This post is about a little workflow to create music or noise in an uncommon way, by painting it.

The Tools

• mypaint
• convert
• arss

First you need a paint tool of sorts. I like mypaint for its many different fancy brushes, but you can use whatever you want. In order to process our painting further we need it as a bitmap (.bmp). Mypaint can't save in this format, so you can use something like the 'convert' program from the imagemagick package, but you could also use gimp or something, but I consider this overkill for such a simple task.

The centerpiece of this setup is ARSS, The Analysis & Resynthesis Sound Spectrograph. It was formerly abbreviated ARSE (Analysis & Reconstruction Sound Engine), which, as mentioned before, I like way more. Anyway, this little piece of software can convert between images and sounds, and we'll use it for just that.

Let's Paint!

Well, there's not much to say here, I certainly can't tell you how or what to paint. I know nothing about painting, but I have a few hints that might help you:

• As far as I can tell, colour doesn't matter, so you don't need to bother with it. But brightness (or value, I don't know, I'm no graphics guy) does matter, so a greyscale picture makes perfect sense.
• Paint white on a black background. White makes sound, so black on white will be very noisy, you'd be painting a 'negative' so to speak.
• It analyses the picture from left to right (time) and from bottom to top (pitch)
• When saving in mypaint, use the 'PNG solid with background (*.png)' option, it's the only one convert can be converted to the correct bitmap format.

You don't necessarily need to paint youself, you could also use your holiday pictures or .. well, anything really. The ARSS homepage has a couple of good examples.

Hardly anything comes as a bitmap, so you'll need to convert it to a 24-bit .bmp. This line oughta do it:

convert -depth 24 source_image.png useable_by_arss_image.bmp

Here's my great painting (scaled down), can you see how much love went into it?

Let the ARSE do its Magick!

Now cast the following spell:

arss useable_by_arss_image.bmp amazing_sounds.wav -s --min-freq 55 -max 16000 --pps 100 -r 44100 -f 32

Don't be scared, you could just call arss without anything and it will prompt you for those parameters one by one. I prefer the commandline because of handy shell features such as history and tab completion. ARSS is reasonably well documented, arss --help, the documentation as well as the aforementioned examples should help you with the parameters. I was lazy here and just used pretty much the example line found in arss --help, but feel free to experiment. In my experience the -s (sine) option is a lot faster than -n (noise) and sounds similar most of the time.

Here's what my great painting from above sounds like, first using -s, then -n (concatenated and converted to vorbis).

mygreatpainting.ogg

ARSS has lots of options and they all can make the same picture sound quite different, so experiment, and have fun painting your sounds :)

Thanks go to gordonjcp this time who introduced me to ARSS several months ago using a painted amen-break.

• ALSA MIDI
• JACK MIDI
• The Two Combined
  • -Xraw
  • -Xseq
  • a2jmidid
• An example for JACK/ALSA MIDI bridging
• Late Addition: Where's my MIDI? A little basic troubleshooting guide

Today I learned a bit about ALSA and JACK MIDI and I think I should share my new knowledge. The whole topic is a bit confusing, so I'll try to bring some light into that jungle from a user point of view.

ALSA MIDI

Whenever you get midi data from outside your machine it goes through ALSA (there might also be OSS midi but I know nothing about OSS, not even if it supports midi, so I won't mention it again). So ALSA is used to talk to hardware, to shove MIDI data in and out of your machine. But ALSA can do more than that, it can also route the MIDI data to applications that talk ALSA MIDI. This is all just ALSA, no JACK involved. ALSA is for talking to hardware, it's important to understand that ALSA MIDI is totally independent of the JACK options you use.

JACK MIDI

Now let's have a look at JACK MIDI. I'm assuming you know what JACK is, so if you don't know it already, read up on it elsewhere. JACK MIDI can be used to have one JACK MIDI capable application send MIDI data to another one. This works very accurately, there's virtually no jitter, it's sample accurate. JACK MIDI is perfect for what it does, have one piece of MIDI software talk to another one, but JACK MIDI can't talk to hardware.

The Two Combined

Both do a good job at what they do, and they complement each other, but this part is a bit complicated, mainly because there are multiple ways to have them talk to each other. Here's a brief overview:

-Xraw

The raw MIDI option in JACK completely bypasses the ALSA sequencer but ALSA is still involved, it's just the ALSA sequencer which does time stamping and takes care of ALSA MIDI software that's bypassed. It only provides real hardware ports as JACK MIDI ports. This doesn't include MIDI devices connected through USB. Since all MIDI hardware I own is connected through USB I could never test it, but it's said to have very bad timing, which means a lot of jitter. USB hardware MIDI ports are exposed as JACK MIDI ports. Whether the timing is worse than with -Xseq is disputable. Thanks to Alex Stone for making me aware of this mistake.

-Xseq

The seq MIDI option has better timing than -Xraw and creates a JACK MIDI port for every ALSA MIDI port, no matter whether it's hardware or software. Here's an example:

  alsa_pcm:Midi-Through/midi_capture_1
  alsa_pcm:Midi-Through/midi_playback_1
  alsa_pcm:UA-25/midi_capture_1
  alsa_pcm:UA-25/midi_playback_1
  alsa_pcm:Portable-Grand/midi_capture_1
  alsa_pcm:Portable-Grand/midi_playback_1

The first two are software ports, there's no hardware involved. The other four ports are the ports of my hardware, connected through USB.

a2jmidid

This is not a JACK option but a bunch of little programs that do the same job, just more accurately than the other methods (less jitter). They can be found here: a2jmidid. The included programs are:

• a2jmidid
• a2jmidi_bridge
• j2amidi_bridge
• a2j
• a2j_control

Most important is a2jmidid. It dynamically creates JACK MIDI ports for all your ALSA MIDI ports. By default it does so only for software ports, but with the -e option it exposes the hardware ports too, so a2jmidid -e is equal to JACK started with -Xseq. The next two programs create a static pair of ports to translate between ALSA and JACK MIDI, so a2jmidi_bridge creates an input port for ALSA MIDI and a corresponding output port for JACK MIDI, with j2amidi_bridge it's the other way around. It's as accurate as a2jmidid, but just a single static port you can use to translate between the two.

The last two, in my opinion, don't offer anything new to the user, a2j is the same as a2jmidid but can only be controlled via DBUS, a2j_control is the corresponding CLI control program.

An example for JACK/ALSA MIDI bridging

On this screenshot of patchage you can see a2jmidid in action. JACK was started without MIDI backend (without -X (seq|raw)), then a2jmidid -e and j2amidi_bridge was run. The two blocks titled a2j are the JACK MIDI ports created by a2jmidid -e, the small one saying j2amidi_brigde by j2amidi_brigde. This setup was deliberately chosen to demonstrate what ALSA/JACK midi bridging can do for you. Without it it would be impossible to play the calf wavetable synthesizer with a hardware keyboard since it only supports JACK MIDI. On the other hand, it would have been impossible to play the phasex ALSA MIDI synthesizer using the epichord JACK MIDI sequencer. Using ALSA/JACK MIDI bridging this is no problem at all. Take Note!

There is one thing you should keep in mind when you bridge between ALSA and JACK MIDI. Due to how JACK works there's always a constant delay of exactly one JACK period added to the midi data, meaning: the data arrives always exactly one JACK period late. This may or may not matter, but you should be aware of it when you say, play an ALSA and a JACK MIDI synthesizer at the same time. I'm don't know whether this applies to the JACK MIDI -> ALSA MIDI process. Recommendation

Use JACK MIDI when you sequence, use ALSA MIDI when you play hardware.

Use JACK without a MIDI driver and in combination with a2jmidi -e. If you start JACK from qjackctl, select 'none' as MIDI driver. If you start it from CLI, omit the -X (seq|raw). My JACK line looks like this:

/usr/bin/jackd -R -M256 -u -dalsa -dhw:UA25 -r48000 -p256 -n3

As you can see, there's no -X and I don't see any JACK MIDI ports unless I run a2jmidid.

You may wonder about the -M256. This is a neat new feature Paul Davis added to the not-yet-released JACK1 0.119 and can at this time only be found in JACK1-svn or the experimental JACK branch "tschack" by Torben Hohn. It allows you to use a different period size for MIDI than for audio, which can be handy. You might want to use a bigger period size for MIDI when you use hundreds of MIDI channels, for example.

Rumor has it that a2jmidid might get integrated in JACK, which might make usage a little bit more convenient and spares us the separate installation, but we'll have to see whether that really happens.

That's all for now. Please feel free to correct me where I'm wrong. I hope I could make clear how the whole thing fits together and ultimately help someone to create some music. Thanks to Paul Davis for answering all my questions and thanks to you for reading.

Late Addition: Where's my MIDI? A little basic troubleshooting guide

By popular demand (by danboid ;->) here's a little troubleshooting section. So you have your keyboard connected to the computer, have it connected via a2jmidid to your arpeggiator, to your sequencer, to your favorite synth and yet, you don't hear a thing. So what's wrong? Where to start?

First you should check that everything is powered on and double-check that everything is connected correctly. If you connect your MIDI gear via USB, triple-check that the correct USB cable is plugged in.

Now that you've made sure that the hardware is connected you can check whether you get any MIDI data. You can do that without special software, simply by issuing:

cat /dev/midi{N}

{N} is the number of the MIDI device. If you have just one, then it's 1, if you have more.. you get the idea, try them all. If there's no output then there's no incoming data. The output in case there is a signal might surprise you, on my terminal emulation it looks like a bunch of questionmarks that get added one by one. Clearly it doesn't know what to make of it. But: If I play something on my keyboard I see some change in the gibberish, we have contact!

There are more MIDI devices which you can use the same way, /dev/dmmidi is the same as /dev/midi, both of OSS origin and for compatibility with old devices. /dev/snd/midiC0D0 is the ALSA node. All of them will show you the same gibberish.

Now wouldn't it be nice if we could see something useful instead? For that we need some specialised program. The simplest might be aseqdump, which can be found in alsa-utils, another popular choice is kmidimon, and for JACK MIDI there's at least the scope tool in the epichord sequencer, all of which will show you pretty much the same data. It might be a good idea to look at the ALSA MIDI data first, and I'll use aseqdump for this purpose. It's quite easy to use, just type aseqdump and hook it up to the right ALSA MIDI port using whichever way you want (aconnect, qjackctl, patchage, ...). Here's what it looks like when it's hooked up and some notes are played:

  Waiting for data at port 129:0. Press Ctrl+C to end.
  Source  Event                  Ch  Data
    0:1   Port subscribed            24:0 -> 129:0
   24:0   Active Sensing
   24:0   Active Sensing
   24:0   Active Sensing
   24:0   Active Sensing
   24:0   Note on                 0, note 62, velocity 41
   24:0   Note on                 0, note 50, velocity 34
   24:0   Control change          0, controller 64, value 48
   24:0   Control change          1, controller 64, value 48
   24:0   Active Sensing
   24:0   Control change          0, controller 64, value 127
   24:0   Control change          1, controller 64, value 127
   24:0   Active Sensing
   24:0   Note on                 0, note 53, velocity 38
   24:0   Note off                0, note 50
   24:0   Active Sensing
   24:0   Note on                 0, note 64, velocity 53
   24:0   Note on                 0, note 57, velocity 39
   24:0   Note off                0, note 53
   24:0   Note off                0, note 62
   24:0   Active Sensing

So if everything works fine so far and you still have issues you know at least that you get MIDI data into the computer and the trouble starts elsewhere.

Thanks to danboid for the suggestion to add this section and for showing me 'cat /dev/midi' as well as recommending kmidimon. Thanks to gellyfish for recommending aseqdump. As alway, I hope it helps someone :)

Hi,

my name is Philipp and I'm a bit of a geek.

I spend a lot of my time with Linux Audio stuff. Ever heard about it? Nope? Here's a place to read about it, among other stuff. I don't really expect anyone to read the stuff I write but I'll try to make it interesting on occasion by showcasing the one or other odd app and telling you about stuff you might not have heard about.

I'll mainly use this blog to write about stuff that has no place in a wiki, like the Arch Linux Wiki.

Well, have fun reading :)