On USB latency

Every so often someone mentions the dreaded USB latency. MIDI is MIDI, and USB is USB, do we need to mix both, and can that work reliably?

So let’s say we want to provide a Raspberry Pi with some USB MIDI connectivity, and run a software synth to produce sound. Since you would need some kind of a MIDI-to-USB conversion box in between, that must inherently produce some kind of sluggishness. Right?

Even worse, let’s run a sequencer on the Raspberry Pi, so that it must receive MIDI notes and knob twists played on a keyboard, and as a response send them to some other synthesizer to produce sound. That must be twice as bad, as the MIDI notes go once through the USB conversion to the Pi, and a second time back.

So how bad is that then? We can measure it!

The Arduino works fine as a simple MIDI device. Its serial ports can be easily configured for the 31250 bps bitrate needed by MIDI, and you only need a few extra components for interfacing with other MIDI devices. It’s not too difficult to program the Arduino to first send some MIDI commands to the Pi, and then make it wait for a reply, and at the same time measure the time that transaction takes to finish.

It’s an Arduino Mega 2560 under the Ethernet shield there! This is a good moment to reuse those MIDI boards from before… Here is the schematic of the MIDI interface electronics. Click for a larger picture.

Use only 4N28 optocouplers or similar with this circuit! 6N138 and PC-900 require a different schematic.

The MIDI0_RXD and MIDI0_TXD wires can be directly connected to the Arduino. The separate driver IC (74AC04 hex inverter suggested here) is not absolutely necessary, so those parts of the schematic are faded out. VCC for this schematic is 3.3V, and for 5V from a standard Arduino you only need to replace the two 56 ohm resistors with 220 ohm instead. If you look carefully, you can see the resistor values in the picture below.

The interface is very simple, and the MIDI interface boards I made earlier look empty with just the mandatory components populated… I soldered wires from the Arduino UART directly to the bottom of the board, on the pins of the through-hole components.

The program for the Arduino goes something like this. I left the timer functions out, get the whole source file instead!

bool measuring = false;

void setup() {
  // initialize UARTs
  
  // Arduino serial monitor
  Serial.begin(115200);
  
  // MIDI in/out
  Serial1.begin(31250);
}

void loop() {
  if (!measuring) {
    // not measuring yet, go!
    timer_start();
    measuring = true;
    
    // start sending a MIDI CLOCK byte
    char c = 0xF8;
    Serial1.write(c);
  }
  else {
    if (timer_overflow()) {
      // no reply before timer overflowed
      Serial.println("timeout");
      measuring = false;
    }
  }
}

void printTime(long time)
{
  // convert timer count to 0.1 milliseconds
  time = time * 10000 / timer_ticks_per_sec;
  Serial.print(time / 10);
  Serial.print(".");
  Serial.print(time % 10);
  Serial.println("ms");
}

void serialEvent1() {
  // if measurement was started, finish and report time
  if (measuring) {
    long endtime = timer_count();
    printTime(endtime);
    measuring = false;
  }
  
  // flush all input
  while (Serial1.available()) {
    Serial1.read();
  }
}

In short, the program tries to send one-byte MIDI clock commands through the MIDI out, waits for a reply, and prints the time in milliseconds from starting of the output to the end of the reception of the reply. It doesn’t care about whether it gets the correct data back… You’re welcome to add that part yourself for practice. :-)

First test should always be to connect the latency tester to itself in loopback, with a single MIDI cable, from output to input. This way I measured about 0.4 milliseconds round-trip time, sometimes 0.3. The MIDI bitrate is 31250 bits per second, and a single transmitted byte includes one start bit and one stop bit, 10 bits total. A single MIDI output port can therefore transmit 31250/10 = 3125 bytes per second, or 1/3125 = 0.0032 seconds = 3.2 milliseconds per byte. Hey, that matches what we measured, the program always sends just one byte at a time! Looking good. And unplugging the cable gives the message “timeout” as expected.

The Edirol MIDI interface I used here has a OUT/THRU switch for each of its inputs, allowing direct output of whatever it received on the input. With the switch engaged, I also measured 0.4 ms round-trip time. So the switch probably just connects the input port directly electrically to the output port.

Now it gets more interesting. With the USB MIDI interface plugged in and configured for both input and output, I start JACK on the Pi, using just the internal audio device since I don’t really care about the audio side for this experiment. Then in Patchage I just hooked the MIDI input directly to the MIDI output, in a loopback configuration. The MIDI commands from the Arduino will be received by the Linux software, the Jack2 daemon specifically, and sent back out as quickly as possible.

The result: 1.4 ms round-trip time! Since it takes 0.32 ms to transmit a byte, and that must happen in order (first in, then out) the overhead of the data passing through USB drivers, the Linux kernel, and the userspace software is 1.4 – 2*0.32 = 0.76 milliseconds. Not bad! An ideal hardware sequencer could only react 0.76 milliseconds faster than the Pi. There is some variance in the results, sometimes it takes 2.0 to 2.3 ms to get a reply. Maybe it would be a good idea to add some kind of average measurement to the program.

The software running on the Pi would of course have to process the input it receives, and prepare the output, but an optimized program should not add much overhead on top of these measurements. For reference, I got the same 1.4 ms minimum latency with the exact same setup on my much more powerful desktop Linux PC, but overall the measured time varied much more, often peaking around 5 ms probably due to Firefox and other programs running on the background.

But any Ethernet traffic or disk I/O immediately degrades the performance on the Raspberry Pi. Connecting to the Pi remotely with SSH and running some simple “ls” and “cd” commands immediately causes the round-trip times to jump up to 10 ms. That’s bad! A real-time kernel might help with the disk I/O, that could be worth investigating further. This page on the Linux-Sound wiki gives another clue: the Ethernet controller is connected to the CPU core over USB. Any Ethernet traffic may steal some bandwidth from all other USB communication. In the end it’s probably easier to just disable all unnecessary peripherals, as instructed on the Linux-Sound wiki.

Still the result is very reassuring: if the Pi is able to respond to MIDI in less than 2 milliseconds, it should be perfectly capable of using standard USB MIDI devices while running some sequencer software.

Next time I’ll measure the round-trip time of the RPC for comparison…