ao2.it
Antonio Ospite, attacchi teoristici

Since Jan. 13^th, 2010 Xorg version 7.5 has landed to Debian unstable; one of the most notable additions to it was the XInput2 system, which incorporates the MPX efforts. So I hooked up a second USB mouse to my machine and started playing with it.

Pointless personal note: I have been training myself to use the mouse with the left hand, and this is how I am currently using it, but the instructions below still assume you are using a right handed setup.

And some nomenclature: a pointer is what (normally) moves on the screen when you move your mouse, a cursor is the visual appearance of a pointer.

I am referring to multi-pointer interaction here, which is a particular case of multi-touch. In a more general meaning, a multi-touch input system can detect surfaces too, like briefly explained by Peter Hutterer in his blog post Multi-touch? or multi-point?.

Setup

This is how you setup multiple pointers in recent Xorg (basically stolen from here):

• Plug you second mouse in and list your input devices:

  $ xinput list
  ⎡ Virtual core pointer                          id=2    [master pointer  (3)]
  ⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
  ⎜   ↳ "Logitech USB-PS/2 Optical Mouse"         id=10   [slave  pointer  (2)]
  ⎜   ↳ "Topro USB Mouse"                         id=12   [slave  pointer  (2)]
  ⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
      ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
      ↳ "Power Button"                            id=6    [slave  keyboard (3)]
      ↳ "Power Button"                            id=7    [slave  keyboard (3)]
      ↳ "Sleep Button"                            id=8    [slave  keyboard (3)]
      ↳ "DVB on-card IR receiver"                 id=9    [slave  keyboard (3)]
      ↳ "AT Translated Set 2 keyboard"            id=11   [slave  keyboard (3)]
  

  I am going to use the "Topro USB Mouse" with id=12 as the auxiliary (left) mouse.
• Create a new master input device and list the devices again:

  $ xinput create-master Auxiliary
  $ xinput list
  ⎡ Virtual core pointer                          id=2    [master pointer  (3)]
  ⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
  ⎜   ↳ "Logitech USB-PS/2 Optical Mouse"         id=10   [slave  pointer  (2)]
  ⎜   ↳ "Topro USB Mouse"                         id=12   [slave  pointer  (2)]
  ⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
      ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
      ↳ "Power Button"                            id=6    [slave  keyboard (3)]
      ↳ "Power Button"                            id=7    [slave  keyboard (3)]
      ↳ "Sleep Button"                            id=8    [slave  keyboard (3)]
      ↳ "DVB on-card IR receiver"                 id=9    [slave  keyboard (3)]
      ↳ "AT Translated Set 2 keyboard"            id=11   [slave  keyboard (3)]
  ⎡ Auxiliary pointer                             id=13   [master pointer  (14)]
  ⎜   ↳ Auxiliary XTEST pointer                   id=15   [slave  pointer  (13)]
  ⎣ Auxiliary keyboard                            id=14   [master keyboard (13)]
      ↳ Auxiliary XTEST keyboard                  id=16   [slave  keyboard (14)]
  

• Attach the mouse to it and check out how the device hierarchy changed:

  $ xinput reattach 12 "Auxiliary pointer"
  $ xinput list
  ⎡ Virtual core pointer                          id=2    [master pointer  (3)]
  ⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
  ⎜   ↳ "Logitech USB-PS/2 Optical Mouse"         id=10   [slave  pointer  (2)]
  ⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
      ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
      ↳ "Power Button"                            id=6    [slave  keyboard (3)]
      ↳ "Power Button"                            id=7    [slave  keyboard (3)]
      ↳ "Sleep Button"                            id=8    [slave  keyboard (3)]
      ↳ "DVB on-card IR receiver"                 id=9    [slave  keyboard (3)]
      ↳ "AT Translated Set 2 keyboard"            id=11   [slave  keyboard (3)]
  ⎡ Auxiliary pointer                             id=13   [master pointer  (14)]
  ⎜   ↳ "Topro USB Mouse"                         id=12   [slave  pointer  (13)]
  ⎜   ↳ Auxiliary XTEST pointer                   id=15   [slave  pointer  (13)]
  ⎣ Auxiliary keyboard                            id=14   [master keyboard (13)]
      ↳ Auxiliary XTEST keyboard                  id=16   [slave  keyboard (14)]
  

Now you have two independent pointers working already, but let's have some extra touches.

• Change the buttons mapping for your left mouse with id=12, in order to have the buttons orientation symmetric between the two mice.

  $ xinput set-button-map 12 3 2 1
  

• We also want pointers to be recognizable, we can change the cursors with xicursorset, a little tool I wrote for the purpose:

  $ sudo aptitude install git-core
  $ git clone git://git.ao2.it/xicursorset.git
  $ cd xicursorset
  $ sudo aptitude install libxi-dev libxcursor-dev
  $ make
  

  We set the cursor for the left hand pointer first (remember, xicursorset expects the master pointer id from xinput):

  $ ./xicursorset 13 right_ptr redglass
  

  And for the right hand pointer too optionally:

  $ ./xicursorset 2 left_ptr whiteglass
  

  Unfortunately I haven't found a way to change the whole cursor theme per-pointer yet, I asked on the xorg mailing list, and it looks like there's no way to do this transparently by only using Xcursor for now.

Tests and observations

Let's try a multi-pointer aware application first, from vinput:

$ git clone git://gitorious.org/vinput/vinput.git
$ cd vinput/demo
$ gcc -o demo-paint demo-paint.c `pkg-config --cflags --libs xi`
$ ./demo-paint

The result is pretty much like in this video about mpx-paint from Quentin Casasnovas.

Now, what is the situation for general use on a standard (GNOME) desktop?

• Gtk+ doesn't support MPX yet, so GIMP, Inkscape and the like (which I guess rely on Gtk+ input system) will only allow you to use one pointer at time when drawing, and you cannot assign different tools to different pointers either. However, there is some work undergoing to support multiple pointers in Gtk+3.
• Metacity is still being dumb here, I can't grab and move two windows using two pointers, let alone gestures. There is a very old guide about bringing multi-pointer to it, but the dedicated git branch is more than two years old.
• Anyhow, using two pointers on two distinct apps works quite well, I can move a window (Metacity) and drag and drop and icon on the desktop (Nautilus) at the same time, but this only gives a loose sensation of a multi-touch experience on the desktop.
• Qt seems to support multi-touch since version 4.6, here's a demo. I haven't tried myself, tho.
• Finally, an issue independent from MPX: Firefox keeps using the default cursors stubbornly ignoring the ones I set with xicursorset.

Multi-pointers setups are interesting from an HCI point of view even without full support: I tried using GIMP with two mice: with the left one I was changing tool or color, and with the right hand I was drawing, it wasn't a bad experience at all.

But having input from multiple pointers is also interesting from a Software Programming perspective, as this opens a whole new class of bugs in applications and toolkits, which assume that only one pointer exists; let me greet you with this weird one: it looks like menus are bound to pointers and not to windows, is this really the expected behavior?