I finally got with the times last week and bought an MP3 player. I rarely get out a CD these days, having converted most of my music to ogg vorbis files ages ago; but it's worth spending a bit of money on a dedicated player just to save the hassle of setting up audio on every linux machine that I use. I decided on the iriver T20, mainly because of the ogg support.
Going by the 'net, my chances of having it work with Linux didn't look too good. This is because it is an MTP device, which is a proprietary protocol built into Windows Media player; no doubt part of Microsoft's strategy to take over the digital media sector. But the MTP is only the T20's default mode: once you update the firmware (you need Windows to do this), run the firmware updater again, and you get the option to switch to UMS (USB mass storage) mode. Once that is done, the device is just a USB memory dongle: it mounts with no problem on Linux, and you just copy over audio files. Once again I am impressed with Ubuntu, which mounts the device and detects it as a portable audio player without any prompting.
The device itself is very cool, clearly modelled on a USB memory stick: the USB connector is built-in (no cable needed), it charges over USB, and is barely larger than a memory stick.