Finally, after delaying and getting the parts, I finally have a programmer for the TPI interface of the Attiny5  that doesn't require a header on the PCB! Behold!

Here it is before stuffing it together:

Here it is with the probes, etc. before assembly:

So here's how it works simplified. There are 5 pins needed from the AVR Ice programmer to the Attiny5 TPI interface:

Clock --> Use a probe needle
Data --> Use a probe needle
Reset --> Use a probe Needle
GND --> Clip the GND clamp of one of the probes to the chassis of the device.
Target Voltage --> Hopefully overridable in AVR studio? If not I'll split open the USB cable and give it 5V. This way it's basically forced to 5V logic levels.

I've tested it and it is quite easy to stick up to three oscilloscope probes into the pads of the Attiny5 without having any other dedicated space (I've run out!).

Also, this is an excuse to make a box that I'll use in the future. I can't believe TPI wasn't supported by the piles of only-just-slightly-to-old programmers I had lying around...