How low can they go, how small can they get?
The MiCorder MP3 Recorder from Ölens Technology goes for $79.99 (MSRP).
About the same size as a pack of cigarettes and maybe a little thinner, it records via a 1/8-inch input (with 1/8-inch – 1/8-inch cable included) or an onboard mic. A well-placed adapter and a handheld mic could be used.
Recording format is MP3 only. The MiCorder uses SD cards as its medium.
It has jog/shuttle controls and a backlit LCD screen for navigation. It ships with earbuds. A rechargeable Li-Ion battery is expected to last more than 10 hours. A USB port allows transfer to computer of music files and recharging.