Joel may not listen to our podcasts after we record them, but I certainly do. I have to in order to put together the show notes with hyperlinks, and the episode summary.
One way to speed up listening to the podcast is to use the fast playback option. In Windows Media Player, you can do this by right-clicking the play button and selecting "Fast Playback".
I'm not sure exactly how it works, but this speeds up the playback without turning us into chipmunks. So you can listen to a 60 minute show in around 40 minutes.
To see more detailed options, select View | Enhancements | Play Speed Settings:
The default "fast" setting is 1.4x, which I find quite listenable. You can accelerate playback by as much as 2x, but it becomes difficult to follow at that point.
I typically use WinAmp, myself, but I've switched to Windows Media Player for accelerated playback of our podcasts. I'm sure this type of accelerated playback option is available in other audio playback software for Mac and Linux, too -- if you know how, please note in the comments!