The Spark, Bang, Buzz website has a lot of really interesting low cost electronics experiments involving homebrewing electronics components. Like making your own vacuum tubes, cathode ray tubes, and negative resistance devices. The last is very cool: using a heat treated piece of zinc, you can make a simple oscillator which was documented in the following youtube video. I'm gonna have to try this sometime.
The transmitter sounds a bit chirpy, but check out the schematic! It's pretty darned simple, it's hard not to accept a little chirp.