I'd recommend getting to know some local ham radio operators and visit
hamfests and computer sidewalk sales. Plenty of old junk for sale, along
with lots of advice for free. Great for parts. Also plenty of electronics
hobby books, manuals, and textbooks for low prices.

When it comes to multimeters, I'd recommend getting one brand-new if at all
possible. Digital multimeters can be damaged any number of ways without
being immediately obvious, and if you're just getting started, it's best to
use equipment that you can believe what it's telling you. Someone selling
an old one at a sidewalk sale (or on eBay, for that matter) may or may not
have forgotten that one of the settings got blown out long ago.

A very handy piece of equipment is a solderless breadboard. There are
different kinds, but I recommend one that looks like a small console. It
comes with dual-polarity power supply, and even a function generator. (A
solderless breadboard is a device that lets you plug in components like a
jigsaw puzzle and push in wires to connect them together, without having to
solder anything. Great for experimenting.)

Also check your local library. Look in the 621 section for
electronics-related books.

Everybody's got broken electronic gizmos just lying around. They probably
can't bear to just throw them away, but if you ask around, they may be
willing to let you look at it, or just flat-out have it.

I don't see this mentioned often, but it's a good idea: keep a notebook.
Write down things that you've worked on, circuits you've put together and
tried out, and ideas you want to try. You'll be amazed how handy that
notebook will be in the future.