When you buy pop rivets you typically have to specify the material for both the rivet and the mandrel. The rivet is what goes into the hole, and the mandrel is what squashes the rivet in the hole and breaks off once the rivet is fully squashed.

If you specify both an aluminum rivet and an aluminum mandrel, there won't be anything in that rivet to rust. If you then use a bit of caulking over the rivet and the joint in the gutter, as Trader suggested, you've got a leak free connection.

Marson makes both good quality pop rivets and a top quality pop rivet tool for this kind of work. If you order a Marson pop rivet kit online you should be able to get the kit at a good price. And, most places listed under "Fasteners" in your yellow pages directory will sell rivets either by the box or individually.