In the days when I was heavily into cycle racing, my rest heart rate was 40 beats per minute. It's now close to 60. I trained like Obree - short, high-intensity bursts of activity rather than blasting it for hours on the road, so overall this meant a reduced number of heart beats.
Tour de France cyclists have a life expectency of 60 years, but that's not taking into account the massive reduction in drug use that's happening now - EPO was killing a several of them a year at one time.
One of the big dangers for retiring athletes is that they don't train down properly - they often stop too suddenly and their heart reduces in size fast. It's better to wind down over many years by cutting back on training rather than cutting it out altogether.
Former athletes can also have trouble with their weight as they're used to stuffing the food in to fuel themselves. Eddy Merckx got extremely fat after he retired, though he seems to be getting the weight off now.
Many of the people who live longest work hard for most of their life, so they're similar to athletes in terms of exercise except that they don't need to drive themselves to the limit the same way. The idea of starving yourself to live longer is similar to this, but it isn't necessary to do the starvation bit if you simply balance your food intake agains exercise properly.