Yes - there are a lot of factors in the final result. Nothing is ever straightforward when you dig deep.

It's all down to Newton's Second Law.

Acceleration = Force / Mass

The 'main' forces, at the start, are due to 1. gravity; this is proportional to the total mass of the rig plus crew and 2. the crew, pushing; fit athletes will have similar power/weight ratios so large crew will contribute more to the total mass and 'dilute' the effect of the bob. By the time the crew have stopped pushing and jumped on board, the speed is not great enough to bring aerodynamics into play.

All the other forces are due, much more, to aerodynamics and friction; They limit the terminal velocity and are not dependent on mass. I suggest that steering would have a similar effect for all masses because of Newton 2, again; a similar change in velocity would involve a similar loss of Kinetic Energy per kg.