"What input resistance can your voltmeter have before your measurement goes wrong?"

The unhelpful answer is that the resistance needs to be infinite.

There are a couple of ways round this. The traditional aproach is to use a potentiometer where (in principle) no current flows from the cell.

A more sensible answer is to use a high resistance voltmeter, say 10M, measure the voltage then put a 10M resistor in parallel with the meter and measure the voltage again.

The second voltage will be a little lower than the first From the difference you can measure the effective source resistance and from that you can calculate the voltage with no load (to a good aproximation it's the initial voltage + the difference between the 2 measured voltages).

BTW, since this thread is in "Chemistry", did anyone expect a biologist's answer?