I've had this a few times, oil will develop under the water pump housing. When this happened last time I replaced the water pump gasket and it came good (!) but there was no sign of oil in the water. Has anyone had this happen? I feel like it might be coming out of one of the bolt holes?

I'm going to order another water pump gasket, but any idea why this would be happening? Any help appreciated, thanks.

Nobody?

Which bike is this on?

I'm struggling to imagine how oil could make it's way out through the water pump as the pump is integral to the engine casing.
If it really is coming from there then the bolt hole would be a more likely candidate.
When you pull the bolts to change the gasket the bolt will show you if it has been allowing oil to pass. If it's dry then it isn't the problem, but as the oil isn't pressurized I'd be surprised if oil will make it's way passed the threads once tightened, which leads me to believe that your oil leak is coming from higher up the engine.

Before dong any of this I'd super clean and dry the engine, blow it with chalk dust, run the engine and watch it with a bright torch. The oil track should reveal itself.
If nothing shows then go for a ride and then inspect again.