If you are just seeing temps at stock speeds/voltages, then running for like an hour or two is good.
If you are overclocking and testing your new clock speeds, voltages, and temps, then each step you take you should be testing like 12+hrs.
If you are undervolting your cpu, then its pretty much the same as above, id recommend 12+hrs per voltage decrease step.
You need to make sure its stable at those settings, so you need to make sure that your computer can last through 12+ hours of usage.
And for your last test (when you are finished with overclocking/undervolting) id suggest that one be your longest test, and id say 24hours for it to be 100% stable.
overclocking/undervolting are NOT just causal things you do. They are very serious, and should be taken as such (though undervolting cant really cause any physical harm to your computer like overclocking can).
They need to be completely stable, and if overclocking you need to make sure your temps are low.
And even though after like an hour you dont really see much of a change in cpu temps, but think if it went up only 0.5C/hour, after 23 more hours of testing, that 11.5C hotter.
Also want to mention, the cpu temps are not the only important thing to worry about, but your VRM temps aswell.
Also think while gaming, your graphics card will also be under load, and be running hot, which will add to the temperature of the case, which will add to the temperature of the cpu and VRM.
EDIT: well overclocking is not a magic way to get free performance. I bet if i actually explained EVERYTHING there is to do with overclocking, you probably wouldnt even want to overclock (just like any other person who actually knows what overclocking is).
Also i was talking about undervolting, not underclocking (there is quite a bit of difference :P)
Undervolting is lowering the voltage under the stock voltage. That will lower the cpu temps, the vrm temps, and power usage.
This is because the stock voltage is always set much higher then it needs to be (and the fact that each cpu is different. im talking about the same model cpus, like every single phenom II x4 945 is not equal. some may be able to be dropped 10 voltage levels and be stable, while another may be unstable after 1 drop)
This is due to the quality of the silicon used and how it reacted after being bonded. (which works both ways, its the same for what levels of overclocking can be obtained are different for every single cpu aswell)
But even though undervolting is beneficial, it still shouldnt be attempted unless you know what you are doing, and are willing to spend the time. There is no, oo it ran fine for 10mins, so its good...
It took me a weeks time to get my laptop's undervolt perfect. (though i cheated, and just used a shortcut, which can work for undervolting but not overclocking)