That lag time is not normal. Here it doesn't take more than one second for a medium size scene, or less if the rollouts are initially closed (Max 2013, i7-980x, Win 7, Quadro 4000).
What Max version do you use ?
Nitrous or other graphics driver ?
Does it happen in all Display modes (Points-cloud, proxy...) ?
And resetting Max and creating a default Forest object ?