For this work-flow I would suggest you not to use the Limit to visibility feature at least for the Light Cache phase. Even with Use camera path option enabled, Forest object(s) will not be rebuild - to have the correct GI you should have all scattered items visible along the whole path.
Instead for the Irradiance map GI calculation, Forest object(s) will be rebuild correctly for each "prerendered" frame. So, the precision of the final GI will depend on total number of "precalculated frames" - if it's calculated for each frame or for every Nth frame (in that case I would suggest to raise the Expand value depending of the camera movement).
Well, the proper adjustment will vary for every scene as it might require a slightly different settings. If you are not sure, that your actual Expand, Far clipping plane, Back Offset values are set correctly, you always may disable the Limit to visibility to easily avoid any possible issues.
Hope that helps.