If the network is working well, probably the problem is not related with the network. But at this point, it's good to test all possibilities.
In fact, i would suggest two tests to compare loading times:
1) Copying all models locally, and reassigning them in the Asset Tracking.
2) Removing all Forest objects from the scene, but keeping the models in the scene.
I can imagine several causes for the high loading times:
a) Loading time of the XRef models (this can be checked with test 2). But really from the Forest side, there is not difference between XRef or Merged models.
Both are stored as standard objects in the "forest_templates" layer. In one case as XRef objects, and other as standard meshes.
b) The first time that Forest uses a model, it is analyzed to generate the point-cloud, hierarchy of elements for Forest Color and other data. This process may take some seconds for a heavy model, and may be significative when using tons of different objects. But this step is identical for merged or XRef'd models, because it's done in memory once the object is loaded.
c) Other problems in the scene, not directly related with Forest.
Since we cannot debug directly the loading process in your configuration, these tests may help to isolate the problem.