Mike,
No, the materials doesn't take too much memory (at least if you don't use very huge textures). Also, if the material is applied to the closest trees, then it's already loaded in memory, so you don't gain nothing removing it for the farthest trees.
Do you get memory errors yet ? Perhaps the problem may be XShadow (if you are using it), because this plugin needs large segments of memory. If so, don't use shadow maps larger than 4096, and check the 'Disable exclusivity buffer' parameter.