Hi,
That's a limitation of the "From Layers" feature implementation - you can use the Display > Update button or directly add these reference points "directly" to the Forest Set (this way, it'll be all updated automatically).
I hope that helps.
Best regards,