Hi Pierre,
First, there's no need to treat the vertical and horizontal areas separately. You can use the to Spline alignment mode, where the X-axis aligns with the first spline segment. If needed, you can adjust the alignment using the Make First vertex option or by reversing the spline.
You can use the Cap Holes feature, but first, you should cap the bottom of the object or add thickness to it - for example, using the Shell modifier.
I've attached a revised version of your scene. I hope this helps.
Best regards,