Hi Gunnar,
1) Sorry to confuse you by mentioning VRay. My material hint was mentioned just to show how you can take advantage of alpha layer in XFrog's images without having to create new opacity textures. Feel free to get inspired from materials in 
Forest Library - 2D - Trees. 
Attaching screenshot showing example of material setup (in this case using 
Standard material as base for 
Forest Material) - please take note that for Mental Ray and billboards, we suggest to use exactly this combination. You can find the suggested settings for billboard scenes 
here.
2) Please try following steps. First right click on 
User's Libraries folder in 
Library Browser and choose 
Create Library (probably before you just created folder). Than right click on black area and select 
Create Item - 
Billboard. By right clicking on this newly created item you'll be able to run 
Import Billboards... wizard. Or by going to 
Properties... you can edit the item one by one.
3) Yes, you are right. Actually 
XFrog Catalog supports just 3D XFrog's packages. We have plans to add some more 2D trees soon.
Kind regards,