Why is Edge mode only supported in V-Ray?
Edge mode works at render time, removing elements which are outside of the scatter area. Therefore, it is required to implement a custom shader with certain capabilities, but actually only V-Ray allows this technique. The "trimmed part" is treated by the V-Ray render engine as a kind of "Matte object" internally (this way render engine ignores it for its calculations).
For other renderers, we have implemented an interface which can be used by the developers of these engines, to write their own shader. But since each render engine and company is different, when or if this feature will be finally implemented will depend of them.
What do I have to do to preview edge mode in the viewport?
Edge Mode does not preview in viewports by default - to improve viewport performance edge trimming is calculated only at render time. This means the effects of this mode will not be previewed in the viewports. However, previews can be enabled by holding down the left CTRL key and clicking on the Edge radio button. (Points-Cloud Viewport mode only)
AO pass doesn't work properly with the Edge mode?
When rendering Ambient Occlusion as an extra pass, to get the VRayDirt map used in VRayExtraTex Render Element to work properly, please mark the "work with transparency" option under VRayDirt Parameters.
Why I'm getting some render artefacts while using the Edge mode? What should i do to avoid it?
In the case of very high density scatters (like dense grass), many overlapping objects could be generated and V-Ray may exceed its default transparency limit (which can cause render artifacts). To fix it:
a) Use lower density values, increasing Distribution Map->Density X/Y Size. (in many cases this also will improve the render performance, because there are less items to render)
b) Under V-Ray render settings, increase Global switches->Max transp. levels (default value is 50). Sometimes may be necessary to modify Secondary rays bias also.