You were right, the little gaps helped. This is the closest I can get before the edge gets confused. The only weird thing is that the edge still gets a little confused when further away from the camera, so I manually had to make bigger gaps per distance.
It's not perfect, maybe I offset the grass so the gaps are in the back of the steps and not in front. I can live with that. Thanks for your quick reply.
*edit* now I noticed the edge is straight, but some elements are still clipping through but they are pitch black. I suggest forest pack is supposed to hide these parts, but instead of hiding, it just renders them black. How would you solve this problem?
I will send you the scene with the isolated issue. You could post the solution here in case others have the same problem.
*edit2* So the solution was to go to Vray settings - Global switches and increase the Max transparency levels.
It's better described here:
https://forum.itoosoft.com/faq/some-key-notes-about-areas-gtboundary-checking-gtedge-mode/Why I'm getting some render artefacts while using the Edge mode? What should i do to avoid it?
In the case of very high density scatters (like dense grass), many overlapping objects could be generated and V-Ray may exceed its default transparency limit (which can cause render artifacts). To fix it:
a) Use lower density values, increasing Distribution Map->Density X/Y Size. (in many cases this also will improve the render performance, because there are less items to render)
b) Under V-Ray render settings, increase Global switches->Max transp. levels (default value is 50). Sometimes may be necessary to modify Secondary rays bias also.
« Last Edit: February 12, 2015, 01:04:05 PM by Paul Roberts »
For those who stumble upon this problem via Google: what you were probably was looking for is the option "work with transparency" from the VrayDirt Material which is also responsible for the black splotches you get in your rendering where the iToo grass is being masked.
"work with transparency" is still in the VrayDirt, but it's now called "ignore invisible objects".