Itoo Software Forum

Author Topic: Frames take 3 min to start in backburner  (Read 4601 times)

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Frames take 3 min to start in backburner
« on: February 22, 2012, 05:07:51 PM »
Hello!

We are trying to render a scene using forest pack pro and HQ grass. We have a network that consits of 2 workstations with installed licenses and 2 render slaves only rendering via backburner. Whenever we render an image sequence (mental ray) using backburner, everything works fine on our workstations. The render slaves render frames but before each frame starts rendering it stays black for around 3 minutes.  I have no idea what it does during that time, I have tryed debugging messages but it didn´t give me any relevant info.
The render slaves only have the render slave installations of course

Has anyone ever experienced this?

/Kristian

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3315
    • iToo Software
Re: Frames take 3 min to start in backburner
« Reply #1 on: February 22, 2012, 08:01:43 PM »

It would be a network issue, probably the render nodes are taking a long time searching some remote file.

Are you using proxies or meshes for the Forest objects ? Are there maps assigned to UNC paths ?

In this case, i would try to render the grass in a separate scene, using only meshes and removing all the textures. If it works, you can add more elements until finding the problem.

Carlos Quintero
iToo Software

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #2 on: February 23, 2012, 09:04:44 AM »
Thanks for the reply. I´ll try to isolate the problematic object, using your method.

Some additional info. We use dropbox to sync files on all computers so we don´t use any UNC paths.
We only get this issue while using forest pack pro, all other scenes render flawles.
We have carefully checked all the paths in Asset tracking. As I mentioned before the problem only appears on the machines that have render slave installations of Forest pack pro.

/Kristian

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #3 on: February 29, 2012, 09:15:59 AM »
Allright this problem har turned out to be quite weird.

We have created a test scene, only containing one plae with a forest object that has som hq grass on it.
We have 5 rendering nodes in our network

2 Workstations with i7 980x cpus and 24G ram
1 Macbook pro with an i7 cpu and 8G ram
2 Supermicro servers with dual xeon 5645 cpus and 24G ram

The scene renders on all machines but on the supermicro servers we have the problem with the screen staying black and the render status bar saying rendering for a few minutes before the render starts (on each frame).

All paths are correct (bb won´t even start if links are broken)
We have tried disbaling materials and we have tried using mr proxies. We also tried installing a full license on one of the machines, all without any success.

The only thing that does help, is decreasing the number of Fp-objects. The freeze happens after the traslation so I have no idea wath it does but for some reason it has to process all the Fp trees before the actual render starts. This only happens on the supermicro servers. The scene starts to render instantly on all other machines.

Weird, huh? I´m willing to give you VPN-access through Hamachi if are willing to do some testing. I have out of ideas =(

/Kristian

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #4 on: February 29, 2012, 09:46:23 AM »
I even tried creating a batch render and ran it on one of the supermicro machines with the same result

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3315
    • iToo Software
Re: Frames take 3 min to start in backburner
« Reply #5 on: February 29, 2012, 11:26:06 AM »

Yes, it is a strange issue. I think the problem would be in Mental Ray itself, because once Forest converts the items to render instances (just after the translation), the plugin practically doesn't participate in the rendering process (except if Forest Material or Forest Color are used).

Probably the problem happens only with Forest because it generates thousands of render instances, stressing the render engine. Usually a typical Max scene contains much less items.

I would like to confirm it. Please, try this:

- Create a Forest test scene with roughly 50.000 items (you can see the number of items from General->Forest Stats).
- Use Forest Tools (Utilities Panel) to convert Forest to stand-alone objects.
- Remove all the Forest objects from the scene.
- Compare the render time between the different render nodes.

If the problem is reproducible or there are significative differences, the problem is not in Forest itself. In this case, should be needed to contact with Autodesk. If not, we will investigate further.

Carlos Quintero
iToo Software

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #6 on: February 29, 2012, 11:49:05 AM »
Ok I tried that this morning.

Render times with max instances is around 1 minute
Render times with fp-object is around 6 minutes

As I said before, this only goes for the supermicro servers. Even the macbook pro generates the trees faster using a fp-object.

/Kristian

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3315
    • iToo Software
Re: Frames take 3 min to start in backburner
« Reply #7 on: February 29, 2012, 11:57:43 AM »

Ok. What Max and Forest version do you use ?

Carlos Quintero
iToo Software

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #8 on: February 29, 2012, 12:00:21 PM »
Max release 14.0 (2012 Design 64bit SP2)

Forest Pro 3.8.0

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3315
    • iToo Software
Re: Frames take 3 min to start in backburner
« Reply #9 on: February 29, 2012, 12:29:10 PM »
I have modified the Mental Ray shader to add some logging info. Please, try this:

1) Copy the attached FShader3.dll to <maxdir>\mentalimages\shaders_autoload\mentalray\shaders, overwritting the existing one.
2) Test it with a batch render test. The log will be displayed in the CMD window (as shown in the screenshot).
3) Send us the results for one of the supermicro servers, and one of the workstations so we may compare them.

Carlos Quintero
iToo Software

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #10 on: February 29, 2012, 12:49:10 PM »
I guess this doesn´t make any sense

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3315
    • iToo Software
Re: Frames take 3 min to start in backburner
« Reply #11 on: February 29, 2012, 01:13:19 PM »

Yes, it failed in both but for different reasons.

The error of the workstation seems a problem with the rayhost file, probably a wrong IPV6 address. You may edit it at <maxdir>\mentalimages and remove all the entries, it is used only for the distributed rendering.

Strangely, the supermicro is not using the updated shader. Please, remove the FShader3.dll from it and try to render a Forest scene (which uses Mental Ray). It should not be able !.  If it renders, there is a duplicated shader or something.

Carlos Quintero
iToo Software

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #12 on: February 29, 2012, 01:36:22 PM »
Sorry!

I failed to follow your instructions. I backed upp the old FShader3 to FShader3_backup.dll and kept it. It seems mr was using it instead of the modified version.
Here is a new comparsion after removing the old shaders completely.

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3315
    • iToo Software
Re: Frames take 3 min to start in backburner
« Reply #13 on: February 29, 2012, 02:16:01 PM »

Ok, both nodes take similar time in the Forest shader (2.5 sec. vs 3.4 sec.). The delay seems to be in the Mental Ray core, which unfortunately we cannot debug.

Please, turn on the debug options shown in the attached screenshot (Customize->Preferences->Rendering). If you cannot run Max interactively, modify the 3dsmax.ini located at:

C:\Users\<username>\AppData\Local\Autodesk\3dsMaxDesign\2012 - 64bit\enu

adding these options to the [Renderer] section:

OpenOnError=1
ShowInformation=1
ShowProgress=1
LogDebug=1
LogFileON=1
LogFileAppend=0
LogFileNameTemplate=E:\tmp\p1.log

Use your own log file at LogFileNameTemplate, repeat the render in both computers and send us the logs.

Thanks.

Carlos Quintero
iToo Software

AdoreAdore

  • Newbie
  • *
  • Posts: 10
Re: Frames take 3 min to start in backburner
« Reply #14 on: February 29, 2012, 02:38:28 PM »
The log files