Itoo Software Forum

Author Topic: How to optimization my scene ? Crashes easily during render now.  (Read 897 times)

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Hello everyone here,
How to optimization my scene ? My complex scene contained about 10 forest objects. Most of the forest objects were made up of 3~4 animated clumped forest objects.
It took about 1.5 min in translate vertex stage before render stage.
This is most of the process of making these 10 large forests.

12 animated grass models(bend mod or Xmesh sequences) ---> 4 clumps of forest objects (4 random samples) ---> 1 large forest object(follow animation).

I found MAX was easily crashed when rendering the scene in Deadline 5.2.
I found if I turn off all the animation of the forest objects. MAX will become much more stable.
I gonna crazy now. I'm not sure the crashing were caused by Forest Pro pack.


Another very important question is,
Does every render node need an independent graphic card for rendering the complex animated forest objects ? There is no  independent video graphic card in most of my render nodes.  They have only low-end integrated video cards.


Vray2.40,FP441,Railclone 2.50,MAX2011 64bit with lastest update pack, Deadline 5.2,  5 computers (dual X5650|32GB ram|500GB harddisk|NV450gts display card and 4  integrated video cards in render nodes.)

I hope you can give me some suggestions.
« Last Edit: March 18, 2016, 09:58:28 PM by cc8boy »
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #1 on: March 18, 2016, 11:25:31 PM »
We found a serious problem after long time render. The big trees Xmesh sequences in forest objects kept frickering in DiffuseFilter pass because of the Forest Color map of the leaves.
Settings of The forest color map:
1. get ID from element | only one default map with color_correction map was used in it | blend in color mode.
2. "Overide" was not checked. The variation was 3/4 item and 1/4 Element.
3. "Get color from Gradent" was checked in material tab of the forest object. Random strength was 20% to 80%.

update: It seems the tint of the big trees was different from that in different render nodes.
The render OPENEXR sequence become unusable because of the color flicker of the leaves. What's wrong with my setting in forest color map?
« Last Edit: March 18, 2016, 11:38:40 PM by cc8boy »
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3319
    • iToo Software
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #2 on: March 19, 2016, 09:03:40 AM »
Hi, it's difficult to know what is causing the problem, because there are many components involved. Futhermore, many of them are deprecated versions (Max 2011, Deadline 5, even V-Ray 2), so probably you are missing multiple fixes from the latest updates.

If you can get a minidump from the crashes, we would have more information. If not, i'm afraid there is not much we can do.

Animated FP objects require more resources and processing time than static models, because the plugin must request several samples for each one of the objects. Although this should not cause crashes.
Processing time depends both of the underlying object and the mesh size. According to our tests, the faster format is V-Ray proxy (although it has some limitations, as not support for Element mode in Forest Color).

Quote
Does every render node need an independent graphic card for rendering the complex animated forest objects ? There is no  independent video graphic card in most of my render nodes.  They have only low-end integrated video cards.

No, graphics card is not used in any way for rendering. Only CPU and RAM.

Quote
We found a serious problem after long time render. The big trees Xmesh sequences in forest objects kept frickering in DiffuseFilter pass because of the Forest Color map of the leaves.

We should do some tests, but i suspect problem is caused by XMesh. Forest Color requires that the geometry structure of the tree don't change along all frames: i mean, that the number and order of vertices and faces be identical (although they can be moved or transformed). I remember that XMesh uses a very particular format to store the animation, so i don't know if it meets the above rule.

What XMesh version do you use ?

Carlos Quintero
iToo Software

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #3 on: March 19, 2016, 10:40:07 AM »
------------------------------------------------------------------------------------------------------------------------------------------------------------
1. I summit three missions to Deadline, one for caculating Lightcache(no crashing), one for caculating irradiance map(no crashing),one for rendering image (Lots of crashing).
2. It crashes at the very start of rendering one frame(The Vray buffer frame became grey out and it told me "xxxx has stopped working xxxx close" when the render buckect squares appear at the starting position on the screen.)

How can I find the minidump in rendernode ? Will the minidump be overwrite again when this rendernode error again during DeadLine render ? I will try to find the correct minidump files.


------------------------------------------------------------------------------------------------------------------------------------------------------------
1.Xmeshloader 1.3.3.58467
2.trees were cached by Xmeshsaver 1.1.3 during making "A villa" video.
3.This forest were rendered well in MAX2012 64bit sp2 PU08 FP441 Vray2.40. But I must finish this project in MAX2011 because of leaking some new version plugins.
4.I am sure the vertex and face number never change during animation.

Another question, are you sure that the color of the varied leaves (not items|not the color of whole tree) won't flicker again if I turn all the xmeshloader into EditableMesh+LoopPointcache in Xref Object ?
We don't have much time now. I hope you can answer soon.

Xmesh, Xref Scene or Pointcache in main scene, which way of caching object will cause the minimum consumption in rendernodes ? Don't consider Vrayproxies because of the varied leaves.
There are many flowers and grass animation created by just one bend modifier with animated value with noise_controller. Should I cached them (plants with bend mod) before final render ?
« Last Edit: March 19, 2016, 10:55:37 AM by cc8boy »
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #4 on: March 19, 2016, 02:30:14 PM »
UPDATE:
Does i tseems that the crashing problem may caused by using the Forest color map twice ?  I'm not sure.
Here's my material setup,

Bitmaptexture ---> Color_correction--->Forest_color(override not checked,for small area variation) ---> Forest_color(override not checked,for Large area (Mapped) variation) --->Vraymtl ---> multimaterial

Did this cause the crashing or flickering?
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #5 on: March 20, 2016, 02:38:24 PM »
UPDATE:
I converted all my xmesh sequences to PointCache MC sequences. The Frequent crashes were gone. The flickering were gone. But the system memory was consumed terribly if I use tranditional Xref Object with max mesh and PointCache MC.

"MAX 2011 64bit+Deadline+ForestPro 4.41+Vray2.40+Xmeshloader1.3.3" = CRASH Frequently?
If I use tranditional mesh with PointCache MC, how to keep the used memory as low as possible? I have no idea now.

How to caculate the used memory while a forest object in random sample animation mode ?
If whole size of the Pointcache MC sequence of the big tree is 3GB. If I set the random sample count to 4, will this tree use 4*3GB=12GB of system memory ?
I have 10 kind of trees with 4 samples, they will use 120GB of system memory ?!
« Last Edit: March 20, 2016, 02:55:50 PM by cc8boy »
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3319
    • iToo Software
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #6 on: March 21, 2016, 09:56:46 AM »
There is not a formula to calculate it, but Forest uses four memory blocks for each animation sample: geometry + forest color + point cloud + vray. Only are stored the samples used in the rendered frame (except frame 0 which never is deleted). This rule applies to all types of objects: animated meshes, procedural objects, proxies, pointcache, etc.

So, really Forest doesn't keep in memory all animation samples, but only the strictly necessary to render the current frame.
But take in account this is the memory used by Forest... Pointcache uses its own memory blocks. I don't know how it works internally, but perhaps is loading all samples at once, causing that large difference.

I will check if there is some mechanism in recent V-Ray releases to get the mesh of a VRayProxy. That would solve the limitation to use Forest Color.

Meanwhile, i'm afraid your problem has not a simple solution. May be creating several point-cache files, and rendering the animation by segments ? (a lot of manual work, i know).

Carlos Quintero
iToo Software

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #7 on: March 21, 2016, 11:00:27 AM »
All the trees are same as before.  It consumed about 15.5GB ram when using Xmesh sequences. But it consumed almost all the 32GB ram when using Pointcache MC sequences. I wonna how well does the Forest support Xmesh in 4.41 version now ?
Xmesh can takes very low memory to render complex mesh. I hope Forest Pro Pack must support it well. I think it is better than Vrayproxies. Because the Vrayproxy file are very very large !
I have no idea except abandon the varied leaves color without Xmesh sequences to use Vrayproxies.

And thanks for your reply.
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3319
    • iToo Software
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #8 on: March 21, 2016, 01:19:25 PM »
Unfortunately we cannot fix the problem with XMesh from our side. We reported this issue to Thinkbox sometime ago, but no idea if it can be solved or not.
XMesh uses a very particular format to store the animation, and it seems incompatible with the standard Max functions to get the render mesh.
Carlos Quintero
iToo Software

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #9 on: March 21, 2016, 01:45:22 PM »
I think there's no hope anymore in using Xmesh for complex big trees. There's no minidump mentioned when the application crashed , see my screenshot.
(my rendernodes cannot connect to Internet)
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #10 on: March 21, 2016, 08:35:34 PM »
What the.... I researched about this day and night for three days. I found some very strange things.

The big trees of the forest is in custom edit mode and random sample animation mode ( 68 offset frames 4 sample counts). The big trees are Xmesh sequences.

---------------------------------------------attempt one-------------------------------------------------------
1. I tried to disable some trees in geometry list. The crashes were gone ! Enabled those trees, the crashes comed again !
2. I tried to disable the opacity map of the trees disabled before, the crashes reduced but still happened !
3. I tried to disable the opacity and forest color map(just add the raw bitmap) in diffuse map, the crashes seldom happened .
4. I tried to disable the opacity and diffuse map at all, the crashes disappeared !

Perhaps there were some problem in these material of these trees. BUT^

---------------------------------------------attempt two-------------------------------------------------------
1. Do not render the old big tree forest at all. Recreate a brand new forest and put in the old Xmesh sequences without disabling any maps. Set the parameter of new forest object manually again.
Probability of crashes become very small again.

I have no clue what was going on now.
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.

iToo

  • Administrator
  • Hero Member
  • *****
  • Posts: 3319
    • iToo Software
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #11 on: March 22, 2016, 08:58:58 AM »
It has sense that Forest Color has some impact in the crashes with XMesh, because as explained above, FC requires that the geometry doesn't change along the animation (same number and order of vertices and faces).
If XMesh is generating different geometry (not confirmed, but i suspect it), that would cause a lot of unexpected problems (invalid pointers, etc.).

At this point, i encourage not using XMesh with Forest, because we cannot guarantee it works.
Carlos Quintero
iToo Software

cc8boy

  • Sr. Member
  • ****
  • Posts: 276
  • I'll do my best.Welcome to www.c7design.cn .
    • C7design
Re: How to optimization my scene ? Crashes easily during render now.
« Reply #12 on: March 22, 2016, 11:47:23 AM »
But what can I use with Forest and it cost low memory as well?
Pointcache MC sequences use extremly high memory. Vray proxies is lack of varied leaves. ABC format is not support in MAX2011.
It seems I have no choise now.  :'(
Or as you said, rendering the animation by segments...
We are working hard in virtual effects in 3DSMAX. Welcome to our website in China.