Houdini supports to split some simulations onto multiple clients.

Please see this help page:
https://www.sidefx.com/docs/houdini/dyno/distribute.html



There is a good video with example scenes as well:
https://www.sidefx.com/tutorials/h15-masterclass-distributed-simulations/


Usage


The Houdini distribution or slice tool automatically creates a HQueue submission node in /out.

Instead of entering all values again into a RR node, RR reads all values from this HQueue submission node instead.

You may connect a rrSubmitter node to the output of the HQueue submission node to submit the job.
Or you can right-click on the HQueue Simulation ROP to submit it.




RR reads these settings from the HQueue submission node:

  • Output Driver
  • Controls node
  • Partitioning type
  • Number of Slices/Slice Divisions



RR creates two jobs:

  1. The first job starts the simulation tracker which is requires for sharing information between the simulation slices.
    It updates the second job with its IP address to let the other clients know where to connect to.
    And it enables the second job that is waiting for this job to be started.

  2. The second job starts the simulation on multiple clients.
    Once all clients are done with the simulation, a post-render script sends a command to the simulation tracker to shut down.



Note:
If you want to reset the job, you have to abort and reset both jobs.
As the first job is required to start first to set the simulation tracker IP.




Simtracker bug fix for Houdini


There is a bug in Houdinis simtracker script.
If you use the http server to view stats about your simulation (See tutorial step 6), 

then it prints a long tracback ... every ... single ... time ... you open the page.
Which results in huge logfiles.



We have confirmed that the bug exists in Houdini v19, v19.5 and the latest version of v20 we got (At this time, v20 is not released for public yet). 
The bug was reported to SideFX Nov 2023.



You can download and install the quickfix yourself:

www.RoyalRender.de/download/231108_Houdini_Simtracker_Fix.zip




Example / Tutorial


  1. Please download the "Lesson Files" 
    https://www.sidefx.com/tutorials/h15-masterclass-distributed-simulations/


  2. Open the scene file explosion_initial.


  3. Either right-click on the /out/distributedsim ROP to submit it or 
    add an rrSubmitter ROP to /out and connect the distributedsim to it to submit it.

  4. At first, one rrClient takes the first job, which is the tracker server.
    This rrJob uses little CPU as it just coordinates the simulation.

  5. Then you require 4 free clients for the second job.
    If all 4 clients have taken the job, they will start to simulate.

  6. Optional:
    Please check the render log of the first "tracker server" job to get the http server address of the tracker.
    This server shows some information about the progress.
    Open any browser to see this information

  7. Once the clients are done with the simulation, the second job executes a post-script to shutdown tracker server.