How to get if child process completed with parallel processing

[supernaro]

Hi TM1 Experts,

We are running month-wise processes for fcst periods using RunTI in a master process. Many times master completes but some internal processes are still running, but the subsequent master processes of the chore triggers. We have been managing through Sleep, but due to data volume inconsistencies, there is a lot of overlapping happening leading to data mismatch.

Can anyone help what's the best practice to handle such a scenario?
We tried logging the status of the child process into a control cube and created a waiting loop of a few mins in Epilog but the loop is not able to identify if it is committed.

Let me know if you need any more details.

[ascheevel]

There was recently a thread on this and it's come up other times as well that you can find with a bit of searching.

File semaphores is probably the safest, but I sometimes use a monitoring cube with "called" and "completed" columns where the calling process updates the called column and the child process updates the completed column. You can also use a Thread_Nbr dim in the cube if you need to limit the number of active parallel threads via Synchronized. The calling process will sleep and periodically check the completed column until it equals the called column. The reason you're not seeing the completed column update from the calling process is because the data commit model that TI is working under is from the moment of execution. Any data updates made outside of the execution thread (RunProcess calls are separate, independent threads) are not visible to an already running process. The trick to get the calling process to see recent data commits is to call CubeSaveData on a cube from the calling process with each sleep/check iteration. It can be any cube and I often have a dummy cube with no data that I call CubeSaveData on to ensure no locking issues and fast response.

[ardi]

In my experience, the easiest way is to use RushTI ( a Cubewise python/TM1Py utility). I have used all kind of different approaches in the past, and I always had issues. Even with Flag-files approach, you end up having files being locked from OS and other type of issues.

RushTI is easy to use and it provides all the flexibility you need to manage the parallel execution, you can even implement process dependencies easily.