TM1RunTI / Runprocess issues

[prameson]

Hi All

We recently upgraded to PA (we are now on 2.08) and to Security Mode 3 (from 2) to simplify user login are having some issues with some processes using TM1RunTI (which worked fine in the previous versions).  We created an admin user account with the name of the Server service account in order to get the RunTis to work.

The processes are to create static versions of a planning model and we use TM1RunTI to copy various years in parallel to speed up the process.

When run as is they now seem to either randomly just run for one year or run for all years but not in parallel.

I have looked to change the processes to use the RunProcess function in the new version.  I can get this to work for the first time run the process, but then in the second run I get errors - "Error: Prolog procedure line (0): Unable to open data source" and the cube name.  The third time I run the process I see "Execution rolling back due to lock exception" errors in the server log but the copy processes seem to re-run and the copy does happen correctly

Here is the logic I have:

Control Process with parameters (Years, Version from/to)
Prolog - uses ExecuteProcess to call a create view process for each year specified  - a source view and a zero out view are created (using ViewConstruct) and the ZeroOut view is zeroed out.

Epilog - uses RunProcess to call a DataLoad process for each year and do the copy using the previously created views (and destroy the views in the Epilog (the source and the zeroout for that year).


Any help or advice would be greatly appreciated - I'm not sure what the best practice is now for parallel processing - should I be looking to use Batch processing / maybe looking to use something like RushTI or is there just some logic issue with my code?

Thanks
Peter

[tomok]

It sounds like you are dealing with lock contentions. You are creating an all encompassing view, for use in the ViewZeroOut, and then calling sub-processes that load into portions of that same area, while the original process is still active. I cannot guarantee that is the problem but it looks very suspicious to me. If it were me I would do the create view, zero it out, create the source view, assign it as the data source and then do the data load all in the sub-processes. I would also set the temporary parameter of the ViewCreate to 1 so that they are destroyed at the end of each run of the sub-process. Probably a little less efficient than your current way but you shouldn't run into any locking issues.

[prameson]

Thanks Tom

I'm creating separate zero out views for each year and then zeroing out in the prolog of the control process. The epilog then calls the DataLoad process.

If I create the views as temporary then surely they won't exist when I call the load processes using RunProcess?

[tomok]

I'm creating separate zero out views for each year and then zeroing out in the prolog of the control process. The epilog then calls the DataLoad process.

You are creating a view to zero out each year which puts a write lock on that view. Now, before that process has a chance to finish and thus release the lock, you are calling a sub-process to load to the same area that the original process has a write lock for. Do you understand?

If I create the views as temporary then surely they won't exist when I call the load processes using RunProcess?

What I am saying is to do the viewzeroout AND the load all in the subprocess. I am not guaranteeing that this is the issue, especially if the sub-process is called by RunProcess, since that is different than ExecuteProcess, but I would not do what your are doing on principle because of the potential for write lock issues.

[prameson]

You are creating a view to zero out each year which puts a write lock on that view. Now, before that process has a chance to finish and thus release the lock, you are calling a sub-process to load to the same area that the original process has a write lock for. Do you understand?

What I am saying is to do the viewzeroout AND the load all in the subprocess. I am not guaranteeing that this is the issue, especially if the sub-process is called by RunProcess, since that is different than ExecuteProcess, but I would not do what your are doing on principle because of the potential for write lock issues.

Yes, I understand there is a lock issue but the zeroout happens in the viewcreate sub process before I call the load subprocess. I don't see how putting the zero out and load into the sub process could possibly help (I've tried - it doesn't).

It seems like I need a way to be sure the lock has been released before the load processes start but not sure how to do this.

[Ajay]

So, I use the following and it hasn't caused any issues for me....not that I have noticed any

Have a master process, that you can from TM1RunTi, or anything else, and then "daisy chain" from it, so below the process called "Master_Process" calls each of the others, which complete before advancing to the next one

Master_Process
-Process_A_Create_View
-Process_B_Zero_View
-Process_C_Load_Data


Note that "Master_Process" contains the following in it's Epilog:

ExecuteProcess('Process_A_Create_View');
ExecuteProcess('Process_B_Zero_View');
ExecuteProcess('Process_C_Load_Data');

Hope this helps.....as I said above, I do this and I've not noticed any problems.

Ajay

[Wim Gielis]

So, I use the following and it hasn't caused any issues for me....not that I have noticed any

Have a master process, that you can from TM1RunTi, or anything else, and then "daisy chain" from it, so below the process called "Master_Process" calls each of the others, which complete before advancing to the next one

Master_Process
-Process_A_Create_View
-Process_B_Zero_View
-Process_C_Load_Data


Note that "Master_Process" contains the following in it's Epilog:

ExecuteProcess('Process_A_Create_View');
ExecuteProcess('Process_B_Zero_View');
ExecuteProcess('Process_C_Load_Data');

Hope this helps.....as I said above, I do this and I've not noticed any problems.

Ajay

And where do you execute processes in parallel and/or use RunProcess instead of ExecuteProcess ?

[prameson]

Thanks Everyone for your help.

I have managed to solve the problem using the following steps:

1. Changed the ExecuteProcess comands that I used for the View create sub process calls to RunProcess comands
2. Removed the ZeroOut view creation from the View create sub process and put it in the prolog of the Data Load sub process

Hope this helps anyone else having similar issues.

Peter