TM1 Forum

PavoGa wrote:I have for several years used master processes in conjunction with a control cube to manage the execution of processes. The basic structure of the system would be:

* a master process reads the control cube for processes to execute along with parameters
* builds a datasource, typically a dimension subset
* the processes are executed via ExecuteProcess function in DATA tab.

This year I've started using TM1RunTI in order to minimize locking issues but have now run into a new problem. And maybe I just need some guidance on what one can/cannot or should do.

Because of the aggravation of migration policy to production environments, we have TI processes built to perform maintenance issues so it does not take four days to migration a dimension/attribute/etc change to production (as long as we use TI processes, we can make all the changes we want). The current problem is a TI process that does nothing but insert an attribute (an alias in this case), locks the server up (contention lock with }Dimensions) and the service has to be stopped and restarted. This particular process runs without a lock contention from the same master process if ExecuteProcess is used instead of TM1RunTI.

So a couple of questions:

1) Is there any problem with executing processes from the DATA section instead of within the PROLOG section?
2) The attribute insert will not run through TM1RunTI without locking up the server regardless of attribute type. Is it because a dimension is being read as the datasource? If so, is there anyway around it other than using ExecuteProcess or ATTRInsert in the master process?

Alan Kirk wrote:The problem is that when you call RunTI you are creating something which is not part of the chore or set of called processes, as the case may be. Sure, you are calling it from a process BUT RunTI creates its own connection to the server, and then does its own updates. It's really no different to another user logging on and trying to do changes. The problem is that they system knows that the first user (your original string of processes) is already making changes to the metadata and hasn't finished yet. Then this new guy in the form of the RunTI connection to the server comes along and tries to do the same. Result? Contention lock.

The Execute command takes two parameters; the second one describes whether to have a synchronous call or an asynchronous call. Cognos TM1 tools should only be called asynchronously (Parameter 0) to avoid server deadlocks if the system is waiting for a lock held by the TurboIntegrator process and the process is waiting for the utility. The same advice applies to any executables called by ExecuteCommand if they login to Cognos TM1 .
Note: Never use a synchronous call if the tool logs into Cognos TM1 .

It can actually get worse than this. If you use ExecuteCommand to call TM1RunTI and set the wait parameter to one, you can completely deadlock TM1. The master process can be waiting for the runti command to finish while that process hit contention with the master process. This kind of deadlock requires manual intervention to cancel (at least) one of the processes.

That's the first time I've heard of the Synchronized() function crashing. It's been my experience that IBM takes crash issues seriously but YMMV

###################
# parameters
# pDebug = debug flag  DEFAULT to 0
# pCube = the cube to load the rules for.
####################

NumericGlobalVariable('ProcessReturnCode');
NumericGlobalVariable('DataMinorErrorCount');

### Inititalize standard variables
strProcessName = GetProcessName();
tNow = NOW;
strNow = TIMST(NOW, '\Y\m\d\h\i\s');
strEnvironment = '';
strErrLogFolder = CELLGETS('Control Globals', 'Value', 'OutputPath');
strPrologErrLogFile = strErrLogFolder | strProcessName | '_PRO_err.log';
strMetaErrLogFile = strErrLogFolder | strProcessName | '_MET_err.log';
strDataErrLogFile = strErrLogFolder | strProcessName | '_DAT_err.log';
strEpilogErrLogFile = strErrLogFolder | strProcessName | '_EPI_err.log';
strDataFolder = CELLGETS('Control Globals', 'Value', 'TestOutputPath');
strTmp = 'tmp_' | strProcessName | strNow;
strEmailProcess = 'eMail - Ctrl';
strUser = TM1User();
cubEmail = 'eMail - Configuration';
nErrorFlag = 0;
nRecordsProcessed = 0;
nDataSourceCount = 0;

### Initialize other variables

cubRuleManagement = '}System - Rule Management';

Synchronized(cubRuleManagement);

4576 [30] INFO 2016-06-08 22:04:03.873 TM1.Process Process "adm_Migration Management" executed by user "admin_ty"
6076 [33] INFO 2016-06-08 22:04:05.355 TM1.Process Process "adm_Load Rule from Rule Cube" executed by user "admin_ty"
6076 [33] DEBUG 2016-06-08 22:04:05.433 TM1.Lock.Exception Contention encountered attempting to acquire lock (0x000000000792A938) on object [Cube "Control Globals", addr=0x000000000792A010, index=R-1290] in IX mode at ..\tm1_r7s\TM1CubeImpl.cpp:19101 during function 'ProcessExecuteEx'.
Entering wait state 'IXCur'.
Blocked by the following 0 threads:

3132 [35] INFO 2016-06-08 22:04:06.416 TM1.Process Process "adm_Load Rule from Rule Cube" executed by user "admin_ty"