TM1RunTI and number of cores/threads

[vladino]

Hi guys,
is there any way how to tell the TM1 instance how many cores can be used for parallel interaction via TM1RunTI and how many cores need to be "reserved" for users?

Let's say I have 16 threads on TM1 server and I run 60 processes in parallel via TM1RunTI. How to tell TM1 that for TI execution only 10 cores can be utilized and other cores are "untouchable"?

Is there any way how to achieve this without any programming within the dispatcher process?

BR
Vladino

[lotsaram]

try https://github.com/cubewise-code/hustle it is a free utility which manages command line queues. All you need to do is tell the program how many threads to run at once and it will do the rest.

[vladino]

That's cool, thanks!

[Drg]

If you use windows server you can monitor threads from parent process;

Code: Select all

TIprc='Process_NAME';
pTreadNum=10;

SCRIPT='powershell -command "& { while(( Get-WmiObject Win32_Process -Filter """"name = ''tm1runti.exe'' and CommandLine like ''%-process """"""""'|  TIprc  |'""""""""%'' """" ).count -gt '|NUMBERTOSTRING(pTreadNum)|' ){sleep -Seconds 1} } "';
ExecuteCommand(SCRIPT, 1);

[vladino]

try https://github.com/cubewise-code/hustle it is a free utility which manages command line queues. All you need to do is tell the program how many threads to run at once and it will do the rest.

Is there any logfile for Hustle? I'm trying to run it but I'm getting this output:

Code: Select all

Starting processes...

Row 1 ::: START
Row 1 ::: ERROR

Row 2 ::: START
Row 2 ::: ERROR

Row 3 ::: START
Row 3 ::: ERROR

Row 4 ::: START
Row 4 ::: ERROR

Row 5 ::: START
Row 5 ::: ERROR

Row 6 ::: START
Row 6 ::: ERROR

Row 7 ::: START
Row 7 ::: ERROR

Row 8 ::: START
Row 8 ::: ERROR

Row 9 ::: START
Row 9 ::: ERROR

Row 10 ::: START
Row 10 ::: ERROR

Row 11 ::: START
Row 11 ::: ERROR

Row 12 ::: START
Row 12 ::: ERROR

Row 13 ::: START
Row 13 ::: ERROR

Row 14 ::: START
Row 14 ::: ERROR

[lotsaram]

You should have 2 text files

• a bat file calling hustle.exe with 2 params being the location of the file containing the runti execution list and how many threads to use

• a txt file specifying on each line the location of tm1runti.exe the process to run and any necessary parameters

If you can post an example of each file it should be pretty easy to spot where you might be going wrong.

(Obviously you don’t need the 1st bat file as this could be passed into the command line directly, but for the purposes of debugging it’s easier to see this also to eliminate anything wrong there)

[vladino]

Trying to run it from command line, from TM1 server "Data" folder. Hustle.exe is placed in the same folder as Data folder, so is HUSTLE.txt file.

Hustle_batch.bat

Code: Select all

..\Hustle.exe ..\HUSTLE.txt 10

HUSTLE.txt

Code: Select all

"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=MNG9999 p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=ORGDIV0 p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=60L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=61L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=62L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=63L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=64L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=65L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=66L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=67L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=68L1DIV p2=Retail
"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user TM1_service -CAMNamespace MSAD -pwd apple123 -process pBatch p1=69L1DIV p2=Retail

Result:

Code: Select all

Row 1 ::: START
Row 1 ::: ERROR

Row 2 ::: START
Row 2 ::: ERROR

Row 3 ::: START
Row 3 ::: ERROR

etc.

[lotsaram]

tm1runti is quite picky about text vs numeric parameter values. In my experience it quite likes or requires the process name and all string parameters to be in double quotes. Depending on how picky it is feeling it might also like the user and pwd in quotes as well. If using CAM it can also be picky about the order of namespace, user & pwd, ... and don't forget that you need the CAMNamespace ID not name and that this field is case sensitive. I normally use a runti ini file to store all the connection params (admin host, instance name, user, pwd or pwd file if using tm1crypt).

What happens if you copy a single line from your hustle instruction file and try to run directly in cmd window?
This should be your litmus test. Once you know this works then proceed with the hustle queuing. If it doesn't work in cmd line directly then figure out first what is wrong. Once fixed then proceed with the full solution.

Is "pBatch" an actual process name or did you accidentally write the string of the variable name to the file and not the value? Assuming all else is good I would first eliminate CAM as a variable and try with security mode 1 the following with all string param values encased in quotes ...

Code: Select all

"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user "TM1_service" -pwd "apple123" -process "pBatch" p1="MNG9999" p2="Retail" 

[Drg]

tm1runti is quite picky about text vs numeric parameter values. In my experience it quite likes or requires the process name and all string parameters to be in double quotes. Depending on how picky it is feeling it might also like the user and pwd in quotes as well. If using CAM it can also be picky about the order of namespace, user & pwd, ... and don't forget that you need the CAMNamespace ID not name and that this field is case sensitive. I normally use a runti ini file to store all the connection params (admin host, instance name, user, pwd or pwd file if using tm1crypt).

What happens if you copy a single line from your hustle instruction file and try to run directly in cmd window?
This should be your litmus test. Once you know this works then proceed with the hustle queuing. If it doesn't work in cmd line directly then figure out first what is wrong. Once fixed then proceed with the full solution.

Is "pBatch" an actual process name or did you accidentally write the string of the variable name to the file and not the value? Assuming all else is good I would first eliminate CAM as a variable and try with security mode 1 the following with all string param values encased in quotes ...

Code: Select all

"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user "TM1_service" -pwd "apple123" -process "pBatch" p1="MNG9999" p2="Retail" 

Code: Select all

"C:/Program Files/ibm/cognos/tm1_64/bin64/tm1runti.exe" -adminhost tm1admin -server TM1 -user "TM1_service" -pwd "apple123" -process "pBatch" "p1=MNG9999" "p2=Retail" 

[MGrain]

Might be going slightly off-topic here but I think it could be useful...

In order to make TM1RunTI a little palatable/easier to use we have a generic TI process (TM1RunTi_Did_It) as a kind of surrogate TM1RunTI. This is a TI process that has a dummy 17 field CSV file as a datasource and a single parameter - pConfigFile. This is a lot easier to run from the command line because you don't have to change anything once you've got TM1RunTI.exe running, the process name, the number and name of the parameters never change.

The pConfigFile parameter is the location and filename of a CSV file containing the name of the actual process we want to execute and all the associated parameters.

e.g.
"ImportData","pCube","GL Staging","pYear","2018"

In the Prolog of the TM1RunTI_Did_It process we change the datasource to be the CSV file we passed in as the pConfigFile parameter.

Code: Select all

DatasourceType='CHARACTERDELIMITED';
DatasourceNameForServer=pConfigFile;

This brings the process name and parameters into the process as variables, the data tab then just needs a bunch of ExecuteProcess commands to kick off the process. This is the top of the Data tab as an example, it goes on all the way down to no parameters but the code gets tedious really fast.

Code: Select all

bFlag = 'False';

IF(VParam8@<>'');
   bFlag='True';
   ExecuteProcess (vProcess,
		vParam1,vParam1Value,
		vParam2,vParam2Value,
		vParam3,vParam3Value,
		vParam4,vParam4Value,
		vParam5,vParam5Value,
		vParam6,vParam6Value,
		vParam7,vParam7Value,
		vParam8,vParam8Value
		);
EndIF;

IF(bFlag@='False' & vParam7@<>'');
   bFlag='True';
   ExecuteProcess (vProcess,
		vParam1,vParam1Value,
		vParam2,vParam2Value,
		vParam3,vParam3Value,
		vParam4,vParam4Value,
		vParam5,vParam5Value,
		vParam6,vParam6Value,
		vParam7,vParam7Value
		);
EndIF;

The other benefit to this approach is you can have multiple rows in the config file and have a sequence of processes run one after the other as each row of the pConfigFile is read into the Data tab; all from a single execution of the TM1RunTI commandline.

Apologies again for going off-topic, it's not *really* relevant to parallel processing and TI, but I thought it might be useful nonetheless.