Parallel Processing

[AmbPin]

Hello,

I have created a suite of processes to perform a data transformation exercise that runs ordinarily overnight following a data load. Occasionally it is necessary to re-run the load and transformation processes for a specific subset of data, let’s say for a specific department. To achieve this I have built the following:-

1) TI process to cycle through all departments to execute the CMD file (2)
ExcecuteCommand passing parameters
psDeptID – 00 for all depts., Dept ID for specific Dept
psDataClearOnly
Wait is set to zero so that all departments are transformed using parallel processing

2) CMD file to launch (3)
tm1runti.exe .... psDeptID, psDataClearOnly

3) TI process performs data clear operation then if calls psDataClearOnly is FALSE calls (4)
ExcecuteProcess ... psDeptID

4) TI process performs data transformation.

This all works perfectly well, with the CMD file logging the execution of each process (3) by dept and process (4) logging the end. Several CMD.EXE’s are shelled on the server which then close down as each finishes processing for their designated department.

I tried to add another parameter to process (1) that would allow me to control the Wait option. If I set Wait to 1 then the CMD.EXE & tm1runti.exe processes that are shelled never close on the server although the data processing is completed and is visible in the target cube. Does anyone have ideas why this might be? I wondered if it was because I used tm1runti.exe to call proc 3 which in turn calls proc 4.


I hope my explanation makes some kind of sense.

[BigG]

from IBM, hope this helps

Simultaneous Connections to the Same TM1 Server
Do not perform any operation within a TurboIntegrator process that creates a new connection (logs
in) to the same TM1® server where the process is already running. This type of scenario could cause
a deadlock situation between the two logins or threads causing the server to hang or possibly crash.
For example, avoid the following scenarios:
● Do not use a TI process to launch an ODBO MDX query (via theTM1 OLE DB MD Provider)
into the same server. This scenario can result in both the process and the query waiting on each
other to finish.
12 IBM Cognos TM1
Chapter 1: TurboIntegrator Basics
● Do not use the TI function, ExecuteCommand, to call out of a TI process to run and wait (Wait
argument set to 1) for an external program that logs back into the same server. This includes
any custom application or any IBM Cognos application, such as the TM1 ETLDAP utility, that
could possibly connect back to the same server.
Be aware that using the ExecuteCommand function with its Wait argument set to 1 risks hanging
the server even if the external program doesn't log back into the same server. If the external program
encounters its own problem and becomes hung, the TI process essentially hangs waiting for the
external program to finish executing.