TM1 Forum

Hi gurus,

I am facing a weird issue which I cannot explain why it happens. I have a TI Process that extracts data from a TM1 Cube. This is huge cube and in order to reduce the Run Time, I have created the TI in such a way that it extracts only the data for a specific Period and a specific Region ( with Period and Region being Dimensions in the cube ) and then I run the TI in parallel multiple times ( one for each combination of Period and Region that has data in the cube ). All works fine, but because all these run in separate threads, I had to build a mechanism that will check if all these induvidual threads are finished executing so i can proceed with the execution of Next TI. For this reason, I have created a cube called Parallel_Execution_Status with following dimension
1. Period
2. Region
3. Parallel_Execution_Status_m ( with 2 flag elements, Execution_Started, Execution_Finished )

So, before I kick in the individual threads ( through TM1RunTI command ), I flag all the threads that are about to be started

Period Region Started Finished
JAN US 1 0
FEB US 1 0
MAR US 1 0
APR US 1 0
MAY US 1 0
JUN US 1 0
JAN CA 1 0
FEB CA 1 0
MAR CA 1 0
APR CA 1 0
MAY CA 1 0
JUN CA 1 0

In this case, 12 threads will start in parallel.

At the end of data_export TI, ( last lines in Epilog ), I put the following Code

#=== Flag the thread as Finished
CELLPUTN ( 1 , 'Parallel_Execution_Status' , pPeriod , pRegion , 'Execution Finished' ) ;

#=== Check if all threads are finished
nTotalStartedThreads = CELLGETN ( 'Parallel_Execution_Status' , 'Total Period', 'Total Region', 'Execution Started' ) ;
nTotalFinishedThreads = CELLGETN ( 'Parallel_Execution_Status' , 'Total Period', 'Total Region', 'Execution Finished' ) ;

IF ( nTotalStartedThreads = nTotalFinishedThreads ) ;
ASCIIOUTPUT ('e:\extract.txt' , 'Data extract is completed' ) ;
ELSE ;
ASCIIOUTPUT ('e:\extract_DEBUG_' | pPeriod | '_' | pRegion | '.txt', pPeriod , pRegion , 'Number of Started Threads=' | NUMBERTOSTRING ( nTotalStartedThreads ) , 'Number of Finished Threads=' | NUMBERTOSTRING ( nTotalFinishedThreads )) ;

ENDIF ;

I monitor the threads while executing in TM1 TOP and one I see that an individual thread is finished, if i refresh my view from cube Parallel_Execution_Status in Architect, I can see that the 'Execution Finished' flag for that thread is = 1 and the Total Number of Finished threads gets incremented, which proves that the TI is maintaining the flag properly. But the Statement
nTotalFinishedThreads = CELLGETN ( 'Parallel_Execution_Status' , 'Total Period', 'Total Region', 'Execution Finished' ) ;

Retuns 1, and sometimes 2, but it never returns the same number that i see in Architect.

DO you know if there is any way to enforce a data commit ( it looks like these threads look at an older version of data in my cube )
So if threads finish executing 2-3 minutes appart from each other, they can see the flags that were commited from previous thread, but if they finish even 60 seconds appart, they don't see the flag that was set from the other thread which is finished 60 seconds earlier.

Weird, right?

Thanks

Is it all in then epilog of the child TI or is the check to see if all have finished in the parent TI?

Also I tend to do this sort of check by putting out simple text files and then checking to see if all exist before deleting them and ending the parent.

Thank you declanr,

I have tried putting the code in Prolog Tab of a child TI Process ( being called by Main process ) and then checking the counts in the Epilog Tab of the child process and still the same issue. It is weird why the individual threads running in parallel cannot get the updated count ( knowing that the other threads are finished executing more than a minute earlier ).

Yes I can make it work if I use the ASCIIOUTPUT / File Count approach but this client is against that so they wont allow us to do that

Thank you

ardi wrote: Yes I can make it work if I use the ASCIIOUTPUT / File Count approach but this client is against that so they wont allow us to do that

So, your client is against a simple solution that will work versus a convoluted one that may or may not?????? If they don't like the idea of a text file sitting in a folder, you could write a flag record to a database table somewhere. I've used that approach a number of times.