TM1 wait state processes are not executed

[AnonimusMax]

I have a TM1 instance that uploads data in TM1, first is updating the dimensions and after the dimensions are successfully updated, after that the tm1 execution flow of processes continues with
the processes that uploads data in cubes.

This process is triggered by a chore that executes a process from 5 to 5 minutes.
The executed process checks in Oracle database for a flag to load data in TM1.
The loaded data are for various combinations of reporting dates for different entities.
If the flag is 0 it runs a load master process that executes in parallel more processes using RunTI.
First process updates the status in Oracle on -2 and executes the rest of the processes from the flow.

The problem appears when there is more than 1 entity that needs to be loaded.
First entity processes are started in parallel and are executed (updating dimensions), the second entity processes and the rest of them are in wait state
if the flow for the first entity reaches the point when data are uploaded in cubes.
I know this is how TM1 works, but the PROBLEM is that when the 1st entity flow successfully ends, the second and third entity flows are not continued, just puts the -2 status in Oracle.
In logs there aren't any errors regardind the processes.
If I manually put 0 flag in Oracle just for 1 entity, the flow runs normally.

My questions are:
Is RunTi limited to a maxim number of running processes in parallel?
Why the processes that are put on Wait state do not continue to Run after the processes that caused them to be on wait are finished?
How can I further investigate this issue and what could be the solutions?

[EvgenyT]

Wait state indicates that target is still locked by another TI. Can you please identify what type of contention it is, e.g. IX etc. ?
You said it triggered by a chore: does the chore run in single/multi commit mode? From the description below its not clear whether there is a single process or multiple processes, i.e ODBC connection check, metadata, data etc.

As far, as the RunTI goes, I dont believe there is a limit per se, it's only governed by a number of available cores. Also, I would suggest turning on log4j.logger.TM1.Lock.Exception to Debug to see what's sort of locking going on there.

First entity processes are started in parallel and are executed (updating dimensions), the second entity processes and the rest of them are in wait state

- It's not entirely clear what do you mean here. If you are doing metadata updates then the IX will be held for the entire duration of the process.

You should provide actual code and bit more about architecture of these chore, so it's easier to give you a helpful advice.

Regards,

Evgeny

[AnonimusMax]

Wait state indicated is IXC.

The Chore runs a master process that include like 10 processes.

So let s say that 3 users try to upload data. Because the same dimensions are used,the process that load data for user 2 and 3 are placed in wait. After the data for user 1 is updated, the processes that are in wait state don t start.

[jim wood]

Ah now you've thrown good old chores in to the mix. I've seen all kinds of funnies with Chores. Are you executing each process in turn within the chore? If you are I would change this to executing one driver process in the chore and try this again. To be fair I try to avoid using chores if I can as there are so many issues. (Systems hangs, GMT, etc....)

[BrianL]

Wait state indicated is IXC.

Just FYI, threads in an IXC wait state will resume running one at a time. Here's an example. If you have one thread running and ten more in an IXC behind that one running thread, when the running thread completes a single thread in IXC state will be 'woken' up and start running. At that time you'll have one running thread and 9 still in IXC.

I strongly recommend using Lock.Exception DEBUG logging to identify the source of this contention so you can try to prevent it.

[EvgenyT]

Does chore run in multi or single commit mode?
Each user runs a chore...?

[AnonimusMax]

Thank you all for you responses.
The chore is auto schedule and executed from 5 to 5 minutes.
It s searches using a select in the DB and it runs the master process for the entities where the flag is 0.

Example:
we have a,b,c,d rows in the DB and the flag for a,b is 0, the chore will run the process for a, and b. So the same 2 Master process will start in the same time with different parameters.

[EvgenyT]

Thank you all for you responses.
The chore is auto schedule and executed from 5 to 5 minutes.
It s searches using a select in the DB and it runs the master process for the entities where the flag is 0.

Example:
we have a,b,c,d rows in the DB and the flag for a,b is 0, the chore will run the process for a, and b. So the same 2 Master process will start in the same time with different parameters.

Sorry, but this does not explain anything.

[AnonimusMax]

The chore is auto schedule and executed from 5 to 5 minutes.

Using a select the chore is reading from the DataBase.
In the DB we have a flag. An ETL process sets the flag=0 if new data needs to be loaded to TM1.
This Chore reads the flag, and if the flag is 0 it runs a MasterProcess with the parameters coming from DB. So if it finds 3 flags=0 then the MasterProcess is launched 3 times with different parameters. ( ex : one of the 3 MasterProcesses is started for product: X, 2015, another for product Y, 2015, and for product Z, 2015)
This Master Process is first updating a dimension that is causing the rest of the processes to be placed in Wait State.

[lotsaram]

This Master Process is first updating a dimension that is causing the rest of the processes to be placed in Wait State.

Well there is your problem identified.

What dimension is being updated and why? Does the master process really need to be updating the dimension?

[EvgenyT]

Metadata update is the most common offender when it comes to locking. Such operations should never be a part of a multi-user concurrent operation. I would be very surprised if metadata needs to updated each time someone executes Master Process

[Steve Rowe]

Not to disagree with the previous couple of posts as it is sub-optimal design to have MD update in any long running chore / TI or chore / TI that you want to multi-thread.

The issue the OP is raising though is that when the multi-threaded TIs get locked, when the lock is released from thread 1 why doesn't thread 2 run?

It should surely?

[EvgenyT]

The issue the OP is raising though is that when the multi-threaded TIs get locked, when the lock is released from thread 1 why doesn't thread 2 run?

Something fails along the way and the rest is trashing perhaps..