LOCKS on subset and views wheen mtq > 0

Post Reply
Drg
Regular Participant
Posts: 159
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

LOCKS on subset and views wheen mtq > 0

Post by Drg »

Hi to all.
Someone noticed a drop in the performance of parallel(runti) processes if mtq is greater than 1.

I caught it when I produced parallel unloading from the model and used the same subsets to form views.

a piece of code that forms subsets based on a custom user subset:

Code: Select all

vLastDim=param1;
vSub=param2;

MDX='{ tm1subsettoset(['|vLastDim|'], "'|vSub|'") }';
SubsetCreate(vLastDim, vSubset, 1 );
SubsetMDXSet(vLastDim, vSubset, MDX );		
SubsetMDXSet(vLastDim, vSubset, '');
Has anyone come across such situations and is there a silver bullet for using MTQ and excluding locks on unload?
Now it is solved by the fact that when the unloading of the ITC is transferred to 1, but this is not always convenient, especially when you don’t know which stream will be the last :?
EvgenyT
Community Contributor
Posts: 324
Joined: Mon Jul 02, 2012 9:39 pm
OLAP Product: TM1
Version: PAL 2.0.8
Excel Version: 2016
Location: Sydney, Australia

Re: LOCKS on subset and views wheen mtq > 0

Post by EvgenyT »

Hi,

You code is showing a subset creation procedure, MTQ is applicable for data reads under following conditions:
  • Queries exceeding 10,000 cell visits.
  • Creation of TM1 ‘stargate’ vew.
  • Views containing the results of rule derived calculations.
Sorry, what do you mean by Parallel Unloading?

Thanks

ET
Drg
Regular Participant
Posts: 159
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

Re: LOCKS on subset and views wheen mtq > 0

Post by Drg »

EvgenyT wrote: Mon Apr 01, 2019 10:02 pm Hi,

You code is showing a subset creation procedure, MTQ is applicable for data reads under following conditions:
  • Queries exceeding 10,000 cell visits.
  • Creation of TM1 ‘stargate’ vew.
  • Views containing the results of rule derived calculations.
Sorry, what do you mean by Parallel Unloading?

Thanks

ET
Thanks for answering.
I mean, having the process of reading data from a cube on certain subset-based filters.
And if I run this process for one cube, but with different filters, in the parallel path I encounter locks between these processes.
Question someone faced with such matters?
If I use these processes with MTQ = 1, then the processes are perfectly executed in the parallel switch without locks.
EvgenyT
Community Contributor
Posts: 324
Joined: Mon Jul 02, 2012 9:39 pm
OLAP Product: TM1
Version: PAL 2.0.8
Excel Version: 2016
Location: Sydney, Australia

Re: LOCKS on subset and views wheen mtq > 0

Post by EvgenyT »

Hi,

Apologies, but I'm still not following. In the nutshell there is no relationship between TM1 Lock model and MTQ.
MTQ leverages multiple processing threads to process a Cube View under certain conditions, one of those being ViewExtractSkipCalcsSet (cube, view,0).

Am I correct to assume that you are using Tm1RunTi (or related parallelism utility) to run data in parallel? If so, a common locking issue may arise if you are not using unique subset/view names when building up your data source. Usually, you would have a subset/view name along the lines of... GetProcessName() | NumberToString (Rand() * Rand()) to avoid locking.

Also, where do you see the "locking" going on? TM1Top? Do you have log4j.logger.TM1.Lock=DEBUG in tm1s-log.properties to troubleshoot locking?

Furthermore, probably not related to locking at all, but how do you know that your TI leverage MTQ? Do you see multiple worker threads in TM1Top?
I suggest to use these loggers below to understand what is going on:

To capture Stargate creation times: log4j.logger.TM1.Cube.Stargate=DEBUG
To capture work unit splitting: log4j.logger.TM1.Parallel=DEBUG
To capture the event of operation threads picking work units: log4j.logger.TM1.OperationThread=DEBUG

Please provide little bit more information about your issue, troubleshooting, process procedure etc. It is hard for me to help you at this stage.

Many Thanks

Evgeny
Post Reply