Page 1 of 1

Parallel run vs. ICX

Posted: Wed Feb 28, 2018 10:40 pm
by vladino
Hi guys,
I'm trying to run several TI processes in parallel - sometimes I'm successful but sometimes not. Here's the thing...

I have one source cube and one target cube. The structure is nearly the same and I need to get the data out of the source cube and transfer it into the target one. For some purposes I'm exporting the data into a text file and then the text file is imported into the target cube.

To make it faster I would like to leverage parallel run of TI processes. I have (nearly) the same source views (there's difference only for ORG dim - ie. for each ORG unit I have a separate view). Then I have created adequate TI processes and I would like to run all these TI processes in the same time.

As far as I know there shouldn't be any IXC lock on both source and target cubes because each TI process is reading/writing from/to a separate ORG unit, right?

Sometimes it works and I can see several TI processes running at the same time. But sometimes there's only one TI process running and others are waiting in IXC state for the previous one to be finished.

Can anyone explain this behavior? Why am I getting IXC locks if each TI process is reading/writing from/to a separate ORG unit.

Thanks in advance!

BR
Vladino

EDIT What about temp subsets? If there are several processes running in parallel and all are trying to create a dynamic temp subset with the same name? Could this be the issue???

Re: Parallel run vs. ICX

Posted: Thu Mar 01, 2018 6:27 am
by lotsaram
All object names need to be unique. For parallel processing from/to the same cube it is important that all views and all subsets be uniquely named.

Re: Parallel run vs. ICX

Posted: Thu Mar 01, 2018 6:51 am
by vladino
lotsaram wrote: Thu Mar 01, 2018 6:27 am All object names need to be unique. For parallel processing from/to the same cube it is important that all views and all subsets be uniquely named.
Even if all views are using the same subset for products, measures etc?
It's not the same cube, I'm just exporting the data from the cube to the text file using Bedrock View export to File.

Re: Parallel run vs. ICX

Posted: Thu Mar 01, 2018 1:18 pm
by lotsaram
vladino wrote: Thu Mar 01, 2018 6:51 am
lotsaram wrote: Thu Mar 01, 2018 6:27 am All object names need to be unique. For parallel processing from/to the same cube it is important that all views and all subsets be uniquely named.
Even if all views are using the same subset for products, measures etc?
It's not the same cube, I'm just exporting the data from the cube to the text file using Bedrock View export to File.
If it's not the same cube then view names could be the same.
If dimensions are common even if cubes aren't then the subsets need to be uniquely named if there is any manipulation (editing) of the subsets. You could use the same subset name where all the TI processes don't touch or edit the subset in any way, just use it. But it is much safer th just make sure that any temp subsets for processing are done with unique names.

Re: Parallel run vs. ICX

Posted: Thu Mar 01, 2018 1:51 pm
by Drg
vladino wrote: Wed Feb 28, 2018 10:40 pm
As far as I know there shouldn't be any IXC lock on both source and target cubes because each TI process is reading/writing from/to a separate ORG unit, right?
dear friend, let's imagine a situation where you open an architect and run one of your processes. after you run another architect and try to log in and execute the second process.
what result will you get?

Re: Parallel run vs. ICX

Posted: Thu Mar 01, 2018 2:31 pm
by vladino
Drg wrote: Thu Mar 01, 2018 1:51 pm dear friend, let's imagine a situation where you open an architect and run one of your processes. after you run another architect and try to log in and execute the second process.
what result will you get?
I guess it depends on what is the purpose of both processes? :)

Re: Parallel run vs. ICX

Posted: Thu Mar 01, 2018 2:35 pm
by vladino
lotsaram wrote: Thu Mar 01, 2018 1:18 pm If it's not the same cube then view names could be the same.
If dimensions are common even if cubes aren't then the subsets need to be uniquely named if there is any manipulation (editing) of the subsets. You could use the same subset name where all the TI processes don't touch or edit the subset in any way, just use it. But it is much safer th just make sure that any temp subsets for processing are done with unique names.
OK so in other words - if I'm using any already existing and non-temp subset I don't need to duplicate this subset with unique name? And all I need is to create a subset with unique name if this subset is changed within any TI process?

Or is there any other reason for IXC locks?

Re: Parallel run vs. ICX

Posted: Fri Mar 02, 2018 12:15 pm
by Drg
vladino wrote: Thu Mar 01, 2018 2:31 pm
Drg wrote: Thu Mar 01, 2018 1:51 pm dear friend, let's imagine a situation where you open an architect and run one of your processes. after you run another architect and try to log in and execute the second process.
what result will you get?
I guess it depends on what is the purpose of both processes? :)
so check that this!
is not a long time to try to run waits or logn execution process(maybe empty while(i<100000)) from two different architects.
In fact, everything is not as fun as using the concept of user thread.

If to be absolutely simple to you for detouring of locks I would advise to launch processes under different users. :roll:

Re: Parallel run vs. ICX

Posted: Fri Mar 02, 2018 12:25 pm
by Drg
I do not know the nature of such locks, I guess only that this is due to the presentation of data at a certain point in time
those.
in the moment you read of the cube, somebody put data into this cube but the reading is not blocked and the record is not blocked either, and the data you read is actually obsolete but not for your session. And perhaps the session time is tied to the user and not to a specific authorization. which leads to the occurrence of locks.

Below is a description of the post where I came across a similar problem
http://tm1forum.com/viewtopic.php?f=3&t=13857

Re: Parallel run vs. ICX

Posted: Fri Mar 16, 2018 2:30 am
by PeteB
Hi Everyone reading this thread,

Firstly apologies if you have read a similar comment on other very similar treads. It is very likely we each have had a similar problem, which can be fixed by IBM in a future release. I am told by IBM this should be a trivial piece of work to deliver the copy function from a slice of rules cells to value cells and the performance should be significantly faster for TM1 to internalise the copy rather than the current approach of exporting the rule values and importing into the snapshot version including across cubes.

Please vote for an RFE for IBM to provide the function to copy rule based cells to values.
https://www.ibm.com/developerworks/rfe/ ... _ID=116669 - TM1 Data Snapshot functionality - Copy rule to values in a TM1 cube

IBM has commented they intend to decline the request as functionality already exists, but I am hoping with enough votes they will reconsider their position.

Re: Parallel run vs. ICX

Posted: Fri Mar 16, 2018 11:19 am
by Steve Rowe
Hi PeteB,

Please don't use other posts to drum up support for your RFE, we have a specific forum to post details of RFE in (http://www.tm1forum.com/viewforum.php?f=19).

If we allow this behaviour posts will become a mess of the OPs topic and people boosting their RFEs.

Many thanks,

Re: Parallel run vs. ICX

Posted: Fri May 25, 2018 3:40 pm
by BrianL
This sounds like a good use case for the TI debugger. If you're running 10.2.2 FP7 or newer you can set a breakpoint in the debugger on the IX lock acquisition. This should help you understand what TI code is causing the lock.