10.2.2 Fix Pack 4 - Temporary Objects in TI

User avatar
George Regateiro
MVP
Posts: 326
Joined: Fri May 16, 2008 3:35 pm
OLAP Product: TM1
Version: 10.1.1
Excel Version: 2007 SP3
Location: Tampa FL USA

10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by George Regateiro »

10.2.2 Fix Pack 4 was just released with what could be a massive improvement if it works well, temporary views and subsets in TI.

http://www-01.ibm.com/support/docview.w ... wg27046436
User avatar
wissew
Posts: 54
Joined: Tue Jun 17, 2008 7:24 pm
OLAP Product: TM1
Version: 9.5.2; 10.2.2; 11
Excel Version: 2003 SP3 - 2013
Location: Beaverton, OR

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by wissew »

Has anyone tested this in real life yet? What's the feed back on it?
nick_leeson
Posts: 98
Joined: Sat Feb 11, 2012 11:13 am
OLAP Product: TM1 9x, BPC, Hyperion, HANA
Version: TM1 10
Excel Version: Excel 2003 - 2010

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by nick_leeson »

As of TM1 10.2.2 Fix Pack 4, the SubsetCreate, SubsetCreateByMDX, and ViewCreate TurboIntegrator functions can all create temporary objects. There is no locking associated with a temporary subset or view, as these objects are never saved. This can result in improved performance, because there is no need for TurboIntegrator to wait for locks to be released before operating upon a temporary object.
Hallelujah !!
lotsaram
MVP
Posts: 3648
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by lotsaram »

This sounds very promising indeed. Not only from the mitigation of potential metadata locks but also the potential performance improvement by avoiding a whole heap of unnecessary I/O (and potential for file errors) by simply not writing a stack of temp .vue and .sub files to disk followed a millisecond later by their deletion.

I have some questions that I think will need testing like what happens if the AsTemporary argument is added but someone forgets to remove the ViewDestroy and SubsetDestroy? Temp objects (presumably) don't deed a destroy and temp objects take precedence to permanent objects of the same name. Do you get an error if trying to destroy a temp object?

The updated documentation STILL makes no mention of the previously undocumented DimensionName argument which allows the creation of an empty set without error for SubsetCreateByMDX. As this was never properly documented it this now depreciated? Or is the function "clever" enough to figure out that if the 3rd argument is a string then it is DimensionName and if numeric/boolean then it must be AsTemporary? What happens when all 4 possible arguments are passed, what order to the 3rd & 4th arguments need to be in? Need to test. Of course it would be nice if IBM just documented this properly in the first place.

-----
Edited to get the argument numbers correct.
Last edited by lotsaram on Wed Sep 16, 2015 8:20 am, edited 1 time in total.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
upali
Posts: 38
Joined: Thu Oct 11, 2012 6:15 am
OLAP Product: TM1
Version: 10.2.2.4
Excel Version: 2010
Location: Melbourne, Australia

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by upali »

Well this works fine.

Code: Select all

cSubset = 'TM1_Temp_Objects';
vDim = 'Year';
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1FILTERBYLEVEL( {TM1SUBSETALL( [' | vDim | '] )}, 0)}, ASC)}', 1);

i = SubsetGetSize(vDim, cSubset);
while (i > 0);
	AsciiOutput('D:\Temp\out.txt', SubsetGetElementName(vDim, cSubset, i));
	i = i -1;
end;
At the end of the TI run, there was no 'TM1_Temp_Objects' in my Year dimension, but out.txt had all the n-level elements.
lotsaram
MVP
Posts: 3648
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by lotsaram »

And what happens when using
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1FILTERBYLEVEL( {TM1SUBSETALL( [' | vDim | '] )}, 0)}, ASC)}', vDim, 1);
OR
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1FILTERBYLEVEL( {TM1SUBSETALL( [' | vDim | '] )}, 0)}, ASC)}', vDim);

Hopefully the first should be identical to your last test and the second should create a permanent subset.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Wim Gielis
MVP
Posts: 3099
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by Wim Gielis »

lotsaram wrote:And what happens when using
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1FILTERBYLEVEL( {TM1SUBSETALL( [' | vDim | '] )}, 0)}, ASC)}', vDim, 1);
OR
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1FILTERBYLEVEL( {TM1SUBSETALL( [' | vDim | '] )}, 0)}, ASC)}', vDim);

Hopefully the first should be identical to your last test and the second should create a permanent subset.
And an additional test would be:

Code: Select all

SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1F...)}', vDim, 1);
AND
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1F...)}', vDim);
meaning a completely wrong syntax but the undocumented DimensionName parameter still leads to the creation of the subset.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
declanr
MVP
Posts: 1815
Joined: Mon Dec 05, 2011 11:51 am
OLAP Product: Cognos TM1
Version: PA2.0 and most of the old ones
Excel Version: All of em
Location: Manchester, United Kingdom
Contact:

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by declanr »

Did some quick tests and found that:

Code: Select all

SubsetCreateByMDX ( sSubName, sMDX, 1 );
- Works to create a temp subset

Code: Select all

SubsetCreateByMDX ( sSubName, sMDX, 0 );
- Works to create a subset

Code: Select all

SubsetCreateByMDX ( sSubName, sMDX );
- Works to create a subset (as long as the MDX is valid)

Code: Select all

SubsetCreateByMDX ( sSubName, sMDX, 1, sDimName );
- Fails (error is that Dimension Name " " could not be found)

Code: Select all

SubsetCreateByMDX ( sSubName, sMDX, sDimName, 1 );
- Works - Creates Temp Subset regardless of whether MDX is valid or not


I will test in more detail later but it seems promising so far.



Edit - Forgot to mention that specifically using MDX which would return an error/empty subset and passing it with:

Code: Select all

SubsetCreateByMDX ( sSubName, sMDX, sDimName );
- Still works to create an empty subset with no process errors. (Meaning we shouldn't have to worry about an upgrade killing our existing TIs if we used the undocumented parameter - which I did A LOT.
Declan Rodger
Wim Gielis
MVP
Posts: 3099
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by Wim Gielis »

declanr wrote:(Meaning we shouldn't have to worry about an upgrade killing our existing TIs if we used the undocumented parameter - which I did A LOT.
I also use it from time for time.

Thanks for testing so far Declan and upali.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
declanr
MVP
Posts: 1815
Joined: Mon Dec 05, 2011 11:51 am
OLAP Product: Cognos TM1
Version: PA2.0 and most of the old ones
Excel Version: All of em
Location: Manchester, United Kingdom
Contact:

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by declanr »

lotsaram wrote: I have some questions that I think will need testing like what happens if the AsTemporary argument is added but someone forgets to remove the ViewDestroy and SubsetDestroy? Temp objects (presumably) don't deed a destroy and temp objects take precedence to permanent objects of the same name. Do you get an error if trying to destroy a temp object?
I created a TI that just has View Destroy and Subset Destroy in it - nothing else.
The very first line in my TI tries to destroy a View that has NEVER existed in my model.
The second and very last line in my TI tries to destroy a Subset that has NEVER existed in my model.
Both the Cube and Dimension for the view/subset in question do exist.

This process completed successfully with no errors - so it appears that trying to destroy a view or subset that don't exist (be that because they are temporary or because you had a typo etc.) will go essentially un-flagged.
Declan Rodger
User avatar
paulsimon
MVP
Posts: 808
Joined: Sat Sep 03, 2011 11:10 pm
OLAP Product: TM1
Version: PA 2.0.5
Excel Version: 2016
Contact:

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by paulsimon »

Hi

Please would someone who has this fix pack installed run a test to see

a) If you can have a temporary view with the same name as as permanent view, and if so, does the temporary view take precedence, eg create a temporary view with the same name as a permanent view but different selection criteria to see if the process asciioutputs the data from the permanent or temporary view

b) Whether the temporary view is session specific, eg if two users have views with the same temporary name but different definitions, if there is any conflict. Harder to test but perhaps set up a read from a view and write to ascii from a reasonably large cube, and while that is running, run a TI creating a temporary view of the same name, with a much smaller selection, to see if this has any impact on the first or second process. If the temporary views are session specific then we don't need to append the user name or timestamp or any other such fiddle to avoid the risk of one view conflicting with another in a multi-user environment.

Thanks

Paul Simon
Edward Stuart
Community Contributor
Posts: 247
Joined: Tue Nov 01, 2011 10:31 am
OLAP Product: TM1
Version: All
Excel Version: All
Location: Manchester
Contact:

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by Edward Stuart »

I've completed some further testing that proves IBMs note that Temporary objects take precedence, point b) will have to wait for now!

I created a Public View which contained a Public Subset defined as 'UK', all other elements were left as is ('All'). I created a TI process with the Public View as the datasource, created a temporary view with the same name and a temporary subset with the same name.

I inserted the element 'US' to the temporary subset and assigned to the Temporary View and ran the ViewZeroOut process (note only temporary views can hold temporary subsets)

The US data was deleted, meaning the Temporary View/ Subset took precedence. I removed the SubsetAssign and on running the process the UK data was deleted

I was not able to ASCIIOUTPUT from a Temporarily created view from the Data tab
User avatar
paulsimon
MVP
Posts: 808
Joined: Sat Sep 03, 2011 11:10 pm
OLAP Product: TM1
Version: PA 2.0.5
Excel Version: 2016
Contact:

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by paulsimon »

Thanks Ed
User avatar
mattgoff
MVP
Posts: 516
Joined: Fri May 16, 2008 1:37 pm
OLAP Product: TM1
Version: 10.2.2.6
Excel Version: O365
Location: Florida, USA

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by mattgoff »

Looks like temporary objects don't work with a dynamic datasource (building the view in the prolog and using DatasourceNameForServer / DatasourceNameForClient / DataSourceType='View' / DatasourceCubeview. Bummer, that's a huge percentage of my use case. Now I have to decide if it's worth refactoring to use a parent process to build the temp object vs. keep writing a kajillion ViewExists/SubsetExists (and Create and Destroy and SubsetDeleteAllElements) statements.
Please read and follow the Request for Assistance Guidelines. It helps us answer your question and saves everyone a lot of time.
lotsaram
MVP
Posts: 3648
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by lotsaram »

mattgoff wrote:Looks like temporary objects don't work with a dynamic datasource (building the view in the prolog and using DatasourceNameForServer / DatasourceNameForClient / DataSourceType='View' / DatasourceCubeview. Bummer, that's a huge percentage of my use case. Now I have to decide if it's worth refactoring to use a parent process to build the temp object vs. keep writing a kajillion ViewExists/SubsetExists (and Create and Destroy and SubsetDeleteAllElements) statements.
Can you explain? Because well errm, that would seem to be the ENTIRE POINT of temp views.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
upali
Posts: 38
Joined: Thu Oct 11, 2012 6:15 am
OLAP Product: TM1
Version: 10.2.2.4
Excel Version: 2010
Location: Melbourne, Australia

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by upali »

Bummer...

I can confirm that,

Code: Select all

cCube = 'System Info';
cSubset = 'TM1_Temp_Objects';
cView = cSubset;
cTemp = 1;

# View
ViewCreate(cCube, cView, cTemp);

vDim = 'System Parameters';
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1FILTERBYLEVEL( {TM1SUBSETALL( [' | vDim | '] )}, 0)}, ASC)}', cTemp);

vDim = 'System Measures';
SubsetCreatebyMDX(cSubset, '{TM1SORT( {TM1FILTERBYLEVEL( {TM1SUBSETALL( [' | vDim | '] )}, 0)}, ASC)}', cTemp);

ViewSubsetAssign(cCube, cView, 'System Parameters', cSubset);
ViewSubsetAssign(cCube, cView, 'System Measures', cSubset);

# Define datasource
DataSourceType = 'VIEW';
DatasourceNameForServer = cCube;
DatasourceNameForClient = cCube;
DatasourceCubeView = cView;

ViewExtractSkipCalcsSet(cCube, cView, 1) ;
ViewExtractSkipRuleValuesSet(cCube, cView, 1) ;
ViewExtractSkipZeroesSet(cCube, cView, 1);
Doesn't work it gives a "Unable to open data source" message.

Setting

Code: Select all

cTemp = 0;
Works fine...
lotsaram
MVP
Posts: 3648
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by lotsaram »

Hmmm. So a temp subset does work but a temp view can't be assigned as a data source ....
... so a temp view does exactly what then? (Answer seems to be "NOTHING")

If anyone from IBM server dev team happens to be listening in.
Temp view MUST be able to assigned as a data source, really zero out and data source are the only 2 uses of a temp view so this has to be supported.
Temp views and subsets MUST be persistent enough to be seen by other TI processes that are part of the same "transaction chain" that is TI calling an ExecuteProcess or as part of the same chore. Pretty much anyone who is doing good quality TM1 development will have invested a lot in function libraries to support modular development for common requirements. If a temp view or subset can only be seen by the individual TI process that created it then this would also make the utility of temp objects as good as useless.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
EvgenyT
Community Contributor
Posts: 324
Joined: Mon Jul 02, 2012 9:39 pm
OLAP Product: TM1
Version: PAL 2.0.8
Excel Version: 2016
Location: Sydney, Australia

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by EvgenyT »

"Shattered dreams"... as I was getting ready to get my hands on this functionally..
upali
Posts: 38
Joined: Thu Oct 11, 2012 6:15 am
OLAP Product: TM1
Version: 10.2.2.4
Excel Version: 2010
Location: Melbourne, Australia

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by upali »

lotsaram wrote:Hmmm. So a temp subset does work but a temp view can't be assigned as a data source ....
... so a temp view does exactly what then? (Answer seems to be "NOTHING")
Just wondering, is there any other use-case for temp views? I find it useless since dynamic data source isn't supported.
lotsaram
MVP
Posts: 3648
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: 10.2.2 Fix Pack 4 - Temporary Objects in TI

Post by lotsaram »

We have logged this and it has been accepted as an APAR.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Post Reply