Dimension Order in an MDX cube view as datasource

User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Dimension Order in an MDX cube view as datasource

Post by PavoGa »

There does not seem be a rational way to determine the dimension order when a TI is based on a cube view created by MDX.

The original cube order:
  1. Org
  2. HRLevel
  3. Period
  4. Adjustments
  5. Measure
The MDX:

Code: Select all

sViewMDX = EXPAND('SELECT        
        {[Measure].[Measure].[FTE]}
        ON COLUMNS,
        NON EMPTY
        TM1SUBSETTOSET( [Org].[%hierOrg%], "Planning Components") *
        TM1SubsetToSet([HRLevel].[HRLevel], "All Leaves")
        ON ROWS
    FROM [stage.HRActuals]    
    WHERE ([Adjustments].[Adjustments].[Effective],
        [Period].[Period].[%psPeriod%])');
The order the dimensions are read in the process (with placement in MDX statement):
  1. HRLevel (ROW-2nd)
  2. Period (WHERE-2nd)
  3. Adjustments (WHERE-1st)
  4. Measure (COLUMN)
  5. Org (ROW-1st)
Have to do a trial run and ASCIIOUTPUT to determine read order, then name the variables. Would have thought there would be some rational way to determine the dimension order. Am I missing something here?
Ty
Cleveland, TN
lotsaram
MVP
Posts: 3652
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Dimension Order in an MDX cube view as datasource

Post by lotsaram »

Is the cube re-ordered ("optimized") in the order of the variables?
  • HRLevel
  • Period
  • Adjustments
  • Measure
  • Org
It then wouldn't be correct how the variables are being ordered, but it would offer an explaination.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Dimension Order in an MDX cube view as datasource

Post by PavoGa »

lotsaram wrote: Tue Jun 18, 2019 8:00 pm Is the cube re-ordered ("optimized") in the order of the variables?
  • HRLevel
  • Period
  • Adjustments
  • Measure
  • Org
It then wouldn't be correct how the variables are being ordered, but it would offer an explaination.
Yes, that was checked and I should have stated so. While I have no explanation, all that seems to have changed is the first dimension (Org) simply shifted to the bottom of the order.
Ty
Cleveland, TN
lotsaram
MVP
Posts: 3652
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Dimension Order in an MDX cube view as datasource

Post by lotsaram »

Ty,

It seems that the ordering is dependent on the view structure
- First WHERE
- Then ROW
- Last COLUMN

As MDX axis 0 is columns there is a requirement that a view contains at least one dimension on columns. I think this is the only requirement but your level of MDX I think is better than mine.

As a solution to the issue of variable ordering it looks like this can be solved by having a "ViewExtract" form for the MDX view (with all dimensions cross-joined on columns in cube index order) as opposed by a "user query" view (containing columns, rows & where).

E.g. for Planning Sample
In this format if you want to get an idea of the cellset (with last dimension on column and all other dimensions cross-join on rows)

Code: Select all

sMDX = '
SELECT 
    NON EMPTY
        { Tm1FilterByLevel( Descendants( [plan_time].[2004] ), 0) }
    ON COLUMNS, 
    NON EMPTY
        { [plan_version].[FY 2004 Budget] }
      * { [plan_business_unit].[UK] }
      * { [plan_department].[Direct] }
      * { Tm1FilterByLevel( Tm1SubsetAll( [plan_chart_of_accounts] ), 0) }
      * { [plan_exchange_rates].[local] }
      * { [plan_source].[input] }
    ON ROWS
FROM [plan_BudgetPlan]
';
Or in this format if it is purely as a source for TI (with all dimensions cross-join on columns). Won't be able to visualize but TI doesn't seem to mind.

Code: Select all

sMDX = '
SELECT 
    NON EMPTY
        { [plan_version].[FY 2004 Budget] }
      * { [plan_business_unit].[UK] }
      * { [plan_department].[Direct] }
      * { Tm1FilterByLevel( Tm1SubsetAll( [plan_chart_of_accounts] ), 0) }
      * { [plan_exchange_rates].[local] }
      * { [plan_source].[input] }
      * { Tm1FilterByLevel( Descendants( [plan_time].[2004] ), 0) }
    ON COLUMNS
FROM [plan_BudgetPlan]
';
Either will achieve the variable assignment in the TABDIM order of the cube. From my perspective this is a pretty acceptable workaround as this is probably how we should be creating "ViewExtract" MDX views anyway (or how IBM assumed people would be creating MDX views for TI processing). I still think it's a defect that variables are ordered depending on the view structure and there should be a mechanism as for native views to ensure the variable ordering is consistent with the cube structure ordering.

Kudos to my very clever colleague who suggested this solution ;-)

Post back in the thread if this works for you and solves your variable assignment problem.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Dimension Order in an MDX cube view as datasource

Post by PavoGa »

lotsaram wrote: Mon Aug 05, 2019 4:08 pm Ty,

It seems that the ordering is dependent on the view structure
- First WHERE
- Then ROW
- Last COLUMN
We have seen this, but also, on a couple of queries, have seen an ordering that has no rhyme or reason to it. Has been rare, but has happened. Enough that we do not trust it and now write the TI's by doing an output to what order things are in.
Either will achieve the variable assignment in the TABDIM order of the cube. From my perspective this is a pretty acceptable workaround as this is probably how we should be creating "ViewExtract" MDX views anyway (or how IBM assumed people would be creating MDX views for TI processing). I still think it's a defect that variables are ordered depending on the view structure and there should be a mechanism as for native views to ensure the variable ordering is consistent with the cube structure ordering.

Kudos to my very clever colleague who suggested this solution ;-)
Ah! Thank you for the tip and I agree we probably should be writing the MDX for view extracts this way henceforth.
Ty
Cleveland, TN
Bakkone
Posts: 119
Joined: Mon Oct 27, 2014 10:50 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

Re: Dimension Order in an MDX cube view as datasource

Post by Bakkone »

Hi,

I haven't started with the MDX views yet as my MDX-skils are pretty crappy (Any tips on resources?).

But does an MDX cube view differ in a "normal" cube view when used as a data source when it comes to dimension orders? Exporting using a "normal" cube view uses the cube dimension order. Is this not the case when exporting an MDX view?
lotsaram
MVP
Posts: 3652
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Dimension Order in an MDX cube view as datasource

Post by lotsaram »

Bakkone wrote: Wed Aug 07, 2019 9:04 am But does an MDX cube view differ in a "normal" cube view when used as a data source when it comes to dimension orders? Exporting using a "normal" cube view uses the cube dimension order. Is this not the case when exporting an MDX view?
So I guess you (didn't) read the contents of the thread then?
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Bakkone
Posts: 119
Joined: Mon Oct 27, 2014 10:50 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

Re: Dimension Order in an MDX cube view as datasource

Post by Bakkone »

lotsaram wrote: Wed Aug 07, 2019 1:23 pm So I guess you (didn't) read the contents of the thread then?
I wish I could say I didn't. Maybe I read too much into it and that it doesn't matter at all, but that you want a predictable order of your variables.
lotsaram
MVP
Posts: 3652
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Dimension Order in an MDX cube view as datasource

Post by lotsaram »

Bakkone wrote: Wed Aug 07, 2019 1:51 pm you want a predictable order of your variables.
Indeed you do! And that's the point. With a MDX view as the data source you don't necessarily get the variables in the index order of the cube (unless you jump through a few hoops and construct the view in a particular way).
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Bakkone
Posts: 119
Joined: Mon Oct 27, 2014 10:50 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2013

Re: Dimension Order in an MDX cube view as datasource

Post by Bakkone »

So the MDX Views are not the magic I was hoping for to force my TI process to loop through the source in a way I want it to then. Thanks for the clarification.
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Dimension Order in an MDX cube view as datasource

Post by PavoGa »

Bakkone wrote: Wed Aug 07, 2019 1:59 pm So the MDX Views are not the magic I was hoping for to force my TI process to loop through the source in a way I want it to then. Thanks for the clarification.
That is an interesting conclusion drawn from lotsaram's post. Suspect that you are not fully understanding what we are saying.

In a traditional view (CreateSubset, CreateView, ViewSubsetAssign), we know the dimension order is going to be the order in which the cube is constructed or available through TABDIM (easily displayed in Architect as well). This order cannot be manipulated in a traditional view.

What lotsaram is saying is that in MDX, it does not default to that order. Where a Version dimension may be the first dimension in the cube and would read as the first variable in a traditional view, depending on how the MDX view is constructed, it may be read as the THIRD variable, appearing as though the Version dimension is the third one in the cube.

An MDX view does not change the data that is retrieved from the cube. It just may change the order in which it presents the dimensions.
Ty
Cleveland, TN
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Dimension Order in an MDX cube view as datasource

Post by PavoGa »

Update. We have noticed a possible pattern with regards to output order not matching dimension order when reading an MDX cube view:

We are observing the output order is by:

*** 08/12/2021 Updated this as the original, did not make a lot of sense ***
1) dimensions that are based on the default hierarchy (dimname:dimname) are grouped first by cube dimension order
2) any dimensions using alternate hierarchies in the MDX query are grouped together by their respective order in the cube and follow the group of dimensions described in #1.

In other words, if any dimensions in the MDX query use alternate hierarchies, the engine creates two groups, one of default hierarchies and one of alternate hierarchies. Within each group, the dimension variables are by the cube dimension order.
*** Update end ***

Say a cube has the following dimensions in this order:

1) Versions
2) Cost Center
3) Accounts
4) Calendar
5) Measure

and the view MDX is like this:

Code: Select all

SELECT
    [Measure].[subset] ON COLUMNS
    {[Version].[element]} *
    [Cost Center].[alt_CostCenter].[subset] *
    [Accounts].[alt_Accounts].[subset] *
    [Calendar].[subset] ON ROWS
FROM [SourceCube]
The order the TI outputs this cube is:

1) Versions
2) Calendar
3) Measure
4) Cost Center
5) Accounts

If this is the MDX (note Accounts is now based on the default hierarchy):

Code: Select all

SELECT
    [Measure].[subset] ON COLUMNS
    {[Version].[element]} *
    [Cost Center].[alt_CostCenter].[subset] *
    [Accounts].[subset] *
    [Calendar].[subset] ON ROWS
FROM [SourceCube]
then the output order becomes:

1) Version
2) Accounts
3) Calendar
4) Measure
5) Cost Center

This is what we have seen now in three different examples. Two, like above, are on the same cube. Going to IBM with this so if anyone happens to see the same pattern or not, would appreciate the feedback.
Last edited by PavoGa on Thu Aug 12, 2021 5:57 pm, edited 2 times in total.
Ty
Cleveland, TN
lotsaram
MVP
Posts: 3652
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Dimension Order in an MDX cube view as datasource

Post by lotsaram »

Do you have a direct line to Stuart King on this issue? I can't fathom why IBM seems to insist that this is a "non-issue". When a TM1 cube is the datasource for a TI process you need 100% consistent order of variable assignment (and that order should match the index order of the dimensions in the cube). Period.
There shouldn't be any impact of how the query is formed on the resulting data extract and the developer shouldn't have to jump through hoops to construct the MDX query in a certain way in order to achieve the correct variable assignment.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Dimension Order in an MDX cube view as datasource

Post by PavoGa »

lotsaram wrote: Wed Dec 09, 2020 9:30 am Do you have a direct line to Stuart King on this issue?


I do not have a direct line to Stuart King. As a contractor, I have to pass my findings/issues through regular staff, but they are knowledgeable and the firm has a substantial relationship with IBM.
I can't fathom why IBM seems to insist that this is a "non-issue". When a TM1 cube is the datasource for a TI process you need 100% consistent order of variable assignment (and that order should match the index order of the dimensions in the cube). Period.
There shouldn't be any impact of how the query is formed on the resulting data extract and the developer shouldn't have to jump through hoops to construct the MDX query in a certain way in order to achieve the correct variable assignment.
I could not agree more. This is frustrating. If the query for a TI is "fixed" and does not change other than parameter substitution, one can simply rearrange the variables to fit what the query is giving (assuming this is the pattern and not random!), but I ran into this (again!) writing a TI that does its work based on a mapping cube to permit a substantial amount of flexibility to the users for a particular application feature. Whether a dimension is queried on it's default or alternate hierarchy could change from cube to cube depending on usage.
Ty
Cleveland, TN
lotsaram
MVP
Posts: 3652
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Dimension Order in an MDX cube view as datasource

Post by lotsaram »

For anyone as equally as flummoxed as I am that fixing this doesn't seem to be a priority for IBM here is the defect which you can at least subscribe to to get an update when the status changes.

PH24729: WRONG VARIABLE ASSIGNMENT ORDER IN MDX-VIEW PROCESSING
https://www.ibm.com/support/pages/apar/PH24729
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
lotsaram
MVP
Posts: 3652
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Dimension Order in an MDX cube view as datasource

Post by lotsaram »

I just noticed that this is reported as fixed in Planning Analytics 2.0.9.10 (September 2021 release)
https://www.ibm.com/support/pages/syste ... _20910.pdf

So as long as you are on 2.0.9.10 or 2.0.9.11 (the latest version as of rght now at the time of writing) then this should be fixed.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Wim Gielis
MVP
Posts: 3113
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Dimension Order in an MDX cube view as datasource

Post by Wim Gielis »

Are AsciiOutput and TextOutput expected to be correct ?
I use them in the Data tab of a process with an MDX view as the source.
What is exported in the text file is not all what I was asking for.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Dimension Order in an MDX cube view as datasource

Post by PavoGa »

I have yet to test this since we upgraded. Sounds like it is not fixed however.
Ty
Cleveland, TN
ascheevel
Community Contributor
Posts: 287
Joined: Fri Feb 15, 2013 5:49 pm
OLAP Product: TM1
Version: PA 2.0.9.1
Excel Version: 365
Location: Minneapolis, USA

Re: Dimension Order in an MDX cube view as datasource

Post by ascheevel »

Was doing a bit of testing in 2.0.9.11 today and see the dim order for an mdx view as data source for TI will be the order specified in mdx statement. Seems we were expecting the order to be the TABDIM order. Funny thing is that on old 2.0.9.1 version (not 2.0.9.10), as long as I wasn't employing multiple hierarchies from the same dim, the dim order in the returned view would align to TABDIM regardless of the order in the mdx statement. From lotsa's link, the dim order was fixed in 2.0.9.10 but I didn't find anywhere what the expected behavior should be. I don't have a 2.0.9.10 version handy to test on to see if there was a change in behavior from .10 to .11/.12.
ascheevel
Community Contributor
Posts: 287
Joined: Fri Feb 15, 2013 5:49 pm
OLAP Product: TM1
Version: PA 2.0.9.1
Excel Version: 365
Location: Minneapolis, USA

Re: Dimension Order in an MDX cube view as datasource

Post by ascheevel »

I got confirmation from IBM that the variable order matching the order of dims in the query is expected behavior and not necessarily the TABDIM order as of 2.0.9.10.

edit: removed repetitive "expected behavior"
Last edited by ascheevel on Wed Mar 23, 2022 6:04 pm, edited 1 time in total.
Post Reply