Change to DimensionElementComponentAdd in PA vs 10.2?

Post Reply
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram »

In the latest TM1 server version 11.5.00000.23 we have detected a change in the behaviour of the TurboIntegrator function DimensionElementComponentAdd. In previous versions if the element supplied as the component to DimensionElementComponentAdd did not already exist in the dimension then it would be automatically added as a leaf element without need for explicitly using the DimensionElementInsert function to create the child element first.

This new stricter behaviour may cause any previously error free (and productive) processes to throw errors and no longer work.

This change might have come in an earlier 2.0.x and not in 2.0.7. Is anyone able to confirm this behaviour and which 11.x version it appeared?

Many people may have exploited the more lenient behaviour of DimensionElementComponentAdd so although dead easy to patch this is definitely something to be aware of.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Wim Gielis
MVP
Posts: 3105
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis »

I haven’t used that functionality that much in the past, I always specify 2 DimensionElementInsert and 1 DimensionElementComponentAdd functions. I can’t help with exact versions.

On a related note if I may. What I found out in the not so distant past is that if DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past - but I’m not100% sure.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram »

Wim Gielis wrote: Mon Jun 24, 2019 8:04 pmif DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past
Yes definitely that used to work. (Not actually a good thing given that converting elements from N to C results in data loss! Good that this has changed). My guess is that that this change was necessary as leaf elements are shared across all hierarchies and name conflict between leaves and consolidations in hierarchies is prohibited. Therefore as the element exists in the Leaves hierarchy it isn't possible to create a consolidation with the same name in another hierarchy. (BTW this also means that we need to be much more careful than before about specifying the correct element type on creation than the old days.)
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Wim Gielis
MVP
Posts: 3105
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis »

Thanks Lotsaram. The problem is for example a classic parent-child hierarchy structure. A is the parent of B (line 1) while B is the parent of C (line 2). But line 1 in the data source does not know about line 2. What should we do then ? Pass over the data source twice and use a temporary dimension to store the information of all records ?
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
User avatar
Steve Rowe
Site Admin
Posts: 2410
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Steve Rowe »

The second new behaviour, being unable to add an a child to an N level and so protecting the application from data loss is an "interesting" change. Whilst for many use cases it is sensible it makes other use cases much more challenging.

In our code base where we want to protect the environment from this we perform a test before the ComponentAdd and in our dimension maintenance tool an elements type is declared at the point of creation.

IBM really shouldn't be making this changes in such a hard way though, as Wim points out there is a very common use case where this change will break peoples existing code base. This is where you don't know if an element is a C or an N until you come to build the relationship that is defined in the data source. Typically this would be from a relational source. There is probably a lot of legacy code that is now going to start failing.

Rather than just changing the behaviour (and, I assume, not announcing it) it would be far better to add an optional 1/0 parameter to the end of the function. 0 being the default protective behaviour and 1 being the old behaviour. This would allow people to easily update their legacy code as required. If this come about as the results of a complaint / RFE they should have just been told to write their code properly and check for an elements type before performing the operation.

Now we need to know an elements type the first time we encounter it, this implies that we need to analyse the whole structure before creating it and so, (as Wim states) we need to process it twice.

Lotsaram, Re the point about hierarchies, this probably makes sense but there is a specific function for adding relationships to hierarchies, this is the one that needs this behaviour. Not the one that is for non-hierarchy legacy builds. It has been stated by IBM many times that the hierarchy functionality required no code changes until you activate it against a specific dimension. This would no longer seem to be the case.

Maybe the new behaviour only arise if hierarchies are allowed in the cfg? (I don't have a testbed to hand to check, sorry)
Technical Director
www.infocat.co.uk
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram »

The first one (automatic insert of element with implicit type N where component not known) appears to be a regression bug. It was reported in 2.0.1 and subsequently fixed, so my guess is this will also be fixed again in 2.0.7.

The 2nd case, as far as I know will only happen in the case of dimensions which contain hierarchies. For dimensions containing only the same names hierarchy and no Leaves hierarchy it should work just as before (but once a dimension has multiple hierarchies it's a no-go for the reasons explained).
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Wim Gielis
MVP
Posts: 3105
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis »

lotsaram wrote: Wed Jun 26, 2019 4:19 pmThe 2nd case, as far as I know will only happen in the case of dimensions which contain hierarchies. For dimensions containing only the same names hierarchy and no Leaves hierarchy it should work just as before (but once a dimension has multiple hierarchies it's a no-go for the reasons explained).
I tested this in 2.0.6 IF3.

If the dimension only contains the same-named hierarchy and a Leaves hierarchy, it works fine.
Once the dimension contains any "custom" hierarchy, it throws an error.

It does not matter whether EnableNewHierarchyCreation=T or F.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
User avatar
Steve Rowe
Site Admin
Posts: 2410
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Steve Rowe »

Thanks for testing, so legacy code will continue to work until hierarchies exist versus the dimension.

Good
Technical Director
www.infocat.co.uk
Wim Gielis
MVP
Posts: 3105
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis »

As Dimsiz of a non-existing dimension returns 0 rather than an error (*),
I'm thinking of testing whether the Dimsiz of 'Hierarchies_' | vDim exceeds 2 or not.
vDim is the dimension to be investigated.
If > 2, it will involve additional logic.

What do others think ?

(*) at least for the time being ;-)
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram »

Wim Gielis wrote: Thu Jun 27, 2019 7:41 am I'm thinking of testing whether the Dimsiz of 'Hierarchies_' | vDim exceeds 2 or not.
vDim is the dimension to be investigated.
If > 2, it will involve additional logic.

What do others think ?
I would create a temporary dynamic subset with MDX

Code: Select all

Expand('{TM1FILTERBYPATTERN( {TM1SUBSETALL( [}Dimensions] )}, "%vDim%:*")}')
and then test that the subset has >= 2 elements (not >=1 as Leaves will always exist if there are other hierarchies and even if all other hierarchies are deleted will remain and will be automatically removed on the next session start.)

I would do this rather than checking the }Hierarchies_ dimension as just say you have a custom "control dimension" (starting with "}") then although you can create hierarchies there will be no }Hierarchies_ dimension created.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
User avatar
PavoGa
MVP
Posts: 616
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by PavoGa »

lotsaram wrote: Mon Jun 24, 2019 2:45 pm
This change might have come in an earlier 2.0.x and not in 2.0.7. Is anyone able to confirm this behaviour and which 11.x version it appeared?

Many people may have exploited the more lenient behaviour of DimensionElementComponentAdd so although dead easy to patch this is definitely something to be aware of.
Ran headlong into this problem after moving to PA. Initially thought it was just the Hierarchy functions, but soon found out otherwise. Actually thought I'd posted about it, but could not find anything on it, so dropped the ball on that from months ago.
Wim Gielis wrote: Mon Jun 24, 2019 8:04 pm On a related note if I may. What I found out in the not so distant past is that if DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past - but I’m not100% sure.
Have not run into this, or if I did, have managed to avoid it since. :|
Ty
Cleveland, TN
Post Reply