TM1 Forum

In the latest TM1 server version 11.5.00000.23 we have detected a change in the behaviour of the TurboIntegrator function DimensionElementComponentAdd. In previous versions if the element supplied as the component to DimensionElementComponentAdd did not already exist in the dimension then it would be automatically added as a leaf element without need for explicitly using the DimensionElementInsert function to create the child element first.

This new stricter behaviour may cause any previously error free (and productive) processes to throw errors and no longer work.

This change might have come in an earlier 2.0.x and not in 2.0.7. Is anyone able to confirm this behaviour and which 11.x version it appeared?

Many people may have exploited the more lenient behaviour of DimensionElementComponentAdd so although dead easy to patch this is definitely something to be aware of.

I haven’t used that functionality that much in the past, I always specify 2 DimensionElementInsert and 1 DimensionElementComponentAdd functions. I can’t help with exact versions.

On a related note if I may. What I found out in the not so distant past is that if DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past - but I’m not100% sure.

Wim Gielis wrote: ↑Mon Jun 24, 2019 8:04 pmif DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past

Yes definitely that used to work. (Not actually a good thing given that converting elements from N to C results in data loss! Good that this has changed). My guess is that that this change was necessary as leaf elements are shared across all hierarchies and name conflict between leaves and consolidations in hierarchies is prohibited. Therefore as the element exists in the Leaves hierarchy it isn't possible to create a consolidation with the same name in another hierarchy. (BTW this also means that we need to be much more careful than before about specifying the correct element type on creation than the old days.)

Thanks Lotsaram. The problem is for example a classic parent-child hierarchy structure. A is the parent of B (line 1) while B is the parent of C (line 2). But line 1 in the data source does not know about line 2. What should we do then ? Pass over the data source twice and use a temporary dimension to store the information of all records ?

The second new behaviour, being unable to add an a child to an N level and so protecting the application from data loss is an "interesting" change. Whilst for many use cases it is sensible it makes other use cases much more challenging.

In our code base where we want to protect the environment from this we perform a test before the ComponentAdd and in our dimension maintenance tool an elements type is declared at the point of creation.

IBM really shouldn't be making this changes in such a hard way though, as Wim points out there is a very common use case where this change will break peoples existing code base. This is where you don't know if an element is a C or an N until you come to build the relationship that is defined in the data source. Typically this would be from a relational source. There is probably a lot of legacy code that is now going to start failing.

Rather than just changing the behaviour (and, I assume, not announcing it) it would be far better to add an optional 1/0 parameter to the end of the function. 0 being the default protective behaviour and 1 being the old behaviour. This would allow people to easily update their legacy code as required. If this come about as the results of a complaint / RFE they should have just been told to write their code properly and check for an elements type before performing the operation.

Now we need to know an elements type the first time we encounter it, this implies that we need to analyse the whole structure before creating it and so, (as Wim states) we need to process it twice.

Lotsaram, Re the point about hierarchies, this probably makes sense but there is a specific function for adding relationships to hierarchies, this is the one that needs this behaviour. Not the one that is for non-hierarchy legacy builds. It has been stated by IBM many times that the hierarchy functionality required no code changes until you activate it against a specific dimension. This would no longer seem to be the case.

Maybe the new behaviour only arise if hierarchies are allowed in the cfg? (I don't have a testbed to hand to check, sorry)

The first one (automatic insert of element with implicit type N where component not known) appears to be a regression bug. It was reported in 2.0.1 and subsequently fixed, so my guess is this will also be fixed again in 2.0.7.

The 2nd case, as far as I know will only happen in the case of dimensions which contain hierarchies. For dimensions containing only the same names hierarchy and no Leaves hierarchy it should work just as before (but once a dimension has multiple hierarchies it's a no-go for the reasons explained).

lotsaram wrote: ↑Wed Jun 26, 2019 4:19 pmThe 2nd case, as far as I know will only happen in the case of dimensions which contain hierarchies. For dimensions containing only the same names hierarchy and no Leaves hierarchy it should work just as before (but once a dimension has multiple hierarchies it's a no-go for the reasons explained).

I tested this in 2.0.6 IF3.

If the dimension only contains the same-named hierarchy and a Leaves hierarchy, it works fine.
Once the dimension contains any "custom" hierarchy, it throws an error.

It does not matter whether EnableNewHierarchyCreation=T or F.

Thanks for testing, so legacy code will continue to work until hierarchies exist versus the dimension.

Good

As Dimsiz of a non-existing dimension returns 0 rather than an error (*),
I'm thinking of testing whether the Dimsiz of 'Hierarchies_' | vDim exceeds 2 or not.
vDim is the dimension to be investigated.
If > 2, it will involve additional logic.

What do others think ?

(*) at least for the time being

Wim Gielis wrote: ↑Thu Jun 27, 2019 7:41 am I'm thinking of testing whether the Dimsiz of 'Hierarchies_' | vDim exceeds 2 or not.
vDim is the dimension to be investigated.
If > 2, it will involve additional logic.

What do others think ?

I would create a temporary dynamic subset with MDX and then test that the subset has >= 2 elements (not >=1 as Leaves will always exist if there are other hierarchies and even if all other hierarchies are deleted will remain and will be automatically removed on the next session start.)

I would do this rather than checking the }Hierarchies_ dimension as just say you have a custom "control dimension" (starting with "}") then although you can create hierarchies there will be no }Hierarchies_ dimension created.

lotsaram wrote: ↑Mon Jun 24, 2019 2:45 pm
This change might have come in an earlier 2.0.x and not in 2.0.7. Is anyone able to confirm this behaviour and which 11.x version it appeared?

Many people may have exploited the more lenient behaviour of DimensionElementComponentAdd so although dead easy to patch this is definitely something to be aware of.

Ran headlong into this problem after moving to PA. Initially thought it was just the Hierarchy functions, but soon found out otherwise. Actually thought I'd posted about it, but could not find anything on it, so dropped the ball on that from months ago.

Wim Gielis wrote: ↑Mon Jun 24, 2019 8:04 pm On a related note if I may. What I found out in the not so distant past is that if DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past - but I’m not100% sure.

Have not run into this, or if I did, have managed to avoid it since.