Can TI's overwrite dimension structure changes (elements) or delete data sent via DBSS?

[wigglyrat]

Hi -

I am very new to TM1 and wanted to ask a question out of interest.

I recently loaded up data via DBSS. In order to do this I had to create a new lowest level element in one of the dimensions, i.e. right click, edit dimension structure.

However at a later stage, the data dissappeared, including that one element that I had added in order to be able to send the data via DBSS.

Apologies for the newbie quesion, if there is a TI that is run that updates this cube, could it possibly overwrite the new element that was added and/or delete the data?

Many thanks in advance.

[Wim Gielis]

Hello

This is possible but in fact, we almost never delete leaf-level elements.
These elements are used when TM1 stores data in the .cube files. So deleting the elements means / could mean a loss of data.
You should have a look at the process itself and for example DimensionDeleteAllElements is a common source of many headaches.

[wigglyrat]

Yes, thanks - DimensionDeleteAllElements is in the TI for that particular dimension.

Then there is

# re-add 'All Sales People' hierarchy
DimensionElementInsert( s_SalesPersonNo, '', 'All Sales Numbers', 'C');

Would that low level element need to be added here for it to not be affected?

thank you again.

[tomok]

I almost can't believe you're asking this question. What do you think a function called DimensionDeleteAllElements does? It deletes all the elements from a dimension (including the one you added manually) meaning there is absolutely NOTHING left after it is done.

[Alan Kirk]

Yes, thanks - DimensionDeleteAllElements is in the TI for that particular dimension.

Then there is

# re-add 'All Sales People' hierarchy
DimensionElementInsert( s_SalesPersonNo, '', 'All Sales Numbers', 'C');

Would that low level element need to be added here for it to not be affected?

I can understand your confusion.

Some might not, but I can.

Here's the piece of information that you're missing because it isn't spelt out well or clearly in the manuals.

When you update a dimension by TI (prior to 10.2.2 when a way was introduced to update the dimension directly) this is what would happen:
(a) A copy of the dimension is created in memory;
(b) Any actions that you do in Prolog or Metadata will affect that copy. That includes the DimensionDeleteAllElements command. This is deleting all of the elements in the dimension copy, which is why you don't lose all of the data that is stored in those elements right away. (A thing that I know puzzles a lot of new users.)
(c) At the end of the Metadata tab the dimension copy is checked for consistency and if everything is OK, then the copy overwrites the original dimension.

So as you can see, when the DimensionDeleteAllElements command is executed it deletes all of the elements in the dimension copy, but the TI will only add back the ones that come from the data source. And that will not include the one that you manually added.

So one solution to this is to have the TI insert your element but no later than the Metadata tab. A second is to add it to the data source for the dimension.

A third, and by far the best, is to follow Wim's advice and modify the TI so that it does NOT do a DimensionDeleteAllElements. There is rarely (not never, but rarely) a good reason for using that command. Often it's done to break down the consolidation hierarchy and rebuild it but there are ways of doing that without deleting all of the elements in the dimension. More to the point if anything happens part way through your Metadata tab such that not all of the records are received, you could end up with an incomplete but still valid dimension copy replacing your live one, meaning massive data loss. It's unlikely, but possible.

By omitting that command from the TI, all of the N elements, including the one you added, will remain in the updated dimension.

Incidentally, it occurs to me that it's possible that you didn't hand write the code but rather relied on the "Wizard" dialogs; the ones which generate the code inside the "Generated Code" blocks of a TI. That thing is quite partial to the DimensionDeleteAllElements command when updating a dimension. You should be aware that the wizard generates more cr@p code than one would have thought humanly possible, and you'll need to get the code out of there by setting all of your variables to "Other" and replace it with hand crafted code.

[wigglyrat]

Thank you for your very clear explanation. This was incredibly helpful for me and explained it perfectly!