TM1 Forum

Hi guys,
I'm using DimensionDeleteAllElements in Prolog and then re-create some of elements back in Metadata to preserve the data.

But what if there's something wrong in Metadata and the script fails? Is my entire dimension (and the data) gone forever?

If yes - is there any way how to avoid this?

BR
Vladino

vladino wrote:Hi guys,
I'm using DimensionDeleteAllElements in Prolog and then re-create some of elements back in Metadata to preserve the data.

But what if there's something wrong in Metadata and the script fails? Is my entire dimension (and the data) gone forever?

If yes - is there any way how to avoid this?

The way that a classic update of a dimension (that is, prior to the new REST API and the "Direct" TI functions) works is this:
- A copy of the dimension is created in memory;
- Updates are made to that copy in the Prolog tab and the Metadata tab;
- At the finish of the Metadata tab the dimension is validated for consistency and if that validation passes, then the copy of the dimension is registered to replace the real dimension.

Here's the problem; in my experience with the classic API a dimension with no elements will pass validation just fine.

So yes, if you delete all of the elements in the Prolog and your data source fails such that the elements are not added back in the metadata, it is conceivable that you will end up with a validated, empty dimension.

As you have envisaged, this will usually be A Very Bad Thing.

The solution? Never use the DimensionDeleteAllElements function for a data -bearing dimension. Ever. Not once. Never. Just don't. No, get your finger off it and step away from the console. Now. Usually, there is no need for it. You may need to strip down the consolidation tree but you don't need to strip out the N elements. And for mine, DimensionDeleteAllElements is just too damn dangerous.

The only time I ever use it is when the dimension is NOT used in a cube which stores data but is used as a pseudo array (since I'll win Powerball before we ever see real arrays in the TI language) and I'm just using it to store a collection of elements.

vladino wrote:But what if there's something wrong in Metadata and the script fails? Is my entire dimension (and the data) gone forever?

I wrote a generic TI process that can go through all processes' PRO files, and listing (in a control cube) many properties, line counts, parameters, ...
Also, it detects the presence of statements like DimensionDeleteAllElements, ExecuteProcess, and so on.
My clients can get a list of processes where this function exists. I use the function exactly like Alan mentions,
but if the client is using the TI wizard or writes his own scripts, I can signal this.

Best regards,

Wim

Wim Gielis wrote:...but if the client is using the TI wizard...

... then his eternal soul is condemned to purgatory, and may the coding gods show mercy.

Alan Kirk wrote:You may need to strip down the consolidation tree but you don't need to strip out the N elements..

Sounds like a good enhancement request. It would be handy to have a DimensionDeleteAllHierarchy as a easier alternative than coding it up yourself.

Matt

mattgoff wrote:

Alan Kirk wrote:You may need to strip down the consolidation tree but you don't need to strip out the N elements..

Sounds like a good enhancement request. It would be handy to have a DimensionDeleteAllHierarchy as a easier alternative than coding it up yourself.

Matt

+1

Tilting at windmills.

http://www.tm1forum.com/viewtopic.php?f=19&t=13369

https://www.ibm.com/developerworks/rfe/ ... _ID=106604

It would be handy to have a DimensionDeleteAllHierarchy as a easier alternative than coding it up yourself.

Yes but having the option of excluding the very top consolidation level would be nice, if somewhat wiping our backsides!

I wrote a generic TI process that can go through all processes' PRO files

Would I be rude in asking to see that?!

Mark RMBC wrote:

It would be handy to have a DimensionDeleteAllHierarchy as a easier alternative than coding it up yourself.

Yes but having the option of excluding the very top consolidation level would be nice, if somewhat wiping our backsides!

Which gets you what, exactly? All you're left with is a C type element which has no children. If all of the other consolidation elements have been destroyed and need to be recreated, one extra consolidation node makes no difference. There is nothing special about a top consolidation node compared to any other one, especially as in any dimension with multiple trees there may be multiple "very top level" consolidations. And just to put the cherry on top of that cake, an element which is the "very top level" consolidation in one hierarchy may not be in another one.

Mark RMBC wrote:

I wrote a generic TI process that can go through all processes' PRO files

Would I be rude in asking to see that?!

Voilà, there you go. 1,900 lines of coding

Unedited these processes will need at least TM1 10.2.2 FP4.

The MAIN process executes the 2 other processes.

Any questions / remarks / errors / improvements: let me know.
Certainly the process to analyse chores is new, all input is welcome.

Among (many) other things, these functions can list which processes execute other processes, how many times, in what tab of the process, etc.

EDIT June 18, 2017;

Attachments were removed and now I created a new topic here:
http://www.tm1forum.com/viewtopic.php?f=21&t=13373
The files in that topic are the latest versions.

mattgoff wrote:Tilting at windmills.

http://www.tm1forum.com/viewtopic.php?f=19&t=13369

https://www.ibm.com/developerworks/rfe/ ... _ID=106604

Happy 500th. I've given you another vote AND a watchlist addition, which apparently matters.

Of course, I was only able to do that after I followed the link above, hit the obligatory page which says:

Oops, that's not right!
Sorry, the page you're looking for cannot be displayed

Please try again later.

which both lies to you (yes, it can be displayed) and misleads you ("trying again later" is not the issue), logged in despite not being prompted to, was returned not to the RFE but to the cutsie little "Oops, that's not right" page, returned to this thread, clicked on the link again, and this time, because I was already logged in, went to the RFE.

All these years, and they still haven't fixed that.

"That's not right" indeed. In a perfect world anyone who was capable of such absolutely cr@p website coding would not be let anywhere near a keyboard, much less a production environment.

At present I just use a generic process for this. I would have written my own but since I already have the Bedrock processes installed Bedrock.Dim.Hierarchy.Unwind.Consolidation and Bedrock.Dim.Hierarchy.Unwind.All work well enough, so I just use those.

Alan Kirk wrote:At present I just use a generic process for this. I would have written my own but since I already have the Bedrock processes installed Bedrock.Dim.Hierarchy.Unwind.Consolidation and Bedrock.Dim.Hierarchy.Unwind.All work well enough, so I just use those.

Bedrock is awesome (with the caveat that a huge portion of what makes it awesome is that it plugs huge holes in the TI language), but it sooo cluttered up my process list, and I didn't really use much of it, so I got rid of it. I wish there were a couple levels of Display Control Objects where I could prefix something with a symbol, say {, or set a flag a la Enable Security Access to hide it in the GUI. Or maybe even if they allowed folders in places other than Applications.

I wish they would fix all this stuff before they invested in refactoring TM1 for containers and TM1 web for Tomcat. I DO appreciate some of the recent advancements, like auto-destructing subsets and views, MTQ, and persistent feeders.

Matt

mattgoff wrote:

Alan Kirk wrote:At present I just use a generic process for this. I would have written my own but since I already have the Bedrock processes installed Bedrock.Dim.Hierarchy.Unwind.Consolidation and Bedrock.Dim.Hierarchy.Unwind.All work well enough, so I just use those.

Bedrock is awesome (with the caveat that a huge portion of what makes it awesome is that it plugs huge holes in the TI language), but it sooo cluttered up my process list, and I didn't really use much of it, so I got rid of it. I wish there were a couple levels of Display Control Objects where I could prefix something with a symbol, say {, or set a flag a la Enable Security Access to hide it in the GUI. Or maybe even if they allowed folders in places other than Applications.

Ayup. I think this thread about grouping objects into folders (from 6 years ago now) pre-dated the RFE system but it was hardly the first time it was brought up. Of course it also pre-dates the "we ain't doin' nothin' with Perspectives ever again" policy, so there isn't much point bringing it up as a formal RFE now.

At least with Bedrock the processes all sit together so I've learnt to just skim over them. The main reason that I've stuck with them is:
(a) I don't need to reinvent the wheel;
(b) Once I get used to them at least I know how they work, and they're already more or less documented (though some of the documentation is flat out wrong and was clearly written for an earlier iteration of the process). When I write my own stuff then something that seemed like a good approach one month may not seem so the following month. So either I sit down and write my own complete and full set of processes which behave consistently (which would seem to be a bit redundant when Bedrock is already there), or I end up with a bunch of generic processes that were written over time and are subtly different in how they act.

That said, I do indeed supplement it with my own generic processes and have "rewritten" one or two using the name BedrockX. Off the top of my head (I'm not connected to TM1 at the moment) I think one of those was where the process didn't have a parameter to disable logging when doing a large data movement, which resulted in multi-gigabyte log files. I therefore needed an incarnation which had the relevant parameter. Now that Bedrock has been moved to Github I may look at doing a branch for such things when I have the time.

mattgoff wrote:I wish they would fix all this stuff before they invested in refactoring TM1 for containers and TM1 web for Tomcat.

Who needs basic usability when you can point to Cutting! Edge! and (in the case of Docker) Flavour! Of! The! Month! development technology?

Though I'll admit that I've come around a little more to the Tomcat thing; I think in the longer term it's going to be a better idea than hitching the wagon to IIS if for no other reason than improved platform flexibility. And certainly the display of websheets seems to me to be a damn sight better than the "guess what I'll look like today" crayon drawings that you sometimes get in ye olde 1001 Excel Sessions method that pre-dated Tomcat.

mattgoff wrote:I DO appreciate some of the recent advancements, like auto-destructing subsets and views, MTQ, and persistent feeders.

Agreed.

Wim – thanks for this, much appreciated.

Alan – Well I was thinking (probably not too clearly) that a deleteallhierarchy function would mean that any of the hierarchy that I create additionally would also be deleted when in fact I would want to keep those in place no matter what. I was thinking about when I link to an ODBC source, use SQL to build the dimension but then in the TI add in a few levels, usually high up to complete the dimensional hierarchy. So for example on a chart of accounts cost centre dimension I added in at the very top a level for the structure being used and underneath this top level I added in a level to capture Total Expenditure and Total Income.

So I wondered if this proposed function deleteallhierarchy could in some way have a parameter to exclude level(s) but then I narrowed it down to the top level because I thought adding in multiple levels to a parameter would be too complex.

Though I then thought, what the hell just go and recreate the additional levels in the TI because how much work would that be, but still thought I would add in the proposal anyway. What I was really after was a link to Wim’s code!

I know I'm a bit late to the party, but I think it's also worth noting that as of PA 2.0 you can explicitly ask TI to rollback and discard all work.

https://www.ibm.com/support/knowledgece ... lback.html

You'd still have to detect the metadata error in your TI code and then ask for a rollback to prevent the dimension destruction, but at least it's possible.