Object locking model

[Steve Rowe]

Afternoon,
I'm trying to track down a document that describes the object locking model of TM1 and how it changed between version 9.0 and 9.1.

Does anyone know where the document might be or more likely if it even exists? I've spent all morning on a rather tedious search of the IBM site and can't find anything.

I did find this statement under a section in document about changes to documentation in 9.1, for me this is a classic example of fixing a bug by changing the documentation. Anyway it might be a handy piece of information for someone.

The SaveDataAll Function Needs to be Called after Metadata Changes are Made to a View in a TurboIntegrator Process
Metadata changes made to a view by a TurboIntegrator function, such as assigning a subset to a view, are not automatically saved. You must explicitly save your changes or the changes will be lost when the server is restarted.
To save the changes, you can use the SaveDataAll() function in the TurboIntegrator process, use the Save Data menu option in the TM1 Server Explorer, or choose to save changes when closing down the server.
This applies to all TurboIntegrator functions that can make metadata changes to a view, such as ViewSubsetAssign, ViewTitleDimensionSet, and ViewTitleElementSet.

Source document link
http://support.cognos.com/supported/doc ... 1sp2u3.pdf

[David Usherwood]

In your place I would raise a formal SR for this. It is a very reasonable request, indeed crucial. If you do and you get something, try to get clearance to make it available here. There has been so much flying around about the locking model the community really needs something solid to get its teeth into.

Having said that, Scott of Cubewise seems to understand the new model well - have a look at his posts on the subject.

Our view remains that 9.4 is not yet ready for serious use, but I'm starting to become concerned that we and our clients are being left behind on other enhancements. It would be good to get back in line.

A client of ours who upgraded to 9.4 when they went O2007 and found that their existing process worksheets ran 30 times slower.

[Steve Rowe]

I thought support for process worksheets had been dropped a whie back now?

My understanding is that the write lock has now moved from the server level to the cube level, though I'd really like to see some official documentation that backs this up (I have a SR in).

If so my view is this means a big change in the approach to desgin of systems if you want to take advantage of this feature. Atypical budget consolidation application would now have one entry cube per entity and then a rule based consolidation cube that contains the entity dimension. This is big change in architecture, in previous releases we would usually build just the large cube and hace everybody enter the data into this.

Anyone else have any views on what the changes will mean to the way we design systems. I can certainly see systems becoming more fragmented, which I'm not sure is a good thing from a users point of view.

Cheers,

[John Hobson]

My understanding is that as of 9.1 the locking model is now at the object (e.g. cube) level.

As you say, given the effect of locks when planning cubes are written to, this has some implications for system design and performance tuning.

Recently with one client we created a mirror cube that allowed one set of users to view management information as a t Monday morning whilst the planners beavered away on reforecasting. A bit of a cludge but worth it for the performance gain.

What makes you say that support for processing sheets has been withdrawn - I do hope not as they can still be rather useful. In my 9.1 sp2 theoptions are still there on the menu - are they gone in the current release?

[David Usherwood]

I think this is on the right track (and is consistent with Scottw's comments) however I don't think it will work with microcubes which are ruled into a central cube. This is because the feeders would fire and cause the central cube to be invalidated. My belief is that a TI is required to transfer the numbers. Not, I agree, the way which we would all like to see, but (in my view) doable. Scottw, do you agree?

[ScottW]

That's definitely correct for 9.1.3 and 9.1.4.
Cubes must have no relationship in order to take advantage of granular locking. Therefore you can't suck in data entry from sub-cubes via rule, TI has to be the mechanism. However note the distinction between write and read locks, even if data entry is going on other users should still be able to read (eg refresh a view).

The intent of the locking model was for the locking to be able to sense the "directionality" of a rule and write lock downstream cubes during data entry in an upstream cube but not visa versa. At least in 9.1 this did not seem to have been achieved and locking was bi-directional. They may well have refined this in 9.4 but I agree technical documentation is lacking in this area.

[mattgoff]

If so my view is this means a big change in the approach to desgin of systems if you want to take advantage of this feature. Atypical budget consolidation application would now have one entry cube per entity and then a rule based consolidation cube that contains the entity dimension.

Do you have a problem with cube locking as a result of end-user write activity? I've only seen issues with locking during long TI load processes-- since TI seems to lock the cube as soon as it starts an ODBC query, slow sources cripple the system much more than the qty of data should imply. For example, during close our GL can take a minute or two before the first row of data is returned even though rows 2 to whatever come across pretty quickly. Our GL cube in TM1 is locked from the moment the process is executed. As a result, I rarely load directly to my user-visible cubes now; instead, I load to a staging cube then copy cube-to-cube.

Matt

[Steve Rowe]

Matt,
I think the area that tends to be the issue is during forecast submissions. If you have a dozen people doing DBS (or other) submission of values to a single cube, you get a lot of read locks on the cube and of course it is calculating results constantly too. The big problem here when you have a lot of end-users submitting data is that the quality of the users network comes into play.

What's the definition of no relationship?

Say you have budget input cubes a , b and c, all rule linked to a central cube D that does the consolidation. I get that b is related to D by the rule and feeder relationship. So if you write to cube b then cubes b and D are locked and cached results are discarded. What about cubes a and b that are related in some sense via cube D but not dependant? Are they related and if so does anyone have any experimental evidence or documentation to back this up?

If the new level of object locking drives you towards linking cubes with a TI processes then I'm not sure I get the point. The big USP of TM1 was allways it's live and dynamic calculation speed. It seems that the new object locking model drives you away from this.

I guess this is not a problem provided we do not lose on general performance side so that a rule linked model performs just as well in old and new style object models.

Cheers,

[mattgoff]

Ah, OK I guess if you have a lot of WAN users or huge submissions that would be an issue. BTW, not sure if my last post was clear on this, but my scheme is only for data pulled in from our GL, not for any user-entered data. Instead of writing actuals directly to the user-accessible cube, I pull them in to a hidden cube that has no rules, in nor out. As ScottW described, I can't use rules as it would lock the user-accessible cube. After it's loaded into TM1, I have a second process that copies cube-to-cube (which is really, really fast).

Matt

[Steve Vincent]

Yeah i get you, you're using an intermedary cube that gets locked as your source is slow to pull the data in, but that avoids the lock on standard users just browsing the cube. The 2nd TI seems like a waste of effort at first glance (repeating what the 1st one does), but in fact its protecting your users from a full lock and thus keeping their TM1 experience a positive one.

From Steve's model that's not really an option. Users enter data and want to see immediate results. Either use rules to do that (and so lock the cubes more often that you'd like) or create a way for the data to be entered and only consolidated thru a TI. I think it comes down to how the users are inputting those numbers, if its Excel based templates then a "send to" button that does all that without them knowing is a good method. If its direct cube input you're really stuck - they either have to live with the locks and slow responses or deal with manually running a TI / scheduling it to run every so often. Neither of those last ones i'd particulary like...

[David Usherwood]

Steve, did you get anything sensible from Cognos? This is a pretty crucial issue for the community and they had better come up with something.

[Steve Rowe]

Hi David.
I had a mail on the 13th saying they were looking but no response other than that....

Cheers,

[Martin Ryan]

If so my view is this means a big change in the approach to desgin of systems if you want to take advantage of this feature. Atypical budget consolidation application would now have one entry cube per entity and then a rule based consolidation cube that contains the entity dimension. This is big change in architecture, in previous releases we would usually build just the large cube and hace everybody enter the data into this.

Anyone else have any views on what the changes will mean to the way we design systems. I can certainly see systems becoming more fragmented, which I'm not sure is a good thing from a users point of view.

Old post, but I've only just come across it via a link from another post and it ties into the poll that's currently running.

I'm not sure I see the disadvantage here. Previously the whole server got locked when something was happening. Now only one object gets locked. From what I can tell that doesn't lead to any worse performance in an existing model, it merely means that if you have big problems with the responsiveness of a system then you have another option in building the model, as per your design suggestions Steve.

I don't think this will change the way I'll design any systems. The original design methodology still works and from what I make of the posts above, hasn't got any worse - there's no reason to abandon that one mega-cube option.

For me this change (theoretically at least) is a vast improvement if you have multiple models running on the one server, which I frequently do. This means that these models will perform better than previously as they will not impact on one another.

I certainly wouldn't throw the baby out with the bath water and say it's a bad thing to have independent locking (once it actually works of course) merely because it means that a mega-cube is now not necessarily the fastest way of doing things. 9 times out 10 it'll still be the simplest/cleanest/best, and the new locking methodology will have benefits.

Martin

[Steve Rowe]

If we take things at face value I mostly agree with you, but....

My understanding is that 9.4 is slower with the mega cube approach? Anyone care to comment who is on 9.4, this might have been a 9.1 thing that has been corrected.
Say I rebuild my model for 9.4 and then when 9.5 comes out IBM have fixed the directional locking problem, I've wasted my time. I don't think we have any information in this area?
My design approach does not work, as soon as there are any rules connecteting the cubes they become related, forcing all cube connections to be done with TI. I _believe_ this is true even if the cubes indirectly connected.
Total lack of offical docs from IBM on how the locking works.

Cheers,

[jonathan.d]

Hi Everyone,

From everything posted, I feel this post on the object locking model of tm1 is most crucial.

I can't seem to understand why IBM wouldn't dish out information on the same. Don't they want to push tm1?

Other alternative, the community come up with a document on how TM1 handles locking.

[Alan Kirk]

From everything posted, I feel this post on the object locking model of tm1 is most crucial.

I hadn't looked at the locking model all that closely because it had never been particularly critical for us. I just accepted the fact that when people wrote it would affect the speed of people reading (notably reading the values that were changing), and possibly require recalculations of consolidations to be done. Our "couple of big cubes" approach has performed well enough that I haven't really had to delve into it, particularly as I've kept our biggest cube rules-free.

However I have been looking at it over the past few days, and... I have to agree with the post that Martin made earlier in this thread. I'm not seeing it as a huge deal since the change in the locking model, as I understand it, isn't going to make your performance any WORSE... it just gives you more options on how you might be able to make it BETTER.

I can't seem to understand why IBM wouldn't dish out information on the same. Don't they want to push tm1?

Other alternative, the community come up with a document on how TM1 handles locking.

I haven't done a lot of tests on 9.1/9.4 yet (though I plan to out of curiosity if nothing else), but if the locking isn't yet working the way it was originally envisaged (particully with rule directionality, as some of the posts in this thread suggest) there my be a reason why they're being a bit reticent about it... at least until they get the bugs shaken out of it. But I do agree with Steve et al... the more we know about not only what the situation is now, but what the plans are for the future, the better. That having been said, if they don't KNOW that they can get things right in 9.5 (or a later SP of 9.4), they probably won't want to go on record as promising it.

[jonathan.d]

Thanks for that Alan!

And your right, don't fix what's not broken!

But it's just something that would put a lot of minds at ease.

To recap, are we saying...

For example,
If you have two cubes Cube1: Reporting & Cube2: Entry, both linked via rules where in
Entry 'feeds' Reporting

Is it better to pass vaules from Entry to Reporting(assuming the reporting cube doesn't have any rules do further manipulate the numbers) throught a TI process? As, with the rule approach from everything what was said with people WRITING data into the Entry cube would lock out people wanting to READ data from the Reporting cube.

However, if the Reporting cube did have rules to manipulate the numbers further after they came in from the Entry cube, then there is no other way and users would just have to deal with the delay due to locking.

Do I have it right?

[Alan Kirk]

Thanks for that Alan!

And your right, don't fix what's not broken!

Buh?? Where did I say that??

Oh wait, now I see what you mean... just for clarity, I wasn't referring to "fixing" the locking model, only to our own (and, I suspect, a fair number of people's) model implementations. This change won't make much of a difference to us as things stand. In agreeing with Martin I was indicating that shifting locking from server level to object level as of 9.1 gives you greater latitude in how you design your model. It may benefit some people, and make no real difference to others.

Specifically, the What's New document in 9.1 stated that one of the changes was:

2. More granular Object-Level Write Locks to improve user concurrency performance
2.1. Updates will not impact unrelated TM1 objects

The problem is that there seems to be something of a shortage of information on how, exactly and in technical detail, that works.

It's interesting to note that the TI Guide in 9.1 refers to the following Appendix:

Appendix B “Locking Considerations in TurboIntegrator” describes how to manage server locking during TurboIntegrator process execution to optimize performance and maximize the amount of time the server is available to users.

Except... the entire appendix is missing.

The 9.4 TI manual doesn't even mention the mysterious Appendix B, and simply renamed Appendix C as Appendix B. Clearly there is something about it that they're not quite ready to talk about at this point.

I have a vision of the manual writer doing a Basil Fawlty impersonation; "Don't mention the locking model!"

But it's just something that would put a lot of minds at ease.

Yup, that's Steve's point exactly and it's a singularly accurate one. Certainty is a good thing.

To recap, are we saying...

For example,
If you have two cubes Cube1: Reporting & Cube2: Entry, both linked via rules where in
Entry 'feeds' Reporting

Is it better to pass vaules from Entry to Reporting(assuming the reporting cube doesn't have any rules do further manipulate the numbers) throught a TI process? As, with the rule approach from everything what was said with people WRITING data into the Entry cube would lock out people wanting to READ data from the Reporting cube.

However, if the Reporting cube did have rules to manipulate the numbers further after they came in from the Entry cube, then there is no other way and users would just have to deal with the delay due to locking.

Do I have it right?

Probably best to take a look at Scott W's post in relation to that; it covers it quite well. But if you do populate Reporting via TI, I don't think that the fact that Reporting may have internal rules (that is, rules which don't link to other cubes) would have any effect on locking as such. When you run the process to update Reporting it would probably mean that the rule-based cells would need to be recalculated, but that doesn't have anything to do with locking.

[David Usherwood]

Without hijacking this thread, I'd like to hear from forumers views on locking versus recalc, how they interact and what affects performance (also across versions). I'll save my views until others have put their own forward, though you can find them quite easily if you search .

[Steve Rowe]

I'm actually more interested in the way the cache of calculated results is preserved as I think in the heavy business modelling apps that I prefer to work in this can be more important than locking (lower user base, more calc cycles). My understanding is that pre 9.1 the cache was discarded if something changed at the server level and post 9.0 it was at the "related cubes" level.

Just like the locking...

Not sure if thsi is actually true of course.