How to prevent data losing for cube

[LP_user]

Dear Cognos TM1 users,

I have one general question about the concept of cube and dimension. The data of the cube will be lost when changing the structure of dimension. How to prevent it? Here is one example: product dimension has the following structure:
product--red(r1,r2,r3)
--blue(b1,b2,b3)
--green(g1,g2,g3)

and the cube has the value like:
red r1 r2 r3 blue b1 b2 b3 green g1 g2 g3
value 33 10 11 12 12 3 4 5 21 6 7 8

Here the value for red is the sum of r1,r2 and r3. The value for blue is the sum of b1,b2 and b3. The value for green is g1, g2 and g3.
If the structure of product is changed to

product--red(r1--b1,r2,r3)
--blue(b2,b3)
--green(g1,g2,g3)
It means to move b1 to r1 so that b1 is the child element of r1. Then the cube will be

red r1 --b1 r2 r3 blue b2 b3 green g1 g2 g3
value 26 3 3 11 12 9 4 5 21 6 7 8

The result is that the original value of r1 (10) is gone. Even I change the structure of product back to the original, the value of r1 is still null.
How can I recover the original value of cube? The only way I know is to hard copy of the whole TM1 file structure before the change of the dimension. If the change of dimension is wrong, restore the whole TM1 structure from the backup.

I would like to ask if there is better way to prevent data losing for cube?

With best regards,

LP_user

[David Usherwood]

If you are doing major, untested changes to your TM1 app then yes, you ought to take a snapshot of the server folder first. Since TM1 is a RAM resident engine the server folder is normally very small. Make sure you do a SaveDataAll first.

[cgaunt]

Hello LP_User

What you are describing does not seem to make sense for typical TM1 behaviour. TM1 stores the values of the children in memory and then on the underlying file system exactly as David describes. Assuming that you problem statement refers to a set of elements and their respective colour roll up on a single dimension, then changing the roll up of elements into different parents should not cause the values to disappear. You should see the incremental value of 'b1' go into the 'Red' total. I somewhat suspect that there is something else in the design of what you have or how you are amending the dimension that is causing you a problem.

For example, if you use the dimension editor to move the required element, then the values should be retained. If you first delete 'b1', save and then add 'b1' to the red total, then the values will be lost. It sounds like the way you are adjusting the dimension is causing r1 to either drop out of the red consolidation, or be deleted totally. Sorry to say that this is likely a user error.

To your other question about restoring data, You can restore data from the transaction log, but remember that deleting an element will not create a transaction on the log. You will have to search for the last entry made for the data that you are looking for. You will then need to manually build a recovery template or process. If your logs are all in one place, then using a dos command such as 'findstr /s "<<your search text>>" tm1s*.log > outputfile.txt' will build a file showing all appropriate entries. Your skill will be in getting the right search text to make this work and be sufficiently succinct to search through only just enough log files.

Hope this helps.

[mattgoff]

cgaunt, it's hard to tell from the post (a code tag would have helped) but I believe LP_user's problem was that he/she made a n-level element into a c-level, then back to an n-level. Since c-levels don't store data, the data which was previously stored was lost (and not restored when it was returned to n-level). There's no way around this other than for he/she to be more careful when making hierarchy changes-- the GUI makes it seem trivial, but a mis-step can be terribly damaging. I believe it's best practice do do a SaveDataAll and backup the db folder first, especially when you're a new admin.

Matt

[lotsaram]

Is there any reason you can't post a real screenshot with real names, illustrating the real dimension structure? (even if you have to obfuscate the values). Why to do this is pretty clearly spelled out in the request for assistance guidelines, because all to often in an attempt to simplify with generalised names or pseudo code the actual problem is lost or disfigured and this appears to be what has happened here.

I agree with cgaunt that what you are describing doesn't seem right. Either r1 is not a leaf and has children itself and the structure there has also changed or there is something else you did that you are not telling us, such as Matt's guess that you may have converted r1 from N to C and back to N again. But either way unless you post full information then we are none the wiser and can't really help.

TM1 cube data is stored at the leaf level so rearranging hierarchies only changes aggregations, the underlying values are unaffected by such structural changes. Data cannot and is not deleted by hierarchies being changed (that is as long as no N elements have their type changed to C). Which brings back to the other point that there must be something you aren't telling us.

[David Usherwood]

Think respondents are being a tad unkind. The OP said:

It means to move b1 to r1 so that b1 is the child element of r1.

So yes, he's promoting r1 from N to C, and is less than overjoyed when the contents of r1 go AWOL. So yes, take snapshots seems the right response. Don't tell me you've never done that

[lotsaram]

Think respondents are being a tad unkind. The OP said:

It means to move b1 to r1 so that b1 is the child element of r1.

Yer right, I missed that. So on reread it's obvious that converting N to C is the reason for data loss. (which is something every tm1 administrator should be aware of!) When I skimmed the question I concentrated on the "diagram" and it looked like r1 and b1 were siblings as it seemed round brackets were being used to indicate a rollup and missed the statement.

[LP_user]

Thanks for all replies. Through the discussion, the problem is clear. Yes, my question is to change N element to C element, the data is lost.
In conclusion, the solution is do the snapshot of the server folder to backup the data, before changing the dimension structure. How does a good TM1 system do the backup? To backup every day, every week, or just on demand?

[mattgoff]

How does a good TM1 system do the backup? To backup every day, every week, or just on demand?

IT backs up my server to tape nightly, but that is more for disaster recovery and SOX compliance. For my own protection (and for faster recovery when needed) I have a script which makes a copy of the db M/W/F and ftps it to my dev server. Copies are timestamped, and it's come in handy more than once to be able to pull up a model from a few months prior to diff against production. That covers "oops" stuff-- if I'm doing any significant changes where I think there's a risk I'll do something Very Bad, I manually do a SaveDataAll and copy the db before starting the work.

Matt

[cgaunt]

Hello All,

Agreed, I went off on the wrong tangent by assuming this was moving an element, not converting N level to C level. Apologies LP_user, my response would have been different otherwise. I think there are other threads that might be of interest about saving a backup of the data directory in the event that you need a quick restore. Between that and creative use of the transaction logs, you should be able to recover any data that you have lost.

[tomok]

Here's a better idea: Don't make changes like this in a production environment without fully vetting them in a DEV or QA environment first. Then you don't have to worry about losing data.

[mattgoff]

Here's a better idea: Don't make changes like this in a production environment without fully vetting them in a DEV or QA environment first. Then you don't have to worry about losing data.

Let he who has never had an OH SH!T moment cast the first stone....

[stex2727]

Let he who has never had an OH SH!T moment cast the first stone....

ah yes my last ever experience with EP.

1) added item x to the application
2) didnt solve problem so I deleted the item x
3) Wonderful wizard asks if I would like to also delete it from the content store.
4) Why yes wonderful wizard item x is off no use to me so yes please delete it from the content store.
5) Application Crashes and Services will not restart
6) Not just EP but also the entire BI reporting as well
7) 2 x 12 hour conference calls to Cognos failed to solve the corrupt content store
8) Backup of Content Store failed to resolve the problem
9) Complete reinstall for a day and then backup from tape restored the environment back to situation 5 days prior

Why o why did that wonderful wizard not say
"Would you also like to delete item x from the content store?"
"Are you sure as this will corrupt the content store?"
"Are you sure because this will destroy every contributor Application built?"
"Did I tell you the Content store also contains important information on every BI report so you'll probably corrupt that environment as well?"
"Would you like me to rename that button to Self Destruct?"

Stex
Beware the wizardation of TM1

[Martin Ryan]

On the backing up issue, you might find this post useful.