Anyone know where Last Time Updated for Cubes held

[John Hammond]

I am assuming this exists to permit replication and log recovery but looking under }CubeProperties yields nothing whereas we immediately find }DimensionProperties.LAST_TIME_UPDATED to contain the value a dimension was last updated.

Please does anyone know where the Last Time Updated for Cubes information is held or if it exists in a queryable form.

Thanks

John

[David Usherwood]

You could read the datestamp of the .cub files to get the last time changes were committed to disc. Can't think of a way to get the last time it was updated in RAM.

[tomok]

You would have to search the log file (tm1s.log or one of the ones saved with the timestamps) in order to get the last transaction to a particular cube. There is no property in a TM1 system cube for this that I am aware of. Wish there was. The Dimension property is used in replication, deciding whether or not the dimension needs replicating. For cubes, the replication system always processes the log files generated since the last replication, no need for a property.

[John Hammond]

Thanks David/Tomok

Yes as you say there is the consideration as to date last updated = time written to disk or to memory.

You could write a wrapper around savedataall to ensure the last write to disk was known but wrapping up cellputn for last write to memory would be a collosal performance overhead in TM1.

Or savedataall last done = timestamp of latest log if I am not mistaken so we can do something for the TM1 server but not individual cubes.

[Alan Kirk]

Yes as you say there is the consideration as to date last updated = time written to disk or to memory.

You could write a wrapper around savedataall to ensure the last write to disk was known

Only if you assume that nobody will ever trigger a save from the GUI. Otherwise David's initial statement remains correct; the date of the .cub file on disk is the sole way of accurately determining this.

but wrapping up cellputn for last write to memory would be a collosal performance overhead in TM1.

Again, this would presuppose that not a single update is ever done by anything other than TI.

Or savedataall last done = timestamp of latest log if I am not mistaken so we can do something for the TM1 server but not individual cubes.

More or less correct, bearing in mind that the timestamp contained in the file name (as opposed to the date and time of the file itself) is in UTC rather than local time.

I suspect that cubes which have "dirty" data are flagged in memory for saving the next time a data save / server shutdown is done. It would certainly be safer than relying on a value stored in a cube, which could be accidentally overwritten by an Admin. However I can't find a property which would allow you to read that flag, or even whether the flag takes the format of a date / time stamp or just a straight out Boolean value.

In the API you can access an object property called TM1ObjectLastTimeUpdated, but when I looked at a list of cubes generated from my own code I noticed something curious about that property.

The property for most of the cubes on my server was the time of reboot in the early hours of this morning. The time for my process control cube was the time of the previous chore execution. Yet the property for the cube that was updated by that chore was... the server boot time.

Apparently the TM1ObjectLastTimeUpdated property is only updated when there are logged changes in the cube. When logging is turned off, the time updated property is not updated. (In the case of the chore mentioned above logging was off on the data cube (which showed no change to the property's value), but on on the process control cube (for which the update property was changed).) Accordingly it's safe to conclude that:
(a) That property is not the one that triggers a save of a particular cube; and
(b) It also can't be used for determining when the last change was made to the cube if the change was not logged.

I'll take a poke around some of the other properties when I have time but it's not looking promising.

Of course, it does raise the question of why you actually need this information, and consequently whether there might be a more practical alternative that could be employed.

[lotsaram]

John,

Along the lines of what David and Alan have said, in terms of getting the last update of a the best you can do if transaction logging is off is to get the timestamp from the OS of when the cube file was updated on disk.

This line of code in a simple batch file will do it for you:
dir *.cub /n/o:n/t:w > Cubes_with_last_save_timestamp.txt

[Steve Vincent]

Might have a bit more info for you, but I doubt it’s what you want to hear.

Replication only works on logged entries. When it is run it stores date/time stamps in the }ConnectionProperties cube including when the last sync was run and when the last record was copied. This is only held for the replication connection, not by object. When a replication is run TM1 uses this date to limit the log files checked to just those after this time stamp, so it doesn’t need to know when individual objects or data has been updated.

It is why replicating large amounts of data, especially over a WAN, is horribly slow. But it means the information you are looking for isn’t necessary for TM1 to function and so most likely not stored anywhere…

[John Hammond]

Thanks Alan, Lotsa, Steve

It is why replicating large amounts of data, especially over a WAN, is horribly slow. But it means the information you are looking for isn’t necessary for TM1 to function and so most likely not stored anywhere…

Was thinking about a log compressor that stripped out stuff zeroed out and filled with the same value.

Of course, it does raise the question of why you actually need this information, and consequently whether there might be a more practical alternative that could be employed

Originally was thinking about using logs to update cubes without replication by tricking TM1 into thinking it had crashed by setting the date last saved so it does an autorecover. However that descended into trying to understand how TM1 works as my original goal had been abandoned.

Thanks I am guessing that on start up looks for windows directory saved date on the cube and compares to the latest log. If the log is more recent the cube gets rolled forward.

[tomok]

Originally was thinking about using logs to update cubes without replication by tricking TM1 into thinking it had crashed by setting the date last saved so it does an autorecover. However that descended into trying to understand how TM1 works as my original goal had been abandoned.

I think you are giving IBM a little too much credit. Whereas the replication functionality looks at the date and time stamp stuff, I'm pretty sure the autorecover does not. How could it since it doesn't know when the last time a cube was updated? TM1 will perform an autoreceover any time it finds a file called "tm1s.log" in the data directory. So, all you have to do is rename any log file from "tm1sxxxxxxxxxxxx.log" to just plain "tm1s.log" and restart the service. TM1 will think that the server crashed and then it will reprocess everything it finds in the tm1s.log file. If you have more than one log file you need to process, just concatenate them all together and call it tm1s.log. I have used this trick in the past and it has worked every time.

[lotsaram]

Originally was thinking about using logs to update cubes without replication by tricking TM1 into thinking it had crashed by setting the date last saved so it does an autorecover. However that descended into trying to understand how TM1 works as my original goal had been abandoned.

Wouldn't a much simpler approach be to simply use the tm1sYYYYMMDDHHMMSS.log file as a data source for TI and just load data to the specific cube(s)? (I believe there is a bedrock process that does exactly this)

I am guessing that on start up looks for windows directory saved date on the cube and compares to the latest log. If the log is more recent the cube gets rolled forward.

Actually it is much simpler. On a Save Data the very last thing the server does in the Commit is to timestamp and release the current locked tm1s.log file so the presence in a data directory of a tm1s.log file of >0KB indicates that there are transactions that have not been saved to disk in .cub files. When a server loads into memory virtually the last thing it does after loading dimensions and cubes from disk and processing feeders is to check for an existing tm1s.log file, if it finds one then it gets loaded. It's that simple. (If running as a service, if running as an application then the user is prompted whether to recover transactions or not, the service just assumes the answer is yes.)

[John Hammond]

Tomok, Lotsa

Thanks for the advice. You are both saying the same thing that becomes self evident once you see how obvious it is! "Forward logic is 10-100 times less costly than reverse logic" is my motto, although what that is in Latin I dont know.

Will take a look at the Bedrock to read the logs and roll forward.

Steve

Will have a crack at a log compressor and post it later on work permitting to see if it can be used to speed up replication/save space on the server/make the logs more human intelligible...

[Steve Vincent]

i'd just be happy if the logs weren't screwed for some data by missing their date stamp, resulting in no replicated data unless you manually amend the file...