View Transaction Log... and View Message Log...

[kiamkhai]

Dear TM1 Developers

Would like to seek your assistance regarding the above. The above two views can be open and viewed by right-clicking at the TM1 Instance. My question is whether the two views save separately in the log folder or save together in the same file, namely tm1s.log?

2. If I want to keep the transaction records perform by the users who make changes to the data forever (to compile with the auditor's requirement) how can I achieve this? At the moment we only retain the tm1s.log files for one month due to the space constraint etc.

Cheers

Regards
Tee Kiam Khai

[Alan Kirk]

Would like to seek your assistance regarding the above. The above two views can be open and viewed by right-clicking at the TM1 Instance. My question is whether the two views save separately in the log folder or save together in the same file, namely tm1s.log?

tm1s.log records details of all of the entries that users make to a cube, provided that logging is turned on for that cube. When you do a data save on the server, that file is renamed using a date/time stamp (eg tm1s20080820215055.log), and a new, blank tm1s.log file is created. This is the Transaction Log.

The Message Log is stored in tm1smsg.log, which just keeps getting bigger and bigger and bigger until one day you kill it just to clear up some disk space, then it starts again. This one records events like the loading of cubes and dimensions when you start up the server.

There's a third log called tm1serr.log, in which I've never found anything remotely useful. When a server crashes because it's out of memory, for example, that event will be recorded in the tm1smsg.log file, not the tm1serr.log.

9.4 introduces a whole new game when it comes to Audit Logs, but I haven't looked into it sufficiently yet to be able to comment on those.

2. If I want to keep the transaction records perform by the users who make changes to the data forever (to compile with the auditor's requirement) how can I achieve this? At the moment we only retain the tm1s.log files for one month due to the space constraint etc.

I store them in an Access database application, but that's mainly so that I can query them more effectively. For storing more of the raw data, WinZip (http://www.winzip.com/index.htm) is your friend.

You may also want to consider buying an external hard drive to store the older ones. I wouldn't trust CD/DVD disks as a "forever" storage medium.

[kiamkhai]

Dear Alan

Yes I am able to find tm1s.log (current) and the historical transaction log file (renamed using a date/time stamp )in the same folder.

2. I am unable to locate the Message Log is stored in tm1smsg.log in the logs folder. How can I get this file? Do I need to configure the "tm1s.cfg" file to get this file. For your information, I notice there is another log file, i.e. tm1server.log which only keeps info of the server. Please advise.

Thanks

Cheers

Tee Kiam Khai

[Michel Zijlema]

Hi,

I guess you're using a TM1 9+ version. In this version the tm1msg.log file was replaced by the tm1server.log file.

Michel

[Alan Kirk]

I am unable to locate the Message Log is stored in tm1smsg.log in the logs folder. How can I get this file? Do I need to configure the "tm1s.cfg" file to get this file. For your information, I notice there is another log file, i.e. tm1server.log which only keeps info of the server. Please advise.

OK, it looks like this is a versioning issue. We're still on 8.2.12 in production, which uses tm1smsg.log. By 9.1 it seems to have been replaced by tm1server.log.

Same thing, different name as is often the case with any new TM1 version.

(I note that in 9.4, ASCII as a TI data source has been replaced by "Text", for example. One giant leap for mankind.)

[Alan Kirk]

OK, it looks like this is a versioning issue. We're still on 8.2.12 in production, which uses tm1smsg.log. By 9.1 it seems to have been replaced by tm1server.log.

Oh, like Michel beat me to the punch in saying...

[Steve Rowe]

On the tm1serr.log this was clearly named by someone with a sense of humour as I can't think of any other reason for making it so confusing.

TM1 SERver Replication = tm1serr.log

Its the log file for replication logs, nothing to do with errors at all..

Cheers,

[Alan Kirk]

(I note that in 9.4, ASCII as a TI data source has been replaced by "Text", for example. One giant leap for mankind.)

Upon reflection, this one does actually make a measure of sense since 9.4 does support non-ASCII text.

Unlike, say, changing long-standing convections like upside down A icons just to make the interface look "fresher".

(It's worth noting that save icons still frequently take the form of 3.5" floppy disks, long after such critters have been consigned to landfill across the globe. There's a reason for that; it's called "familiarity"...)

[Alan Kirk]

On the tm1serr.log this was clearly named by someone with a sense of humour as I can't think of any other reason for making it so confusing.
TM1 SERver Replication = tm1serr.log
Its the log file for replication logs, nothing to do with errors at all..

That's interesting; I'll have to check the 8.2.12 documentation when I get in tomorrow, but if I recall correctly mine is choc-full of messages about ODBC connections (presumably the ones that we make with the GL system) rather than replication messages.

Either way, it's just byte-fill on the hard drive.

[Martin Ryan]

On the tm1serr.log this was clearly named by someone with a sense of humour as I can't think of any other reason for making it so confusing.

TM1 SERver Replication = tm1serr.log

Its the log file for replication logs, nothing to do with errors at all..

Cheers,

I inaugarate the Muppet of the Week award. This week it goes to that guy.

[Steve Rowe]

Err, looks like could be claiming the award , the file on our system just contains a records like this

20080820114511 [TIUSER] =>

not sure what they mean other than a TI was called. Anyway they don't contain error messages

[Eric]

kiamkhai wrote:2. If I want to keep the transaction records perform by the users who make changes to the data forever (to compile with the auditor's requirement) how can I achieve this? At the moment we only retain the tm1s.log files for one month due to the space constraint etc.

I store them in an Access database application, but that's mainly so that I can query them more effectively. For storing more of the raw data, WinZip (http://www.winzip.com/index.htm) is your friend.

Doh! I never thought of that. I hate the query feature in TM1 because it locks up the system and takes FOREVER! Probably implementing that idea next week.

[Alan Kirk]

kiamkhai wrote:2. If I want to keep the transaction records perform by the users who make changes to the data forever (to compile with the auditor's requirement) how can I achieve this? At the moment we only retain the tm1s.log files for one month due to the space constraint etc.

I store them in an Access database application, but that's mainly so that I can query them more effectively. For storing more of the raw data, WinZip (http://www.winzip.com/index.htm) is your friend.

Doh! I never thought of that. I hate the query feature in TM1 because it locks up the system and takes FOREVER! Probably implementing that idea next week.

Actually I find the query feature in TM1 to be quite fast. We generate maybe 30 or 40 megs worth of log files per week (outside of budget periods, when they can easily double), but it seldom even takes a minute to query them going back to the start of the previous month.

Its weakness is that if you want to find out who made entries for products 1234, 1235, 1236 and 1237 you have to run 4 separate queries. And the fact that you have to leave the log files uncompressed in their original location for as far back as you want to query them.

Don't get over-enthusiastic about Access, though; the file size will still be significant, with the back ends needing regular compacting (and with the 2 Gig limit on .mdb size you'll ultimately have to use multiple back ends)... and of course when you start to get tables of any size Access' query speed is terrible, oh it's just soooooo terrrriiibblleee...

Not that that matters to MS, of course, since the priority isn't the DB engine that hasn't been overhauled in years ("Jet"; yeah, right, in the sense of an ME-262 maybe...) but rather New! User! Interfaces! (And preventing Access from cannibalising SQL Server sales.)

We must have the priorities right, y'see.

Way back when I started developing this I thought about normalising all of the element fields to keep the file size down but the performance was just sooooo bad.... I ended up normalising the cube names and user names only.

I like what Access is capable of. I hate that Access can't do it QUICKLY.

[Alan Kirk]

On the tm1serr.log this was clearly named by someone with a sense of humour as I can't think of any other reason for making it so confusing.
TM1 SERver Replication = tm1serr.log
Its the log file for replication logs, nothing to do with errors at all..

Alas, 'tain't what the 8.2.12 Admin Guide says:
File Name: Tm1serr.log
Installed Location: Either the directory pointed to by the DataBaseDirectory parameter in the Tm1s.cfg file or the -d command-line parameter whenthe server is brought up (if this parameter is used).
Description: Error log file for a remote server, which gets created by the TM1 software.
Associated products: TM1 server

HOWEVER!

I agree with you that whatever it was SUPPOSED to be for, that ain't (generally) what it IS used for.

I looked at one of the ones that I copied after a server crash and found that almost all of the lines looked something like this:

Code: Select all

20080722021505 [R*PNL1_UpdatePNL] => [Microsoft][ODBC SQL Server Driver][SQL Server]Changed database context to 'FNPRDDB'.
20080722021505 [R*PNL1_UpdatePNL] => [Microsoft][ODBC SQL Server Driver][SQL Server]Changed database context to 'FNPRDDB'.

Though the veeeerrrry last line in that log was:

Code: Select all

20080722023443 [?] => &TM1 server is out of virtual memory...

So with the exception of that one line at the end, I suspect that whoever wrote the code that creates the file, and passed the details of the file on to the documenters, didn't talk very much with whoever wrote the code that writes output to the file.

Not that we've ever seen that happen before at Applix. Much.

[Eric]

Its weakness is that if you want to find out who made entries for products 1234, 1235, 1236 and 1237 you have to run 4 separate queries. And the fact that you have to leave the log files uncompressed in their original location for as far back as you want to query them.

You should be able to do this with one query. Not connected to my server to confirm, but I swear I have done it a bunch of times. Select you 4 product, click the keep button, and then choose the rollup button instead of OK or Cancel and should should be good to go. <Crossing fingers that I am right>

[Alan Kirk]

Its weakness is that if you want to find out who made entries for products 1234, 1235, 1236 and 1237 you have to run 4 separate queries. And the fact that you have to leave the log files uncompressed in their original location for as far back as you want to query them.

You should be able to do this with one query. Not connected to my server to confirm, but I swear I have done it a bunch of times. Select you 4 product, click the keep button, and then choose the rollup button instead of OK or Cancel and should should be good to go. <Crossing fingers that I am right>

I was going to say that that was a neat trick. Unfortunately... in 8.2.12 it just ignores the selection of the elements in the roll-up and returns everything anyway.

So, thinks I, maybe it's something that was fixed in a later version. (Albeit fixed in an undocumented fashion as far as I know.) Hey, let's try it on 9.4!!!

BUGBUGBUGBUGBUG!!! LOOKS LIKE I GOT THE FIRST BUG IN 9.4!!!

Oh. Wait. That's NOT a good thing, right?

I entered transactions into three product codes, then queried the first two of these using Eric's the rollup method. They appeared, and the third one didn't. So far so good.

Or so it seemed.

For I then queried everything for the cube, and... still got the same results.

Uh-oh.

So I queried the entries for the third product code. Worked fine.

Then I queried EVERYTHING, not just that cube. I saw something that I'd allegedly done in the }DimensionProperties cube. (Specifically, changing the Last Time Updated; you what? All I did was create some new views.) This was followed by the entries that I'd made in the main data cube... for the first two products.

You know how the log query dialog buffers 100 rows at a time, then when you page down or arrow down it will return the next 100 and so on?

It looks like in 9.4 they forgot to add the "grab the next buffer" code. No matter what you query, you get a maximum of 100 rows of data.

Can anyone else confirm?