TM1 Forum

Dear all,

we have a number of small processes that do administrative things (mainly tailormade logging in SQL Server) which are called in all our TI processes. The drawback of this is, we have a lot of log entries that distract us from the actual things we would like to see in tm1server.log, since each and every of these adds a line to the log when they start and finish.

Apart from what you can do in the tm1s-log.properties file, is there a way how we can avoid that?

Regards
Holger

Other than what you've already mentioned around the log properties file settings I don't know of any way to stop the detail getting written to the log (especially for only certain TIs and not others.)

However I have previously written a small bit of code in TI to periodically scan the log file and put the contents into a cube or SQL database so that I can search through it much more easily. You could do something along similar lines and then just skip the items you don't want included in your other copy. I used to take the whole lot across though and then just filtered out annoying/pointless things (like ssl errors etc) in the destination e.g. a SQL view or TM1 cube or websheet.

Another option is working on the log file itself. As it is just a text file you can manipulate it easily.
For instance, a VBScript that filters on the "records" and deletes records that match certain criteria.
It requires some scripting but Declan's solution also is not part of the software so needs to be scripted as well.

Another way to look at it is to simply not use tm1server.log for the purposes of application logging (as opposed to server activity logging). E.g. at the end of various TIs in your system you can do this: Then you get to avoid the swamp of a file that tm1server.log can turn into and also get to tweak your logging. Obviously, setting up a generic TI that builds the output string from parameters and logs it would be useful and further underlines the utility of doing this as you will be muddying up tm1server.log with calls to your alternate logging TI...

If you want to mimic the format of tm1server.log, then conversion pattern is AFAIK: Google log4j conversion pattern if you need to dig into that stuff.

Actually we have a log outside TM1 in SQL Server, but there are situations where you like to get back to the TM1 log. Thanks anyway for your ideas and comments - I guess we will find a way to reduce the number of process calls which will somewhat improve things.

rmackenzie wrote:Another way to look at it is to simply not use tm1server.log for the purposes of application logging (as opposed to server activity logging). E.g. at the end of various TIs in your system you can do this:

Code: Select all

# assumes you have a line counter and process timer variable
sOutput = 'Process ' | GetProcessName | ' completed in: ' | sDuration | ', rows processed: ' | NumberToString ( nRowCounter ) | ', pFoo value: ' | pFoo;
sApplicationLogFile = GetProcessErrorFileDirectory | 'my_gl_reporting_application.log';
sCommand = 'cmd /c "echo ' | sOutput | ' >> ' | sApplicationLogFile  | '"';
ExecuteCommand ( sCommand, 0 );

Then you get to avoid the swamp of a file that tm1server.log can turn into and also get to tweak your logging. Obviously, setting up a generic TI that builds the output string from parameters and logs it would be useful and further underlines the utility of doing this as you will be muddying up tm1server.log with calls to your alternate logging TI...

If you want to mimic the format of tm1server.log, then conversion pattern is AFAIK:

Code: Select all

%d [%t] %-5p (%x) %c - %m%n

Google log4j conversion pattern if you need to dig into that stuff.

This is a very smart workaround of the ASCIIOutput, but I have to switch the wait parm to 1 because sometimes TM1's code is executed faster than the echo command. Also, the piece of code should not be used in either metadata or data tab since it would significantly decrease the performance.

hyunjia wrote: Also, the piece of code should not be used in either metadata or data tab since it would significantly decrease the performance.

That's right, and a good point too - each call to ExecuteCommand spawns a new cmd.exe so coding like this isn't what they call 'thread-safe' and should definitely be avoided for logging during Data and Metadata. The time to use it is e.g. in those systems that have big overnight batch processes which have hundreds or thousands of TI executions but you may only be interested in the outcomes of 10-20 points in the entire flow of the chore/ set of processes.

holger_b wrote:Actually we have a log outside TM1 in SQL Server, but there are situations where you like to get back to the TM1 log.

Sure, but are you using your SQL Server logging for the application or system events? Seems to me that the two are significantly different especially where TM1 is a piece of BI middleware or is connected to more than 2 or 3 upstream sources for enterprise consolidation systems.

It's just about application events. Basically every process logs start time, parameter values, end time and a success flag. We did not bother to log system events in the SQL log so far, and there is no plan to do so. These days we decided to have all the necessary code in the prolog resp. epilog, instead of calling a separate process. This is a bit less modular but reduces the number of TM1 log entries dramatically, even though it does not make the processes easier to read for someone who is not familiar with the project.