Process Error Log Not Available

[tomok]

I have a routine I have used for a long time that accumulates a list of all the error log files generated during the running of a chore and the last process of the chore generates an email that sends a message with all the error logs as an attachment. I now have a need to do this for one process at a time and not involve a chore (since chores can't be called from an action button). However, I can't get the process to attach the error log file. It appears that the reason is that the error log file itself is locked by the process because a FileExists check on the error log file says it does not exist even though I know that it does. I have tried breaking out the email part into a separate process and calling it via ExecuteProcess but that doesn't work either, ostensibly because the original process is still active. Does anyone know how I can make this work without involving a chore?

[Steve Rowe]

I've had a very erratic file lock issue in one environment we look after where the a text file remains locked by TM1 even after the all TI processes have stopped running. The only way to clear this (that I know of) is to restart the instance.

We can tell it's happening as we can not delete the file from the disc using windows explorer.

The other issue may just be that you are in an environment that running faster /slower than normal, have you tried putting a sleep statement between interactions?

[gtonkin]

Until the process ends, the processerror log is still named .log$
As a test, change your FileExists to see if the .log$ version exists - if it does, it is a timing issue.

What I have done on my email process is to call it with the necessary parameters from a RunTI - I do however prefix the RunTI command with a:
ping -n5 localhost&C:\Program Files\...\TM1RunTI...

[tomok]

Until the process ends, the processerror log is still named .log$

Thanks for the idea. I modified my code to put a "?" at the end of the file name but that didn't work. So, I put a one minute sleep in the process for testing and did a continuous query of the log directory and it appears that the log file is not actually created on disk until after the process completes which would explain why the email process cannot find it. It looks like I am stuck.

My new idea is to write the name of the file to an attribute on the }Processes dimension called "Lastest Error Log". I can then create a separate process that queries this attribute and emails the file represented by "Latest Error Log" attribute to the email address of the person that executes the process. I can then put a separate action button on the sheet (right next to the one that runs the original process) that sends the error log. I will call the button "Get Error Log" or something like that. I will give that a go and report back.

[tomok]

I went ahead and added a new attribute to the }Processes dimension called "Last Error Log" and changed my process that updates process attributes (I have a generic one that I call at the end of every TI process) to update this value with the name of the error log, if any. I also created a new process called "Get Process Error Log" that takes the name of the process as a parameter and uses that to do a ATTRS against the process and get the name of the last error log. As long as the file exists, it calls our existing Powershell email script which sends an email attaching the latest error log of the original process to the person who executes the Get Process Error Log process.

I am now going to be putting new action buttons on all the forms we have where users execute TI processes that will call this new Get Process Error Log process and pass the name of the original process they ran. Then all I need is a little user education to tell them to click the new button in the event of an error message from the original process indicating it had errors.

[paulsimon]

Hi Tom

Your solution is pretty much what we do. However, one suggestion is that rather than have the attribute on the process dimension, have it on the client dimension or a copy of that.

The reason being that if two users run the same process and both get an error, then there will only be a record of the error file generated by the last user's run of the process, so only the user that clicks the Get Error Details button first will see the details of their error. The error user might see a confusing set of errors that don't relate to them.

By comparison if you store the name of the TM1ProcessError log file against the user, then hopefully if the user gets an error that they can't understand just from the error message alone, then they will click the Get Error Details button before they try running anything else. The typical scenario is where the user is loading up a CSV file, or submitting some numbers that have validations that can be failed such as eg staff numbers must be positive. This will lead to the Data Tab rejecting some records to the TM1ProcessError Log file. What we actually do, is rather than just have a button to get the error detail, we have a button that does this and also transfers them to a generic worksheet which displays the first 100 lines of error lines that were loaded up by the process. The button is the same on any sheet since it runs the same process, which uses TM1User to get the name of the user and retrieves the name of their last error file and loads the contents of this into a cube that is user x lines and it is this cube that gets displayed. You can cut the import at 100 lines or whatever is sensible to prevent a huge cube being populated.

Regards

Paul Simon

[lotsaram]

You could just launch the emailer process with ExecuteCommand and tm1runti from the Epilog and put in a small wait so that by the time the emailer checks for the error log file the calling process has finished its commit and the lock on the log file is released. Or if you are on PA the RunProcess should achieve the same but without need for tm1runti.

[tomok]

Or if you are on PA the RunProcess should achieve the same but without need for tm1runti.

I am on PA because we're in the IBM Cloud. I'll give the RunProcess thing a try. I was using ExecuteProcess and that didn't work. As I said earlier up the thread, the file is not actually being written to disk until the original process finishes. Since the ExecuteProcess that was generating the email was called on the Epilog, the original process was not finishing until the branch off email process finished. FWIW, I put in a five minute sleep immediately before the ExecuteProcess when testing and that still didn't work.

[tomok]

I'll give the RunProcess thing a try.

No go on the RunProcess thing either. The issue is that the error log file does not exist on disk until the original process is completely finished. Sticking a RunProcess on the Epilog means it is running before the original process finishes. I even put in a one minute sleep right before the RunProcess, to give TM1 time to generate the file, but the file still does not exist until after the process finishes. I confirmed this by opening Windows Explorer and continually refreshing the view of the log folder. The file does not show up until the process finishes. I am going to stick with the idea of the separate process that can be called by the user via an action button. Thanks for the ideas.

[gtonkin]

The main difference is that I using ExecuteCommand NOT ExecuteProcess i.e. forking out to the command prompt, waiting 5 seconds then executing the TI emailer from outside of TM1 and the process that should be complete.

Admittedly this is 10.2.2 and on premise so may not work for your purposes but will check anyway.

[gtonkin]

This is what I found and ran some basics tests on:
PROLOG

Code: Select all

NumericGlobalVariable('PrologMinorErrorCount');
NumericGlobalVariable('MetaDataMinorErrorCount');
NumericGlobalVariable('DataMinorErrorCount');

EPILOG

Code: Select all

'Will not help if the error is triggered in Epilog unfortunately-GetProcessErrorFilename() returns blank
nErrorCount=PrologMinorErrorCount + MetaDataMinorErrorCount + DataMinorErrorCount;

'Could also possibly just check if GetProcessErrorFilename() is not blank but we log the error count to a process tracking cube
IF(nErrorCount>0);
sErrorLogName=GetProcessErrorFileDirectory()|GetProcessErrorFilename();

#=====SEND EMAIL=====
#--Doing a ping in command prompt to delay 5 seconds then executing FILE EXISTS check
sCommand='cmd /C ping -n 5 localhost & IF EXIST "%sErrorLogName%" ( Echo File Found!!>emailer.txt )';
sCommand=EXPAND(sCommand);
ExecuteCommand(sCommand,0);
#======================================

The "Echo File Found!!>emailer.txt " was purely to illustrate the point and would be replaced with the TM1RunTI command and parameters to call your TI process to do the actual email with the error log attached.

Not sure if this helps but this is what I have.

Good luck!!

[lotsaram]

No go on the RunProcess thing either. The issue is that the error log file does not exist on disk until the original process is completely finished. Sticking a RunProcess on the Epilog means it is running before the original process finishes. I even put in a one minute sleep right before the RunProcess, to give TM1 time to generate the file, but the file still does not exist until after the process finishes.

Well Duh. You need to put the wait IN THE EMAILER PROCESS that is called with RunProcess. No point putting a wait before calling the emailer as like you observed the error log isn’t released until the process finishes. So you could wait a day and still won’t work.

As RunProcess launches in an independent thread you put a wait in the prolog there. By the time the wait finished the calling process has completely closed and the file exists. If you are launching the emailer process with RunProcess from the Epilog of the process generating the error then a short wait should be sufficient unless the commit phase of the other process is unusually long.

[tomok]

Well Duh. You need to put the wait IN THE EMAILER PROCESS that is called with RunProcess. No point putting a wait before calling the emailer as like you observed the error log isn’t released until the process finishes. So you could wait a day and still won’t work.

Thanks. That did the trick. Not the first Well Duh moment I've had and it certainly won't be the last.