Page 1 of 1

Instance crash after turning on Performance Monitor

Posted: Wed Aug 23, 2017 12:54 pm
by kenship
Hi all,

Our production instance has been crashing lately without warning or any message that we can find.

We originally thought it's one of the small chore but after suspending it we believe Performance Monitor is causing it.

Nothing has been done to the server hardware, configuration etc. I turned on Performance Monitor from time to time and never had a crash.

Memory usage was low, about 50% of what's available. Our Windows server administrator reported that hardware is running at tip top.

My question is: where should I start? I'll start looking at TM1 Server Crash Playbook but will appreciate other ideas as well.

If I'm missing in providing any critical information, please point out.

Ken

Re: Instance crash after turning on Performance Monitor

Posted: Wed Aug 23, 2017 1:43 pm
by Edward Stuart
I'd check the Event Log first to see if anything of interest occurred around the time of the crash and if anything is in the application error

Then I'd check the tm1server.log file for anything of interest around the crash time

Then tm1s.log at the time of the crash

If these turn up nothing then I'd look at the TM1 Server Crash random fault section:
How many users were on the system?
Were any TI processes running, or scheduled to run around the time?
How large is the model in memory? On disk?
Were any new business rules applied recently?
Versions of ALL software, including operating system and hardware.
Most importantly, are there a set of steps that cause the fault to repeat?

Re: Instance crash after turning on Performance Monitor

Posted: Wed Aug 23, 2017 1:57 pm
by kenship
Thanks for the reply.

For our last crash,
User: About 2 or 3
TI running: none;
Memory: 5.9G used, 900MB in garbage; 16GB total available

Checked some log messages:
Server Message Log has no warning.
TM1s recorded nothing out of ordinary.

Chores been running for years without recent modification.

Like I reported, the trigger seems to be Performance Monitor.

We are testing the same set of data in Staging to see how it reacts now.

Re: Instance crash after turning on Performance Monitor

Posted: Wed Aug 23, 2017 2:47 pm
by Edward Stuart
Most importantly, are there a set of steps that cause the fault to repeat?
Does Performance Monitor crash the server every time it is run? If so, then it could well be a hardware issue than a Cognos TM1 issue

Re: Instance crash after turning on Performance Monitor

Posted: Wed Aug 23, 2017 3:27 pm
by kenship
Quite positive it's Performance Monitor that causes the problem.

We copied the all data and dropped it in another instance on our staging server.

Within next to no one (just me) logged in, all chores ran fine for an hour, then minutes after I started Performance Monitor it crashed.

Since the same happened in another environment, I'm ruling out hardware as well.

Kenneth