Send Email when TM1 service is down

[armas]

Hi

I have TM1 running as Windows Service.
Is there a way to send an email whenever the server crashes or comes down for any reason??

[George Regateiro]

We created a .net exe that runs on our server. It is scheduled through the windows scheduler to run every hour. This program will then send out emails if the services are not running.

[armas]

Well the thing is that when a TM1 user loads a large a view or any other funny thing crashes the TM1 process, the service keeps showing as Started, so I suppose the scheduler will not detect the issue.

[George Regateiro]

If they are just loading a large view then the service is technically still running properly it would just appear locked up to some users trying to access the same information. As long as the service is in a started or running state I dont know that there is really anything you could do. I will have to defer to the others on the forum for this since I have never attempted the type of logging/monitoring that you are attempting.

The only thing that I can think of is tracking memory usage over a period of time and attempting to do a comparison based on that information, but that makes alot of assumptions about how your model is setup and used.You could get alot of false positives from normal usage and not just when there is a problem.

[Lukas Meyer]

Hi,

You have to distinguish between
a) Server crashed - the process/service terminated
b) Server does not respond - someone was writing a TI-process using WHILE and the condition can not be false, or someone opened a really, really big view and the server is locked for a couple of minutes

checking whether a) occurred is easy - just a little program reading the services on startup, fetching the PID (process id) of the desired server and checking every now and then if the PID is still in the processes (and if it is occupied by tm1sd.exe ^^). (You could also check the services every now and then and see if the PID is still the same...)
b) is tricky - the server will probably respond a bit later - or not. I guess you could use the API to log in a client (with no rights to view anything at all ) - if it fails you might have a problem - try again in p.ex 5 minutes. Failed again? Mommy should look after her little server immediately - little server might forgot how to breath

None of those approaches have been tested productive - usually users will complain very soon if the server won't respond...
(Although I wrote a little program listing all tm1 servers on the machine and telling me their PID - and then I started to make the poor little program really complex (reading the servers' settings) - and I could go on adding features for some days (p.ex all taken ports by tm1-servers, then all bound ports of any application and a suggestion for the next port I should use when setting up yet another tm1-server without stopping a previous one to make my co-workers annoyed )

Hope this helped
Lukas

[ScottW]

To check for case b as defined by Lukas you could also write a simple app to check tm1top log output. Where you have a thread with a long processing time and other threads start backing up with a wait status then something is up and you get the app to send the tm1 admins an email or SMS.

I can remember this being discussed on the old forum, if you do a search you might find it.

[bihints.com]

Your IT department should be running some sort of network monitoring software like zabbix, nagios, zenoss...
All of these are free/multi-OS and generate reports / send out alerts when something goes down or reach predefined thresholds (low disk space, RAM, CPU hogging, you name it...)

[Steve Vincent]

or run Ben's version of TM1top. That comes to the foreground when a server is stopped. If a server is busy then the colour of the icon in the toolbar is a good indicator, but i usually get a call from a user before i notice it anyway. Unless its me causing the holdup

[Eric]

run Ben's version of TM1top

???? Never heard of this

[Martin Erlmoser]

Hi

I have TM1 running as Windows Service.
Is there a way to send an email whenever the server crashes or comes down for any reason??

you can define in the service properties (services.msc) action On First,second or subsequent failures.
which also can be a application,script whatever

therefore you don't need another program/service which checks the pid of the process because ms already implemented it..

of course you can use service monitor programs like nagios for such things but the windows implementation should be enough for you..

btw my settings are:
restart, restart restart

/aaah damn save button

edit:

or run Ben's version of TM1top. That comes to the foreground when a server is stopped. If a server is busy then the colour of the icon in the toolbar is a good indicator, but i usually get a call from a user before i notice it anyway. Unless its me causing the holdup

nice, my version of ben's tm1top crashes when a tm1server crashes.


edit2: i really don't recommend using a solution like scheduled task which runs every xx minutes ..

[Steve Vincent]

run Ben's version of TM1top

???? Never heard of this

Couldn't find it here so i've just uploaded it to the useful code thread, you can find it here.

I have had it crash a few times when a server has crashed too, but even then you get a windows error box pop up which is better than nothing

[Alan Kirk]

Hi

I have TM1 running as Windows Service.
Is there a way to send an email whenever the server crashes or comes down for any reason??

you can define in the service properties (services.msc) action On First,second or subsequent failures.
which also can be a application,script whatever

therefore you don't need another program/service which checks the pid of the process because ms already implemented it..

of course you can use service monitor programs like nagios for such things but the windows implementation should be enough for you..

btw my settings are:
restart, restart restart

Provided, of course, that the crash of the service doesn't cause a Windows dialog box to pop up as described here:

http://forums.olapforums.com/viewtopic.php?f=18&t=456

meaning that your Restart specification doesn't work until you manually go in and clear said dialog. This of course rather defeats the purpose of those otherwise-useful restart settings

[Martin Erlmoser]

i thought that this happens only when you are connected on the console of the server - then maybe a message pops up und you have to click ok.

but normally you are not connected on the console and even nagios or a program which checks the PID of the tm1server can do anything here..

when this message pops up the tm1sd.exe is running but not working .. when you click ok the tm1sd.exe will be killed and now the server will be restarted or a program can see that the tm1server is not avaiable anymore and sends mail or whatever.

you can create a program which trys to connect to the tm1server but thats not a really good workaround (ok, it's really bad.)

i'm very happy with the MS built-in solution and for my requirements its enough.
i get these errormessages, if nobody is connected on the console, written into the eventlog and thats nice.

[bonafiden]

Does TM1 server crash often?

[Steve Rowe]

I'd say the answer to this no.

In my current operational envronment which is used by my users, I don't think the server has ever crashed, possibly once or twice at most in a couple of years. I don't keep stats.

You will find people who say that TM1 crashes alot though. This is usally for two reasons.
1. They have hit or are clost to the memory limit of what their TM1 environment can cope with.
2. They are on an unstable release of the software.

If you are just starting out in TM1 it's probably a good idea not to go to the latest release and start on the latest 9.0, this advice might change when the next service pack comes out.

Cheers,

[Martin Erlmoser]

Does TM1 server crash often?

depends on the version and the skills of the developers and how you define "often"

in comparison to the mssql server - i would say definitely yes, its much more difficult to geht a mssql server crashed than a tm1server

[Steve Vincent]

Code: Select all

i = 1;
While ( i < 2 );
   ASCIIOUTPUT ( 'ohhowwelaughed.cma' , v1 , v2 , v3 , v4 );
End;

Its that easy in fact

(sorry, its Monday and for some reason i found that funny...)