API Functions for TM1Top

[TM1Sampler]

I’ve browsed these forums for the past couple of days looking for answers, and I have not been able to find any yet. But I did come to the conclusion that I fall in the “I’m new to TM1” category, but not so much in the “my issue is super urgent” category. More like, I’m new to TM1 and my question has medium-level urgency at this point .

I’m responsible for a couple systems that interact with TM1 via the API, and my question is:
Where does TM1Top get the information that it displays? I’m mostly interested in two columns, “ID” (process ID I believe), and “State” (Idle\Run\Commit etc.).

I know there is an extended version of TM1Top (Ben Hill’s), so I assumed that Ben used the API to query the data for his version, but where does the data come from and what API methods does one use to get it? Is it an MDX query to an obscure propietary data source? Or just data in some cube? I’ve browsed through the various API documents, but I wonder if I’m looking in the wrong places.

The goal is to build some logic in an external application, which checks in real time the level of activity in the TM1 server, in order to decide how to proceed with other processes.

Thanks in advance for your help.

Sampler

[Mike Cowie]

Hi Sampler:

First, you say you're new to TM1 so I'll give the usual words of caution: If you don't really need to use the TM1 API (I'm talking mainly the C/VB TM1 API here) then you'll probably be much happier overall. The TM1 API certainly has a place in the TM1 world, but it's not an essential tool for all implementations and just because you can create a custom application/front-end doesn't mean you should. Building and supporting a custom application isn't trivial and the TM1 C/VB API, in particular, isn't natively as user-friendly as other API's you might have used elsewhere. It is also unlikely that your customer/company will find a large body of other people who can jump in and support a TM1 API app if you get hit by a bus.

OK, that's out of the way.

As far as I'm aware, TM1TOP and, similarly, Ben's TM1TOP app and components of Cubewise's Vizier get their TM1TOP information from the TM1 Server via calls to TM1 (C/VB) API functions that are not officially documented or published by IBM. This doesn't mean you can't ask IBM for the proper declarations via normal support/service request channels if you believe you have a need to use the same functionality in your application. Some of the functions you might be looking for declarations/instructions for (which I don't have reliable details on, unfortunately) might include these and more: TM1TopConnect, TM1TopGetCurrentState, TM1TopVerifyAdmin, TM1TopKillTiProcess

I do not believe there are any special TM1 or other (MDX, non-TM1 data source, etc) components involved here with TM1TOP, outside of something within the TM1 Server application that is servicing these TM1TOP requests and providing information about what is going on across user threads on the TM1 Server.

Your goals sound fairly ambitious given the tools available, particularly since it sounds like you're trying to work out the best time to kick off processes based on information via TM1TOP. The first problem is that TM1TOP is only sort-of realtime in that you get the latest information, I believe, when you call these TM1TOP API function(s), but, you may not really want to call TM1TOP over and over again in a loop to get that close to realtime information since I believe it refreshes all TM1TOP data for all threads with each call and that may have some impact on the TM1 Server - you'd have to test and see. Ideally you'd ask TM1 to notify you of certain TM1TOP-related information for certain objects on the TM1 Server that you'd like to watch, but I doubt this is possible currently. The other, probably bigger, problem is that you can't easily control what other users try to do or will try do next - it might look like the TM1 Server isn't busy one minute, but then 3 minutes later someone kicks off a 20 minute TI process and you're not much better off than you were before.

Don't get me wrong - what you're trying to do sounds very interesting and creative. Maybe these aren't things you need to worry about and you just want to monitor and find a point where 5 minutes have gone by without user activity before kicking off a data import process. I'm just pointing out that there might be other, simpler, options for solving this kind of problem and by trying to come up with a solution like the one you briefly describe you could inadvertently introduce additional problems/complications that you really don't need. Feel free to share more details about what you're ultimately trying to do if possible/appropriate.

Hope that helps and keep us posted on how things go.

Regards,
Mike

[Kyro]

Hi TM1Sampler, Ben Hill here.

We are in the process of beta testing a new revision of the Custom TM1 Top. This one has had a lot of new features added and includes a server side monitor service which provides a 24/7 Picture of server activity including outages. This holds log data in a SQLite database and can export XML logs. One thing I will say though is that this version unlike the last, will not be free.





We will open beta testing to the public within the month if you are interested.

[Paul Segal]

Looks good Ben. Before the stampede. I'd definately like to beta test!

Regards

[Andy Key]

...within the month...

But that's what you told me last month...

[Christopher Kernahan]

Hi TM1Sampler,

The goal is to build some logic in an external application, which checks in real time the level of activity in the TM1 server, in order to decide how to proceed with other processes.

As Mike says, TM1 Top won't be 'real' real-time so I would be careful using this to rely on for indications of when to execute processes. A better approach may be to log and graph server activity so you can see when the busy periods are (daily and through a month-end cycle) and the set your chores around these periods.

If you are analysing activity because you are worried about things like process/user contention, then depending on your version I think you'd be better off using the 'Semaphore' design practice.

Kyro,

Would also be keen to beta.

[TM1Sampler]

Than you very much for everyone’s responses.

A more advanced TM1 developer seems to have come up with a solution that doesn’t involve using the API. I’ll wait and see how that goes. I’d suggested the API approach, because that’s the only side of TM1 that I’d know until recently. One of my applications uses it extensively. I’ll remain unofficially curious about the TM1Top API calls though.

Mike,
Again, thanks for all the feedback. I would agree with you %100 on your recommendation not to use the TM1 API, unless it’s your last option. The application that uses the API in our org was passed on to me in a state that I would call “rough prototype”. After a major overhaul, a lot of sweat, tears and sleepless nights, it finally went to production in a state of “fair stability” My organization has a very unique business requirement, which forced us to go the API route for this particular item.

Ben,
It looks nice. Yes, I’d be interested in trying out the Beta. I’ll keep my eyes open for that. – thanks

Christopher,
Yes, the Semaphore approach could be what the other developer has in mind. BTW, how much of a delay does TM1Top have anyway? I always assumed that it was only a second or two. Am I wrong?

Sampler

[Mike Cowie]

Hi Sampler:

As far as I know, it's not that there's a big delay in TM1TOP's information, but there are a couple potential things you need to keep in mind with its data:

1) It gives you all active thread data when you ask for it, so it's not as though it's feeding you realtime updates - I think this is more of what we meant by not being realtime (at least it's what I meant). If you're using the TM1TOP app, the most you can have it refresh is every second, and although you could write an app that refreshes more often than that, it could have unintended consequences.

2) TM1TOP is giving you a snapshot of activity. For things like TM1ViewArrayConstruct and TM1ProcessExecute "commands", it's pretty easy to see what's happening with TM1TOP. But, if you've got a user with 20,000 DBR formulas recalculating in a workbook it may not give you a clear picture on what's happening - you may catch the odd TM1CubeCellValueGet popping up for that user, but seeing that may be hit or miss for these kinds of brief "commands" that are displayed in TM1TOP. Only the API functions that take a long time to run are the ones that are easiest to spot/track via TM1TOP because it's so focused on TM1 API functions.

Good luck!

Regards,
Mike