TM1 Forum

Hi,

I am working on a model where the requirement is to update a cube from base data (CSV) "instantly" when a new CSV becomes available. I am looking at whether to have a Chore running a simple TI every 30 seconds that checks for the existence of the CSV file, if it finds one then it runs an EXECUTEPROCESS to load the data.

I realise this is a massive generalisation and I will need to test this approach but I wanted to find out if anyone has had any experience of running chores to this level of repetition and have seen any performance overhead or issues in doing this? Alternatively is there a better way of doing it? - eg I've looked at running a Windows Scheduled Task and using this to kick off a Tm1RunTI but I have concerns over Task Scheduler's reliability.


Thanks

A chore with a TI behind it that looks for the existence of a file is a tried and true method. I don't know about the every 30 seconds though. It depends on exactly how reliable your network is. Any hiccup and you can get in trouble with only 30 seconds between attempts. I've been burned before with a small window like that. I personally would never do anything like that with an interval less than two or three minutes.

Thanks

I don't know if you have experience with .NET but there is a file monitor class that will monitor a directory for operations (add/modify/delete) and can execute code based on those triggers. You can then use the REST API or command line operations to kick off that process. I prefer this method just because it gets around the file not being there or possibly being locked when the TI kicks off. Also if you have locking in your system a 30 second window is short enough that you can have stacking processes and potentially make your locking worse.

If you go the chore route I would look in the forum. There are plenty of examples of building a resilient TI process that checks of the files existence and availability so that you are not generating unneeded errors.

You also need to be aware of the logs when you run chores that frequently. We use to have two chores running every minute, with each chore running a TI Process. This meant that every time we looked in the transaction log all we saw a lot of the time was the repeating logs of the minute chores. This made it more difficult when looking for transaction errors.

What is the source that is creating the csv? That same source, if at all possible, could fire the TM1RunTI.exe tool that could fire the TI that picks up the flat file.

Another way is to use Java to kick off TI process if you use latest TM1.

Another way is to use Java to kick off TI process if you use latest TM1.

How exactly would you use Java to kick off a TI Process? By latest TM1 do you mean 10.2 fix pack 3?

Thanks.

Trevor.

TrevorGoss wrote:

Another way is to use Java to kick off TI process if you use latest TM1.

How exactly would you use Java to kick off a TI Process? By latest TM1 do you mean 10.2 fix pack 3?

There seems to have been a bit of a misunderstanding about that; I've heard this a couple of times now.

TM1 has had a client side Java API for many, many years, and many, many versions. That API could be used to do many things that any of the other APIs can do, and that includes triggering off chores and processes via the TM1Chore class which I didn't discuss in my article, but can if you really want me to.

What, no-one? {Scans the room looking at all of the upturned chairs, the hastily slammed door and hearing dozens of footsteps running for the exit.} OK, take my word for it then.

Java applications, like most languages, can also monitor directories, though in Java's case this became easier when the nio API was introduced.

Consequently it would be possible for a Java application to be running in the background (on either the server or a client), scanning a folder for new data files and triggering the upload when the file appears. There's no need for the latest TM1 version to do this.

The change that has been made in the latest version has been to allow TI extensions, which is where Java code is called by TI, not where Java calls TI.

The other thing that has changed in the latest versions has of course been the RESTful API... but that is not Java-specific and is in fact language agnostic. Yes, a Java application could use it to trigger a chore just as it could trigger the chore through the classic Java API as I described above... but a program that uses the RESTful API need not be only Java.

TrevorGoss wrote:

Another way is to use Java to kick off TI process if you use latest TM1.

How exactly would you use Java to kick off a TI Process? By latest TM1 do you mean 10.2 fix pack 3?

Thanks.

Trevor.

Yes, sorry for the confusion. As Alan said, you can use Java to run TI via REST API, or use TI to run Java via TI Java extension.

or use TI to run Java via TI Java extension

TI Java extension? I am yet to hear of this? Can you provide more information on this please? such as links? Or again, is this just the API? lol

To run a TI outside of TM1 I just use the TM1RunTI.exe tool, but yes you can also use the APIs.

Hi,

I am working on a model where the requirement is to update a cube from base data (CSV) "instantly" when a new CSV becomes available. I am looking at whether to have a Chore running a simple TI every 30 seconds that checks for the existence of the CSV file, if it finds one then it runs an EXECUTEPROCESS to load the data.

I realise this is a massive generalisation and I will need to test this approach but I wanted to find out if anyone has had any experience of running chores to this level of repetition and have seen any performance overhead or issues in doing this? Alternatively is there a better way of doing it? - eg I've looked at running a Windows Scheduled Task and using this to kick off a Tm1RunTI but I have concerns over Task Scheduler's reliability.


Thanks

conflax, did you ever solve this?

Thanks

TrevorGoss wrote:

or use TI to run Java via TI Java extension

TI Java extension? I am yet to hear of this? Can you provide more information on this please? such as links? Or again, is this just the API? lol

No, it's not the API. It's on the server side. The APIs are on the client side.

It's been discussed here before but you may be unaware of it because the process for implementing them has been so extensively "documented" in the way that IBM "documents" so many things. But start here and here.

I'll be writing up an instruction guide similar to, but I expect rather shorter than, my Java API guide once my fingers recover from the latter, and any readers' eyeballs recover.

TrevorGoss wrote:To run a TI outside of TM1 I just use the TM1RunTI.exe tool, but yes you can also use the APIs.

Quite so.

No, it's not the API. It's on the server side. The APIs are on the client side.

It's been discussed here before but you may be unaware of it because the process for implementing them has been so extensively "documented" in the way that IBM "documents" so many things. But start here and here.

Excellent, thank you.