TM1 Forum

Hi guys,

i have TI where I Export Data.
When I Start the TI via a scheduled Job, the Job stops after a while.
I get the following message in the tm1server.log

11828 [] DEBUG 2016-11-01 11:44:50.665 TM1.Process ExecuteProcedures: Cancel request received.
11828 [] INFO 2016-11-01 11:44:50.712 TM1.Chore Chore "Jobname" - server command exception
11828 [] INFO 2016-11-01 11:44:50.712 TM1.Chore Chore "Jobname" finished executing
9972 [190] INFO 2016-11-01 11:49:03.807 TM1.Chore Deactivating chore: Jobname

My colleagues say, nobdy kill the Job over tm1top oder the operations console

Do you guys have any idea, why the Job stops and gets deactivated?

Thanks for your help

Memo

Ops console has a watchdog function where you can automatically set processes to cancel if criteria are met.
Have you checked to see if thats the case?

Hi,

no there ist no criteria defined.

I don't really have the time or inclination to do a full battery of tests on this but this line:

11828 [] INFO 2016-11-01 11:44:50.712 TM1.Chore Chore "Jobname" - server command exception

smells more like the source of the problem to me than the "Cancel Request" does. What I would do:
- Check that every process in the chore is still compiled correctly. Up until 9.5.2 a process with an error in compilation was really easy to spot. Then they "improved" the icons in 10.x. Still, I wouldn't even trust the icons but would instead open the processes and re-save them to ensure that they compile and save correctly.
- Check whether there are any process parameters for which no default values have been set in the chore.
- Check whether there are any default chore parameter values which are feeding a value which looks numeric into a process' string parameter. Processes store the data type with their parameters, chores don't and if there is a process which is expecting (say) the string '2017' and you have the chore default defined as 2017, that's going to go from the chore to the process as a number. I've seen that cause errors previously as well.
- Check that you haven't added an AsciiOutput statement in any of the processes whose first argument can't be resolved as a filepath.

My bet is that there is something in either the chore or the processes that is causing this to happen.

Hi Alan,

i will check your points, thanks.


As i sad, i have to export a lot of data. The TI starts exporting the data and generates an Asciiouput, but after 4 hours it stops.
Maybe the reason is the TI takes to long and it is too much data?!

Memo

Memo66 wrote:Hi Alan,

i will check your points, thanks.


As i sad, i have to export a lot of data. The TI starts exporting the data and generates an Asciiouput, but after 4 hours it stops.
Maybe the reason is the TI takes to long and it is too much data?!

Memo

4 hours of writing with ASCIIOutput???? Are you sure you're not running into some sort of file size limitation and the job is stopping because you've exceeded it?

Hi,

i have another TI where i also export a lot of data. The Export gets 4,5 GB.It takes over 2 hours but it works.
But this one stops at 3 MB.

And, the Process writes only 2 rows in a second.
When i change the output from .txt to .csv it writes 2500 rows in the first second then again only 2 or 3 rows in one second.

Memo66 wrote: i have another TI where i also export a lot of data. The Export gets 4,5 GB.It takes over 2 hours but it works.
But this one stops at 3 MB.

And, the Process writes only 2 rows in a second.
When i change the output from .txt to .csv it writes 2500 rows in the first second then again only 2 or 3 rows in one second.

Leaving aside the highly questionable practice of having 4.5 gigabyte files coming out of an OLAP system (please, please tell me that you aren't exporting zero values as well), doesn't the speed suggest to you that:
(a) There is something wrong with what you are selecting as a data source; and/or
(b) That you may be using rules that are really, really badly written and that the calculations are taking forever?

Also if 4.5 gigabyte export files are your thing you may want to check that your disks aren't choking. Drives are larger than they once were but 4.5 gigabytes still isn't nothing and it will be even worse if you are writing to a network location rather than locally.

I know it sounds like a nightmare, and believe me it is.
But we have to do it for now.

We dont have any rules in this cube and we dont export Zero values.

I have to check my source again then.


Thx:)

Hi

Have you checked that you have Skip Consols, Rule Calc Values, and Zeroes ticked on the source view. My guess would you that you have a lot of deep hierarchies and Skip Consols is not ticked.

Regards

Paul Simon

Hi,

yes, the Dimensions are very big.

I now tried to export a view with 7 rows with right click and export in the architect. It's the same problem.

I ticked the Skip for Zeros. Ich habe Consolidated Elements in my View. So i don't ticked that.

Perhaps have a play with the dimension order? If you have no rules and zeros etc checked then that would be my next port of call.

Is the large export that is working from a different cube to the one that isn't? Suggest you look hard at the differences between the two exports and sources.

Double / triple check all the things you have told us in this thread.... (This still feels like a rule problem as previously mentioned).

Hi,

i now tried to export data step by step beginning with a view with n-level elements. This works.
Then i exported the data with one consolidated dimension und the rest in n-leven.
Then with two consolidated dimensions und the rest with n-level and so on.

Every time took one mor consolidated dimension in my view it got slower.

The large export which works is from the same cube but with less consolidated dimensions. I guess this is why it works.



@Steve Rowe
There are no rules in the cube.

Memo66 wrote:Every time took one mor consolidated dimension in my view it got slower.

Yeah, no kidding. Calculating a consolidated element takes processing time, as well as memory. Of course it's going to be exponentially slower than exporting a bunch of leaf level elements. Given that you are exporting a bunch of consols you probably need to ramp up the VMM setting for the cube you are exporting. I would make it as big as possible.

Hi

It is rare that you would need to export all possible combinations of consolidations. Normally if you turn skip consols off, you would use subsets to limit probably all but one dimension to leaf level. Why do you need to export all possible combinations of consols?

Regards

Paul Simon

Hi,

the VMM setting didn't helped.

So i created a Cube, where i transfer all my Data and add the values so i can export the data from this cube. So my new Cube has only leaf level elements.

Memo