TI generate file slowly

[wuruimao]

I used the same TI to generate several cma files,and watching the size of the cma file updating. I found that the speed of file generation more and more slowly. For example, to generate the first cma file, it's about 1M per second, very quick,however, when it was generating the 10th fie, the speed was about 10k per second, too slow..Seeking for your useful idea, my dear friends.

[Michel Zijlema]

I moved the post from the General to the Cognos TM1 forum.
BTW, you have an interesting TM1 version listed in your profile...

[wuruimao]

any suggestion? I just typed in the version carelessly...Sorry about that..

[lotsaram]

What is the disk space like on the server being written to, and what is the general health and performance if the resource being used for the file share?

We have had this issue intermittently (where any TI writing out has horribly slow performance but otherwise performance is normal) and found restarting the server hosting the file share typically resolves the problem. If it is just a disk space issue then just clear some space! (usually once a level of only 1 - 2% available HD is reached things really slow down writing to disk as the Os has to struggle to locate the sectors available to write to and they become increasingly fragmented.)

[Alan Kirk]

I used the same TI to generate several cma files,and watching the size of the cma file updating. I found that the speed of file generation more and more slowly. For example, to generate the first cma file, it's about 1M per second, very quick,however, when it was generating the 10th fie, the speed was about 10k per second, too slow..Seeking for your useful idea, my dear friends.

We don't have anywhere near enough information.

It depends to some extent on the data that you're exporting. If the first file is mostly stored values, it's likely to be fast. If the 10th file is generating rules calculated values it may well be slower.

Similarly if the first file is being written to a drive on the local server, while the 10th one is being written to a network share, you may have network speed issues.

Everything depends on context.

(Lotsa beat me to point 2, but point 1 is also relevant.)

[wuruimao]

What is the disk space like on the server being written to, and what is the general health and performance if the resource being used for the file share?

We have had this issue intermittently (where any TI writing out has horribly slow performance but otherwise performance is normal) and found restarting the server hosting the file share typically resolves the problem. If it is just a disk space issue then just clear some space! (usually once a level of only 1 - 2% available HD is reached things really slow down writing to disk as the Os has to struggle to locate the sectors available to write to and they become increasingly fragmented.)

Thanks for ur reply.
Actually our server has enough space. Everytime the TI process run faster at the begining. But after run 20 times in for loop, it become very very slow..
I thought that something held the RAM after ran the TI several times.

[wuruimao]

I used the same TI to generate several cma files,and watching the size of the cma file updating. I found that the speed of file generation more and more slowly. For example, to generate the first cma file, it's about 1M per second, very quick,however, when it was generating the 10th fie, the speed was about 10k per second, too slow..Seeking for your useful idea, my dear friends.

We don't have anywhere near enough information.

It depends to some extent on the data that you're exporting. If the first file is mostly stored values, it's likely to be fast. If the 10th file is generating rules calculated values it may well be slower.

Similarly if the first file is being written to a drive on the local server, while the 10th one is being written to a network share, you may have network speed issues.

Everything depends on context.

(Lotsa beat me to point 2, but point 1 is also relevant.)

Thanks.
Firstly, all the data I want to output are in the same cube.And all the cma files are writed to same path( even the cma file names are the same(replace the previous one) ).So I feel strange abt the speed issue..

[wuruimao]

another example: to generate a 70M file, if it's the first file to generate, it will only take 2 mins. however, if it's the 20th file, it will take 30 mins..

[Michel Zijlema]

If you're talking about 1st, ..., 20th is this counted from the restart of the server, or is this counting the number of iterations in a single process/chore?

Michel

[wuruimao]

in my java program,
for(int i = 0;i<**;i++){
make a new view;
run TI ----> generate cma file
calculate with the cma file.
}
suppose all the files' size to generate are 70M, then,the first one(i = 0) cost less than 2mins when ' run TI ',when i=19,,'run TI' cost more than 45 min.

i cannot explain why the file generation speed will change from 1M/s -> 10k/s.
because of the RAM?

[Michel Zijlema]

in my java program,
for(int i = 0;i<**;i++){
make a new view;
run TI ----> generate cma file
calculate with the cma file.
}
suppose all the files' size to generate are 70M, then,the first one(i = 0) cost less than 2mins when ' run TI ',when i=19,,'run TI' cost more than 45 min.

i cannot explain why the file generation speed will change from 1M/s -> 10k/s.
because of the RAM?

Did you monitor RAM usage during the operation? Could it be that creating the views is pushing the TM1 Server into the use of virtual memory?

Michel

[csjean]

in my java program,
for(int i = 0;i<**;i++){
make a new view;
run TI ----> generate cma file
calculate with the cma file.
}
suppose all the files' size to generate are 70M, then,the first one(i = 0) cost less than 2mins when ' run TI ',when i=19,,'run TI' cost more than 45 min.

i cannot explain why the file generation speed will change from 1M/s -> 10k/s.
because of the RAM?

Hi,

Is the degradation gradual (ie 1st file 1M/s, 2nd 0.95M/s ... last 10k/s)
or is it by leap (ie 1st .. nth file = 1M/s, n+1 file = 10k/s)?

if its by leaps, it could RAM problems. I would recommend go with Michel's recommendation and monitor RAM (and CPU usage while your at it)...

Hope this helps.