startup hang after }StatsForServer

[image2x]

Over the last couple days, our tm1 instance has gone from a 10 minute startup to a 40 minute startup. Much development has been done in that period so unfortunately I can't point to a single change. Performance once it starts up doesn't seem to have changed.

It still takes only 10 minutes to run thru all the way up to "Done Loading Cube }StatsForServer". It will then prompt to recover changes (yes/no doesn't make a difference). And then it takes 30 minutes to get to the final "TM1 Server is Ready" entry (the virtual core is pegged at 100% during this time). TM1s.cfg is configured with MaximumCubeLoadThreads=0.

Anyone know what TM1 does in the final stage between cube loads being finished and server startup? Security perhaps? We have added 60+ TM1 groups over the last couple days but with sparse user assignment.

Thanks again to forum... I hope to get to the point where I can do more than just pepper with questions in the future!

-- John

[Alan Kirk]

Over the last couple days, our tm1 instance has gone from a 10 minute startup to a 40 minute startup. Much development has been done in that period so unfortunately I can't point to a single change. Performance once it starts up doesn't seem to have changed.

It still takes only 10 minutes to run thru all the way up to "Done Loading Cube }StatsForServer". It will then prompt to recover changes (yes/no doesn't make a difference).

If you're seeing that at all (and I'm assuming that you're therefore running the server as an application rather than a service, since a service will automatically assume "Yes" to that), you already have a problem.

That dialog will only appear if there is a tm1s.log file in your logging directory. And that means that your server didn't do a full data save before it shut down. It would be worthwhile to investigate the cause of that, and how the server is being shut down because it looks like the application is being terminated before it's ready to be.

If there's a log file in existence and you do reload it, then the server will take some time to read in all of the data from the log file before it makes itself available again. If it's a very large log file, that could take a while. However you're saying that there's no difference in the speed regardless of whether you accept or reject the unsaved changes, and if that really is the case it's not the central issue.

And then it takes 30 minutes to get to the final "TM1 Server is Ready" entry (the virtual core is pegged at 100% during this time). TM1s.cfg is configured with MaximumCubeLoadThreads=0.

Anyone know what TM1 does in the final stage between cube loads being finished and server startup? Security perhaps? We have added 60+ TM1 groups over the last couple days but with sparse user assignment.

One thing it does is to activate any of the chores that are supposed to be scheduled. In an early release of version 9.1 the BigDSter reported a massive blowout in the time that it took to do that from 1 second to about 10 seconds per chore which elevated his startup time (with 84 active chores) for that part of the startup from 84 seconds to 14 minutes (http://applixforum.olapforums.com/viewP ... adID=11127) though I've not heard of that problem with other versions.

Still, it may be worth looking into. The activation time for each chore should be visible in the message log.

[Martin Ingram]

Security perhaps? We have added 60+ TM1 groups over the last couple days but with sparse user assignment.

Quite possibly - I've been testing on 9.5 the last couple of days and noticed something similar. We've been taking a look at Contributor which automatically adds additional user groups to handle the approval heirarchy - this change led to start-up time increasing to about 30mins (up from about 2 mins).

Easy way to isolate the issue would be to run up another version of the server in parallel, with the new user groups deleted, and see if it changes the start-up time.

[image2x]

Thanks for info guys...

I did more testing and it is definitely is group related. It's disappointing that the security overhead on startup seems so disportionate to the cube load times.

I'm wondering if cube design factors like overfeeding play a big role in driving security setup time. I also think I'll try exploring if some of the groups can be reduced by using rules...

[Steve Rowe]

The normal thing that drives start up time is the amount of feeding that takes place.

Not sure if you have security rules with feeding (probably not given the other thread that seems to suggest that feeding security rules breaks the object).