Tm1 10.2.2 - Weird start up issues

[danielpeeroo]

Hi All

So we have recently upgraded one of our smaller prod servers from 9.5.2 to 10.2.2 which is being used for some new development before we release to users. However we are getting some rather erroneous issues when the Tm1 service automatically starts up.

So basically we have a Ti process that brings down the server at 4 am and then a scheduled bat program that starts it up a while later. However on start up in most of the instances it seems to do the following:

1) Starts up but with no security. Therefore as any user other than admin (even users with admin access) you cant see any cubes, dimensions or anything at all even though it allows you to connect. However even using the admin user there are no security privileges in that we cant access logs, or refresh security or do a save data all. All these options are greyed out. I've looked in the message log and there are no errors on the start up to point me towards why the security has gotten out of kilter. So if I then go and do the last resort of restarting the service from Admin Tools-Services when the model starts up everything is back to normal.
2) We do a save data all on restart yet for some reason on numerous occasions the data we have loaded the day before hasn't been there after start up. The server completes the save data function and then on the next restart the data is gone? We are loading millions of records from an amazon rds box and as a result we turn off the cube logging in the TI Process but I the save data all should still be keeping the data that is already in the cube? This one worries us more as we are having to reload data quite frequently and when this server goes live missing data will be a bigger issue. A

So a further thought was on the Cognos config tool. So far we don't really use it to do start ups as we use the services directly in the admin tools - with this newer version are there any other services we may need to restart in order to replicate what the cognos config tool does? Reading a lot of other forums I cant see that any one else has had this issue so we are a bit stuck at how to fix this so any help would be most appreciated as I assume its something to do with our restart procedure which worked for 9.5.2. but may not work for 10.2.2.

Many Thanks for any help in advance

dan

[pandinus]

Can you please explain why you use a TI script to shut down the server and a scheduled batch script to get it started again?
In all my implementations I use a batch script for both stopping and starting the server instance (doing a copy of the data folder in between).

I am not saying this is related to your issue, but it's just me being curious.

[danielpeeroo]

Hi

I think it was originally done in the same way i.e. bat files for both start up and stop but from what I've been told for 9.5.2 they changed it to be shut down by a two process chore with the first deleting the feeder files and the second then shutting down the server. Its just legacy I think - in that its the way they have done all of their servers so we are just being consistent.

Thanks

Dan

[BrianL]

I think it was originally done in the same way i.e. bat files for both start up and stop but from what I've been told for 9.5.2 they changed it to be shut down by a two process chore with the first deleting the feeder files and the second then shutting down the server.

If you don't want feeder files, just disable PersistentFeeders=F in the tm1s.cfg. However, having the .feeder files typically has a large improvement in startup times.

I'm very curious as to what your TI's are actually doing to shutdown TM1. Is the chore running in multiple-commit mode, or single commit? It could be that the newer version of TM1 isn't as forgiving of any manual steps you are taking.

Are there any errors or warning messages in the tm1server.log file when this happens?

[danielpeeroo]

Hi

Yes we need the feeder files as we need the persistent feeders for the model start up, if we ever need to restart the model during business hours.

So apologies as on further investigation it seems all the TI does it run a bat file to stop the Tm1 service.

Regarding the Chore it is in Single Commit mode.

Looking through the logs I have found a feeder error for a cube actually. never seen this error before and it says "Feeder file error/inconsistency detected for Subs Revenue cube reason: Previously found a cube with rules but no feeder file but there is a feeder file for this cube" - see screenshot attached

Now the chore that runs at 3 am is supposed to delete all feeder files so when this model is starting up there should be no feeder files in the data folder anyway so I'm not sure why it would get this error unless there was a feeder file in the data directory?

[BrianL]

Yes we need the feeder files as we need the persistent feeders for the model start up, if we ever need to restart the model during business hours.
...
Now the chore that runs at 3 am is supposed to delete all feeder files so when this model is starting up there should be no feeder files in the data folder anyway so I'm not sure why it would get this error unless there was a feeder file in the data directory?

I'm still confused. You're saying you both need the feeder files and your deliberately deleting them. Why are you deleting them? If persistent feeders are enabled, they will be saved during shutdown anyway. The error message would seem to indicate that it's your messing around with feeder files that's causing your problems.

Also, I don't see any screenshot attached.

[danielpeeroo]

Hi Brian

I understand your confusion but we had a few issues with the feeder files in previous versions. I cant remember which version but defo one of the 9.5's after a rule change in some cases it wouldn't start up the model properly. In one case it started up our Production cubes with no rules calculating. So since then we found it safer to have them deleted and rebuilt on start up and then any emergencies during the day we can fire up the model with the feeder files which saves us a lot of downtime during the critical business hours. I understand that with newer versions of tm1 this may not be necessary any more so its defo something we could look at changing with the 10.2.2 enhancements to the logic it uses for them in start up.

So we will look at changing the way our persistent feeders logic currently works and see if that sorts out the issue. But it just seems weird that an error with a feeder file would cause the security to be affected and cause data not be locked into memory. But we'll try it and see if the issue goes away

Thanks for your help

Apologies as it didnt seem to attach first time but its there this time defo

[BrianL]

I believe this is caused by PI16796.

In some releases, TM1 only handles when ALL feeder files are present or when they are ALL missing. If some cubes have feeder and some don't the server startup can have issues. I believe this is fixed in 10.2 RP2 FP1.

As a workaround, make sure if you're deleting feeder files you delete all of them. This is probably best achieved AFTER the server has shutdown so you know TM1 won't be creating any more .feeder files.

[paulsimon]

Hi Daniel

I would suggest that you first optimise your feeders to eliminate over-feeding, and then see if you really need the feeders. I have had so many problems with Persistent Feeders that I have always turned that option off.

Certainly in 10.2.1 there was bug whereby Persistent Feeders were actually causing the server to crash, so yes it recovered faster but it wouldn't have gone down in the first place if it hadn't been for the persistent feeders. Turning that option off reduced the crashes, although it took a hot fix to finally stop them.

Regards

Paul Simon