Save Data/Server Shutdown/Backup questions...

[tcasey]

Save Data/Server Shutdown/Backup questions...

I've done a lot of reading/searching the internet/this forum and IBM documentation, and not finding it easy to put this together. I am an accountant (so be gentle) and have been the sole developer of our TM1 budgeting applications for about two years now. (We run the server in our dept and have in-house IT staff who looks after the server for us).

Up til now I have been manually saving data, shutting down the server and doing a backup. I want to automate this now, and followed the info in Nightly/Weekly TM1 Restart Implementation and Nightly TM1 backups:

http://blog.tm1tutorials.com/2011/10/03 ... mentation/
http://blog.tm1tutorials.com/2011/10/04 ... 1-backups/

Here is what I have done and my questions/problems:

-running server shut down from right click server/server manager/shutdown server works
-ti process to shut down server does not work completely (ti process obtained in above posts)

in Prolog
SaveDataAll();
ServerShutDown(0);

It takes down the server from the client TM1 Perspectives Server Explorer, but running this command in a dos window:
sc \\eh-cognos query budgetingtest

shows the server is still running, so probably not safe to start a backup from the same ti process using xcopy?

-save data:
right click a server while logged in as admin I see the following:

Menu Items Available (not greyed out)
-------------------------------------- --------------------------
Save Data Yes
Recycle(Clear memory for Local server) No
Shutdown No
View Transaction Log... Yes
View Message Log... Yes
View Audit Log... No
Process Audit Log Events... No
Start Performance Monitor Yes
Deferred Updates Yes
Server Manager Yes
Cancel Shutdown No
Who Am I Yes
Login As... No
Log Out Yes

When I left click save data, the menu immediately disappears. When I right click the menu again, Save Data is available (not greyed out). I can wait 20 minutes and the same. It appears it does not save the data. If it did, and nothing else changed (as nobody else using the server or logged in) wouldn't Save Data be greyed as nothing to save? Or is it possible this is saving OK and normally how it works?

From my reading, I understand TM1 Server can be set up as a service or an application. We have it running as a service. I've also read in the IBM Cognos TM1 Back-up and Recovery Guide that "if you are running TM1 Server as a service, shutting down the service will automatically cause a 'Save Changes' to occur."

So my concerns are, while running TM1 Server as a service, what is the appropriate method to Save Data, Restart TM1 to free "garbage memory", and backup the Data/Log directories.

Thanks,

...Tom

[tomok]

Why do you feel the need to shut down the server in order to do a backup? It's not necessary as the TM1 application doesn't maintain a lock on the files, except for the tm1s.log file. As long as you exclude the tm1s.log file from the backup and you issue a SaveData command before the backup starts, then everything is hunky dorey.

[tcasey]

Thanks tomok for your response.

My reason for wanting to take down the server to do backups is two fold:

- The IBM Cognos TM1 Back-up and Recovery Guide states that doing a live backup has the potential risk of a loss of data between "saves" if the data directory files become corrupt.

- Also I've read (and we've experiencesd this ourselves) that TM1 does not release memory once it has been used, so over time this can cause a problem with server resources, that only a restart will fix.

We only use TM1 during office hours, so taking it down at night is no issue.

I still don't know if running TM1 Server as an application or a service has impact on how Save Data works. Can someone answer my questions above how sava data does not seem to do anything? Does it look that way to everyone else? I hit Save Data in Server Explorer as admin and the menu disappears with no response it did anything. I look in the menu again and Save Data is not greyed out. How can I know if Sava Data done this way or run in a TI process actually works?

Thanks,

...Tom

[qml]

Also I've read (and we've experiencesd this ourselves) that TM1 does not release memory once it has been used, so over time this can cause a problem with server resources, that only a restart will fix.

Yes, but some of this memory is used by cached results, which can speed up reporting in your model significantly. If you stop the service, you lose the cached data. It's a trade-off. You gain some, you lose some.

I look in the menu again and Save Data is not greyed out.

It doesn't ever get greyed out, not even if there is nothing to save.

How can I know if Sava Data done this way or run in a TI process actually works?

It will have worked if you are able to find a newly created tm1s*.log file in the logging directory, where * will be the current timestamp.

[tcasey]

Thanks qml...

I didn't think of looking for the saved log file and new one being created. I'll check that out.

Anyone know why a ServerShutDown(0); run from a ti process only takes down the TM1 client and not the server?
Does it work when running TM1 Server as an application but not when running TM1 Server as a service?

[lotsaram]

ServerShutDown only works if the TM1 server is running as an application not as a service (I think this is covered in the operations manual ... somewhere, but who knows with TM1 documentation )

By the sounds of it your application is small and there is no way you need t bother with a daily server restart but some form of regular restart can be beneficial for performance (weekly or monthly maybe).

For a TM1 server running as a service the "standard" approach for automatically scheduling a restart would be:
1/ run a scheduled SaveDataAll
2/ wait long enough for the normal maximum time for a data save
3/ execute a batch file with a Net Stop "<service name>"
4/ wait again just in case the data save wasn't complete
5/ execute a Net Start "<service name>"
... if using 9.5 with PersistentFeeders=T
6/ wait a few minutes
7/ execute a 2nd Net Start "<service name>" in case the 1st server start failed due to inconsistencies with persistent feeders
(note 6 & 7 don't apply to you since you aren't using that version but are there for completeness. If a net start is called and the service is already runnign then the commend has no effect.)

There are different ways to do this. Steps 3 - 7 could all be part of the same batch file and called with ExecuteCommand from the same TI that runs the SaveDataAll. Or they could be separate batch files called with the windows scheduler in which case you don't need to code in the waits.)

[tcasey]

Thanks lotsaram, very helpful...

I (again, not knowing a lot about the server side) have been given security access to stop, query, and start the server from a dos window using these sample commands:

sc \\eh-cognos stop budgetingtest
sc \\eh-cognos query budgetingtest
sc \\eh-cognos start budgetingtest

So will running these commands in an execute command from ti work the same as the "Net Stop "<service name>" and Net Start "<service name>" ? I don't know if net stop/start is right for our server setup?

Also how can you insert wait time in a ti process, is there like a sleep command or something?

[qml]

So will running these commands in an execute command from ti work the same as the "Net Stop "<service name>" and Net Start "<service name>" ?

Yes, sc is just another command line program used to control and access services.

Also how can you insert wait time in a ti process, is there like a sleep command or something?

Not directly, but you can try inserting the following line into TI to get the same effect (the below one gives a delay of 2000ms = 2s):

Code: Select all

ExecuteCommand ('ping 127.0.0.1 -n 2 -w 2000', 1);

[lotsaram]

You can put a "wait" into TI but it is not quite the same as a true pause or sleep ...

nWaitSec = 120;
nStop = NOW + nWaitSec / 86400;
While( NOW < nStop );
#holding pattern
End;

This will "pause" a TI process for the value of nWaitSec but the reality is the CPU running the TI will continue to churn away furiously reevaluating the server clock time until the condition is broken. Much better to use a batch script with a sleep command. But as not all Windows OS support sleep the most universal and bombproof way of programming in a wait is the one given by qml of a ping request with a wait time.

[Kyro]

Instead of making complicated batch processes to wait, use windows scheduled tasks.

• Use a scheduled TI To Save and Shutdown the Server
• Have a Windows Scheduled Task Archive and Start the Server about an Hour later

Just a comment on running Backups while the server is running: IBM makes a point the issue of data loss between backups from a live backup but in reality the same risk applies to shutting the server down and then backing up the server.
In both cases:

• Logged changes are committed to the data directory and the log is renamed.
• Those changes (to the data files) are backed up. (The shutdown process unnecessarily backs up old logs which have been committed to the files)
• If the server has an issue where the drives melted, You would still lose the changes in between backups because you would lose the logfile of changes since the last data save.

Your best defence against this is a scheduled TI Process which Saves the Database every 15/30 or 60 min along with a nightly backup to a network drive - because ultimately drives don't melt very often and more often than not you'll simply be backing out transactions from users who accidentally hit "C" and enter at the top of your Profit & Loss.... true story...

For reference for others - Here is what IBM States in reference to live backups:

The disadvantages are that there is initial configuration that
must be performed and the potential exists for lost data between “Saves” if
the Data Directory files become corrupt.

[tcasey]

Thanks to all for you replies...very helpful. I'll post back in a few days with my results once I get some time to try this out.

...Tom

[RT1107]

Hi all,

I am trying to stop the tm1 server running as a windows service. I have a TI Process as discussed above to execute the batch file. The TI runs successfully but has no effect on the service. It still shows started as the status.

I have both the TM1 applications on my local machine and OS is Windows 7. What I observed is that when I run the batch file manually, using "Run as Administrator" option, it brings the service down. But when the same is executed from TI Process it has no effect.

How can I make the TI Process to execute the batch file as an administrator?
Any suggestions?

Thanks,
Aarti

[qml]

Try executing your shell command(s) like that:

Code: Select all

runas /user:Administator "cmd.exe <rest of your command here>"

Edit: This will still promt for password, so it's no good. If you are allowed to install 3rd party software then there is a tool called Elevate written by John Robbins that you can use to execute any other program with elevated rights (in this case - cmd.exe). Maybe someone else will have a better idea.

[jameswebber]

In order to fix stopping and staring services within TM1 you can change the credentials the service uses.
I have created a domain account that is a local admin on the box. I have changed my congos express services to run under this account. Now TM1 can run a command lines like:

Code: Select all

vCmd = 'net stop "IBM Cognos Express Advisor"';
ExecuteCommand(vCmd,1);


vCmd = 'net start "IBM Cognos Express Advisor"';
ExecuteCommand(vCmd,1);

On the whole backup thing, I was using the inbuilt Cognos Express Backups but these are a waste of time.
Because it stops all the services to take the backup and can only be changed by a reboot. Also the files are huge and if you delete one backup via file explorer you need to delete the whole lot (see http://www.tm1forum.com/viewtopic.php?f=3&t=6499)

I did play with using ServerShutDown(0);
I don't like ServerShutDown(0) seems temperamental.
I have decided if I am going to bring the server down I might as well reboot it and apply windows updates.

So nightly instead I am backing up the data and dimensions and ziping this.

I schedule a chore with a process to save data all then a second process to run my backup script

Code: Select all

vCmd = 'C:\Scripts\CreateTM1Backups.bat';
ExecuteCommand(vCmd,1);

The batch file looks like this (I use 7zip to zip it up)

Code: Select all

REM  This batch file creates TM1 backups.
REM I was using the inbuilt backup but it stops the server and if you delete one backup you can't see the rest in the web interface so it's useless
REM Run as the xxx User on the local server
REM Date time added to zip files so they can be run more than once in a day

set dt=%date:~10,4%%date:~4,2%%date:~7,2%_%time:~0,2%%time:~3,2%%time:~6,2%


set zipname="C:\Program Files (x86)\IBM\Cognos Express Backups\%dt%_CXMD.zip"
7za.exe a %zipname% "C:\Program Files (x86)\IBM\Cognos Express\Xcelerator\Custom\TM1Data\CXMD"

set zipname="C:\Program Files (x86)\IBM\Cognos Express Backups\%dt%_Dimensions.zip"
7za.exe a %zipname% "C:\Program Files (x86)\IBM\Cognos Express\Xcelerator\Custom\TM1Data\Dimensions"

I then have another batch file that tidies up my zip files periodically.

I see Kyros backup http://blog.tm1tutorials.com/2011/10/04 ... 1-backups/ does something simlar. But hadn't seen this at the time of writing the code.

[RJ!]

We've been having issues with out backups of late, our main GL cube has been resisting our efforts to have it backed up (via RoboCopy) and after a SaveData we end up with a GL.cub$ file.

Currently we have 3 SaveData chores set up: 7am, 12pm & 8:40pm.
After the 8:40pm SaveData we have a UnloadCube process to unload the GL Cube from Memory. The RoboCopy is then scheduled at 9pm.

Odd thing is the 12pm SaveData is our last good save of the GL cube, this is when the Cube is more likely to be used...

Anyone have experiance with incomplete saves of large cubes?

[K Vuong]

Hi RJ,

We had this problem recently that the nice people at Infocat help to resolve.

Can you check to see if...
• That the 'SaveDataAll' command is in the Epilog
• That the chore does not include any other processes

I personally thought it didn't really matter where you put the Savedataall (prolog or Epilog) but apparently it works.

Hope this helps,

Ken

[RJ!]

Hi Ken,

Tried that method and still have the same issues...

Error lines in the log file:

10140 [1a8] ERROR 2014-12-19 10:27:19.764 TM1.Server sf_Rename: Failed to rename (\\LocalHost\tm1\systemfiles\production\prod_data\GL.cub) to (\\LocalHost\tm1\systemfiles\production\prod_data\GL.cub$$$). Error: error code:32 reason:"The process cannot access the file because it is being used by another process."

10140 [1a8] ERROR 2014-12-19 10:27:21.777 TM1.Server sf_SafeRename() could not rename file, directory="\\LocalHost\tm1\systemfiles\production\prod_data\", "GL.cub" to "GL.cub$$$" after 10 retrys in 20 seconds. Operation Error: 2

I found the below on the IBM site:

Error description
If file is locked when data save is executed, customer has
requested that TM1 retry the rename (and save) and/or recognize
that a .CUB$ file exists and rename it to .CUB on server
restart.

****************************************************************
* USERS AFFECTED: *
* All Users *
****************************************************************
* PROBLEM DESCRIPTION: *
* See error description. *
****************************************************************
* RECOMMENDATION: *
* Upgrade to TM1 10.1 RP1 FP2 *
****************************************************************

We are currently on build number 10.2.20100.123 so looks like another PMR to be logged with IBM as this isn't fixed...

[gtonkin]

HI RJ,
I have had a similar issue and it went away once I disabled the anti-virus from scanning All files with extensions of:
• CUB, CUB$*
• DIM, DIM$*
• PRO, PRO$*
• CHO, CHO$*
• LOG, LOG$*
• SUB, SUB$*
• RUX, RUX$*
• BLB, BLB$*
You may want to exclude the whole data folder first and see if it helps, if it does then apply per above.
Good luck.

[RJ!]

Hi Mate,

Seems to be only 1 file that is an issue, all of our other files are able to be saved without issue.
It's only our main Cube that has this issue.

RJ

[BrianL]

Seems to be only 1 file that is an issue, all of our other files are able to be saved without issue.
It's only our main Cube that has this issue.
RJ

Makes sense to me. If antivirus is interfering, you're most likely to see problems with the biggest cube. It's the largest file so any operations (such as a virus scan) will take longer and have an increased chance of causing tm1 problems.

There's always the option of using ProcessExplorer.exe or similar tool to find out what's blocking TM1 with an open file handle.