Replication:transaction log file

Post Reply
Drg
Regular Participant
Posts: 159
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

Replication:transaction log file

Post by Drg »

Hi to all.
Does anyone have any knowledge of how replication happens ??
How does TM1 know which transaction log(tm1s[TIMESTMP]) file is being replicated for replication?
What happens if replication fails, but a new transaction log(tm1s[TIMESTMP]) file has already been created?
What happens if a part of data is replicated and part is not ?As in this case tm1 collects pieces of transaction log files. how tm1 do it? where store new custom transaction log file?

Where are checkpoints stored in tm1 witch last good replication?
:?: :?: :?: :?: :?:
User avatar
mattgoff
MVP
Posts: 516
Joined: Fri May 16, 2008 1:37 pm
OLAP Product: TM1
Version: 10.2.2.6
Excel Version: O365
Location: Florida, USA

Re: Replication:transaction log file

Post by mattgoff »

Most of your questions are answered in the documentation: https://www.ibm.com/support/knowledgece ... ss_N1106BB.

Traversal of multiple tx log files is handled automatically-- doesn't matter if data is still in tm1s.log or in a timestamped log. If rep is not 100% successful TM1 will start from same start date/time at next rep (either immediate retry or next scheduled). Date/time of last successful repsync in }ConnectionProperties cube. Once rep completes successfully, date/time is updated there.

Matt
Please read and follow the Request for Assistance Guidelines. It helps us answer your question and saves everyone a lot of time.
Drg
Regular Participant
Posts: 159
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

Re: Replication:transaction log file

Post by Drg »

mattgoff wrote:Most of your questions are answered in the documentation: https://www.ibm.com/support/knowledgece ... ss_N1106BB.

Traversal of multiple tx log files is handled automatically-- doesn't matter if data is still in tm1s.log or in a timestamped log. If rep is not 100% successful TM1 will start from same start date/time at next rep (either immediate retry or next scheduled). Date/time of last successful repsync in }ConnectionProperties cube. Once rep completes successfully, date/time is updated there.

Matt
All right, but then, if at the moment the sync. error occurred, then further replication is impossible. IN next Tm1 will start replication from the last successful one, followed by the transaction log generating the error in replication. TRUE?
What I see when I analyze the replication log files is that on some day I had a replication error on the "A" cube, let's assume rollback was triggered.
Then by logic At the next replication, if the data in the "A" cube did not change the replication again will not work and roll back, But I see that the replication is successful. What do I miss in the official docs?
User avatar
mattgoff
MVP
Posts: 516
Joined: Fri May 16, 2008 1:37 pm
OLAP Product: TM1
Version: 10.2.2.6
Excel Version: O365
Location: Florida, USA

Re: Replication:transaction log file

Post by mattgoff »

What I see when I analyze the replication log files is that on some day I had a replication error on the "A" cube, let's assume rollback was triggered.
Then by logic At the next replication, if the data in the "A" cube did not change the replication again will not work and roll back, But I see that the replication is successful.
I'm not fully understanding your scenario, particularly the part I bolded/italicized. The only reason repsync should fail is due to a network issue. If a repsync fails, there's no reason it should fail again on the next attempt (unless the network issues persist).

Matt
Please read and follow the Request for Assistance Guidelines. It helps us answer your question and saves everyone a lot of time.
tomok
MVP
Posts: 2831
Joined: Tue Feb 16, 2010 2:39 pm
OLAP Product: TM1, Palo
Version: Beginning of time thru 10.2
Excel Version: 2003-2007-2010-2013
Location: Atlanta, GA
Contact:

Re: Replication:transaction log file

Post by tomok »

The Replication/Synchronization feature in TM1 is not the most robust feature of the product and I don't think anything about is has changed in quite a few years. Relying on text files and timestamps on text files is not, IMO, technically sound. However, it can be a useful tool if used judicially. If you intend to use it then here are a few things that I always make sure to do. YMMV.
  • If you can avoid it, do not synchronize dimension updates and rule changes. If the main point of the synch is to make sure data is the same across servers then you may not need it. Although synchronizing dimensions is not terrible, it can add instability to the synch. IN NO SITUATION SHOULD YOU SYNCH RULE CHANGES!!! You are asking for trouble if you do this.
  • Do a thorough review of your logging settings on cubes and turn off logging wherever possible. Keep in mind that the synch process uses log files to see what needs processing. Clogging those files with unnecessary entries will slow things down. For example, do not log things that can be recovered by a TI process. An example would be daily updates from your GL system. Why log that when the answer can just be re-run it if the server crashes? Try to only log manual inputs, if possible.
  • Don't synch too often. Try to do it maybe twice a day and, if possible, when a bunch if users are not in TM1 entering data.
As long as you understand the concept of the log files being used and that the bigger they are the worse off you are, and you adjust accordingly, then rep/synch can be a good feature/
Tom O'Kelley - Manager Finance Systems
American Tower
http://www.onlinecourtreservations.com/
User avatar
mattgoff
MVP
Posts: 516
Joined: Fri May 16, 2008 1:37 pm
OLAP Product: TM1
Version: 10.2.2.6
Excel Version: O365
Location: Florida, USA

Re: Replication:transaction log file

Post by mattgoff »

tomok wrote:The Replication/Synchronization feature in TM1 is not the most robust feature of the product and I don't think anything about is has changed in quite a few years.
I'm at the point now where I think it is reliable if you set everything up right. I do agree that it should be used with great caution, especially for TM1 newbies, but I think if someone is willing to put in the effort it's a solid tool.
tomok wrote:
  • If you can avoid it, do not synchronize dimension updates and rule changes. If the main point of the synch is to make sure data is the same across servers then you may not need it. Although synchronizing dimensions is not terrible, it can add instability to the synch. IN NO SITUATION SHOULD YOU SYNCH RULE CHANGES!!! You are asking for trouble if you do this.
Interesting. Why? I've been syncing dims and rules for a decade w/o issue. The only sync issue I've dealt with over that time was related to cube data that TM1 thought was fully synched but was not. That one hit me on-and-off for a long time and is insidious since it fails silently. I still have a control cube with checkvalues from each planet and a process on the star that emails me if they're mismatched, but I haven't had that hit me in years.
tomok wrote:
  • Do a thorough review of your logging settings on cubes and turn off logging wherever possible. Keep in mind that the synch process uses log files to see what needs processing. Clogging those files with unnecessary entries will slow things down. For example, do not log things that can be recovered by a TI process. An example would be daily updates from your GL system. Why log that when the answer can just be re-run it if the server crashes? Try to only log manual inputs, if possible.
Agree on all-- best practice for everyone, not just replication users. You can trim a lot of storage and speed up crash recovery by doing this right.
tomok wrote:
  • Don't synch too often. Try to do it maybe twice a day and, if possible, when a bunch if users are not in TM1 entering data.
In one of the 10.2 releases they made most of the sync non-blocking, so users don't even know it's happening. I sync every hour and I can see users interacting with the server w/o issue while it runs. It was always non-blocking on the star-- they just fixed it on the planet side.

Matt
Please read and follow the Request for Assistance Guidelines. It helps us answer your question and saves everyone a lot of time.
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Replication:transaction log file

Post by lotsaram »

mattgoff wrote:In one of the 10.2 releases they made most of the sync non-blocking, so users don't even know it's happening. I sync every hour and I can see users interacting with the server w/o issue while it runs. It was always non-blocking on the star-- they just fixed it on the planet side.
Interesting (and still great) that sync has become more robust and reliable.

Although now with central cloud server and Restful API UIs I can't see much need for a Star-Planet setup anymore. I suspect the replication API is something IBM would rather didn't exist.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Post Reply