Replication:transaction log file

[Drg]

Hi to all.
Does anyone have any knowledge of how replication happens ??
How does TM1 know which transaction log(tm1s[TIMESTMP]) file is being replicated for replication?
What happens if replication fails, but a new transaction log(tm1s[TIMESTMP]) file has already been created?
What happens if a part of data is replicated and part is not ?As in this case tm1 collects pieces of transaction log files. how tm1 do it? where store new custom transaction log file?

Where are checkpoints stored in tm1 witch last good replication?

[mattgoff]

Most of your questions are answered in the documentation: https://www.ibm.com/support/knowledgece ... ss_N1106BB.

Traversal of multiple tx log files is handled automatically-- doesn't matter if data is still in tm1s.log or in a timestamped log. If rep is not 100% successful TM1 will start from same start date/time at next rep (either immediate retry or next scheduled). Date/time of last successful repsync in }ConnectionProperties cube. Once rep completes successfully, date/time is updated there.

Matt

[Drg]

Most of your questions are answered in the documentation: https://www.ibm.com/support/knowledgece ... ss_N1106BB.

Traversal of multiple tx log files is handled automatically-- doesn't matter if data is still in tm1s.log or in a timestamped log. If rep is not 100% successful TM1 will start from same start date/time at next rep (either immediate retry or next scheduled). Date/time of last successful repsync in }ConnectionProperties cube. Once rep completes successfully, date/time is updated there.

Matt

All right, but then, if at the moment the sync. error occurred, then further replication is impossible. IN next Tm1 will start replication from the last successful one, followed by the transaction log generating the error in replication. TRUE?
What I see when I analyze the replication log files is that on some day I had a replication error on the "A" cube, let's assume rollback was triggered.
Then by logic At the next replication, if the data in the "A" cube did not change the replication again will not work and roll back, But I see that the replication is successful. What do I miss in the official docs?

[mattgoff]

What I see when I analyze the replication log files is that on some day I had a replication error on the "A" cube, let's assume rollback was triggered.
Then by logic At the next replication, if the data in the "A" cube did not change the replication again will not work and roll back, But I see that the replication is successful.

I'm not fully understanding your scenario, particularly the part I bolded/italicized. The only reason repsync should fail is due to a network issue. If a repsync fails, there's no reason it should fail again on the next attempt (unless the network issues persist).

Matt

[tomok]

The Replication/Synchronization feature in TM1 is not the most robust feature of the product and I don't think anything about is has changed in quite a few years. Relying on text files and timestamps on text files is not, IMO, technically sound. However, it can be a useful tool if used judicially. If you intend to use it then here are a few things that I always make sure to do. YMMV.

• If you can avoid it, do not synchronize dimension updates and rule changes. If the main point of the synch is to make sure data is the same across servers then you may not need it. Although synchronizing dimensions is not terrible, it can add instability to the synch. IN NO SITUATION SHOULD YOU SYNCH RULE CHANGES!!! You are asking for trouble if you do this.
• Do a thorough review of your logging settings on cubes and turn off logging wherever possible. Keep in mind that the synch process uses log files to see what needs processing. Clogging those files with unnecessary entries will slow things down. For example, do not log things that can be recovered by a TI process. An example would be daily updates from your GL system. Why log that when the answer can just be re-run it if the server crashes? Try to only log manual inputs, if possible.
• Don't synch too often. Try to do it maybe twice a day and, if possible, when a bunch if users are not in TM1 entering data.

As long as you understand the concept of the log files being used and that the bigger they are the worse off you are, and you adjust accordingly, then rep/synch can be a good feature/

[mattgoff]

The Replication/Synchronization feature in TM1 is not the most robust feature of the product and I don't think anything about is has changed in quite a few years.

I'm at the point now where I think it is reliable if you set everything up right. I do agree that it should be used with great caution, especially for TM1 newbies, but I think if someone is willing to put in the effort it's a solid tool.

• If you can avoid it, do not synchronize dimension updates and rule changes. If the main point of the synch is to make sure data is the same across servers then you may not need it. Although synchronizing dimensions is not terrible, it can add instability to the synch. IN NO SITUATION SHOULD YOU SYNCH RULE CHANGES!!! You are asking for trouble if you do this.

Interesting. Why? I've been syncing dims and rules for a decade w/o issue. The only sync issue I've dealt with over that time was related to cube data that TM1 thought was fully synched but was not. That one hit me on-and-off for a long time and is insidious since it fails silently. I still have a control cube with checkvalues from each planet and a process on the star that emails me if they're mismatched, but I haven't had that hit me in years.

• Do a thorough review of your logging settings on cubes and turn off logging wherever possible. Keep in mind that the synch process uses log files to see what needs processing. Clogging those files with unnecessary entries will slow things down. For example, do not log things that can be recovered by a TI process. An example would be daily updates from your GL system. Why log that when the answer can just be re-run it if the server crashes? Try to only log manual inputs, if possible.

Agree on all-- best practice for everyone, not just replication users. You can trim a lot of storage and speed up crash recovery by doing this right.

• Don't synch too often. Try to do it maybe twice a day and, if possible, when a bunch if users are not in TM1 entering data.

In one of the 10.2 releases they made most of the sync non-blocking, so users don't even know it's happening. I sync every hour and I can see users interacting with the server w/o issue while it runs. It was always non-blocking on the star-- they just fixed it on the planet side.

Matt

[lotsaram]

In one of the 10.2 releases they made most of the sync non-blocking, so users don't even know it's happening. I sync every hour and I can see users interacting with the server w/o issue while it runs. It was always non-blocking on the star-- they just fixed it on the planet side.

Interesting (and still great) that sync has become more robust and reliable.

Although now with central cloud server and Restful API UIs I can't see much need for a Star-Planet setup anymore. I suspect the replication API is something IBM would rather didn't exist.