SaveDataAll with Bulk mode invalidates other chores

[mvaspal]

Hi,

I am facing a weird issue, hope someone can help. I saw it in CX 10.1 and TM1 10.1.1 FP1.

We have a SaveDataAll chore, including Enable- and DisableBulkLoadMode. We tried it with 2 scenarios, same result: one TI, EnableBulkLoadMode on Prolog, then Disable on Epilog and SaveDataAll on Prolog (or Epliog). In the other scenario, 3 TIs in the chore, multiple commit mode, 1st TI enables Bulk mode, second saves data, 3rd disables Bulk load mode. I set it to 1 hour frequency (for testing purposes).

What we saw is that after a few hours, it simply invalidates all other chores. These chores get deactivated and whatever we try to do with them (edit, delete, etc.), the setrver hangs. I cannot even stop the thread (of editing, deleting, etc.) in TM1Top. And we could not figure it out what happens when other chores get invalid. No user activity, no other server maintenance, etc. The chore does not run at the same time as any other chores or TIs.

It is important to note that the server does not seem to be stuck in bulk load mode. Users can open cubes, edit data, etc. I'm using an AD account user in TM1 (when registering the chore) and the server runs under a different domain account. However, I created the chore with my AD user.

There are totally no records of any errors in the tm1server.log.

So I would have two questions:
1. Anyone experienced this and could work around?

2. If I cannot use Bulk Load Mode when saving data to disk, how to prevent users from work for this period? Why we tried to use bulk load mode was that running TI processes could get into an infinite loop sometimes when running at the same time as the SaveDataAll.

Thanks
Matyas

[macsir]

When the server enters Bulk Load Mode, all processing by other threads is paused.

Any existing user threads and running chores will be suspended. Only the thread that initiated Bulk Load Mode will remain active. All scheduled chores will be deactivated, except the chore that initiates Bulk Load Mode. All system-specific threads and TM1Top connections will also be suspended.

For the best practice, savedataall should be used in one individual chore alone in order to prevent any file lock issue. Just schedule this saving chore at a proper timing.

[mvaspal]

Hi

Thanks for the reply. As I wrote the server does not seem to be stuck in bulk mode. And interesting is that it is working fine for a while, then just stops working without any user activity or other concurrent chores.

"Just schedule this saving chore at a proper timing." And this is the real problem. If we want to save data during the day then how to prevent users from running anything during save data times other than bulk mode? Of course we can ask them that please do not run processes between 9-9:02, 13:00-13:02 and so on but it is not the real solution.

Thanks again.
Matyas

[macsir]

May I know why you want to use this mode rather than the multi-user mode? Not quite efficient for huge data loading?

[macsir]

Usually, I would let server save data at noon time or midnight. And the saving is pretty quick. It won't block user's operation too much. I don't think it is a problem to me.
Or another workaround, by using TM1 api, you can check if someone is running something before triggering your saving chore. I have seen someone in this forum uses this method.

[mvaspal]

Hi

Thanks for your answers!

May I know why you want to use this mode rather than the multi-user mode? Not quite efficient for huge data loading?

I would have liked to use bulk load mode for SaveDataAll. The reason is that we experienced that sometimes it can concur with running TIs (executed by users) and these TIs may get into an infinite loop and we have to kill them through TM1Top; it seems that users' TI cannot commit in such cases.

And the reason why we prefer running a save data during the day is that there are a couple of TIs that users can run during the day and move considerable amount of data that we did not want to log, so we switch off logging in Prolog and switch on back in the Epilog. However in such cases, if the server box comes down unexpectedly (as it happened 1-2x), data moved by these TIs will be lost.


The interesting thing with SaveData and Bulk mode is that it is working for a while and it also does not seem that the server stuck in the bulk mode.

Thanks again
Matyas

[Duncan P]

Have you thought of using CubeSaveData at the end of the TIs that are not logging?

[mvaspal]

CubaSaveData is a good one, I totally forgot about this option, thanks!

To the topc of the bulk mode and save data: we are using this in 9.5.2 FP2 without any issues, it is just v10 where it started to cause us problems.

Thanks again for the tip.