TM1 Data Load Failing everyday

SBK88 · Post by **SBK88** » Tue May 12, 2015 5:31 am

Hi All,

Help me in this. We have a TM1 model (size approx 78 GB), Windows server R2 enterprise, 64 Bit OS, RAM 640 GB, Intel 2.26GHz Processor, Harddisc 1400GB.(other models also present in the same server).
Our data load is failing againg and again (daily - scheduled).
Data load is running through scripts.( TI uses executecommand process), and that too around 40 scripts it is calling in a single process.
Dalata load is failing on the daily basis, Is it because of the data size of our model, or some technical issue.
Kindly help.

failurehappening · Post by **failurehappening** » Tue May 12, 2015 6:04 am

Hi

What do the error logs say?

SBK88 · Post by **SBK88** » Tue May 12, 2015 6:23 am

Process "MACO_ODBC_DATA_BEID_PER_MEASURE": finished executing with errors.
Error: Prolog procedure line (19): Errors occurred during execution of process "MACO_ODBC_DATA_BEID_PER_MEASURE". Check error log file for that process for more details.

The log for the script running this process shows (NO End) :-
6:45:47.80: Start
6:45:47.81: PERIOD TO RUN = 2015M02
6:45:47.81: admin_recreate_fx_rates_alias_swap
6:45:52.05: admin_recreate_fx_rates
6:45:59.64: ADMIN_CREATE_FILL_ADMIN_PARAMETERS

failurehappening · Post by **failurehappening** » Tue May 12, 2015 6:28 am

Can you post the details of the log for the process MACO_ODBC_DATA_BEID_PER_MEASURE

Additionally, what does the tm1server.log have in it?

SBK88 · Post by **SBK88** » Tue May 12, 2015 7:44 am

TM1server log shows - "Process "MACO_ODBC_DATA_BEID_PER_MEASURE": finished executing with errors."

failurehappening · Post by **failurehappening** » Tue May 12, 2015 7:54 am

It looks like your dimension 'MACO_Staging_Customer Ship-To' is missing a load of elements.

Is there a process to update the dimensions from the source data that you can run?

SBK88 · Post by **SBK88** » Tue May 12, 2015 7:56 am

Script that is running that process "MACO_ODBC_DATA_BEID_PER_MEASURE" is getting stuck again n again somewhere in the midway itself.

SBK88 · Post by **SBK88** » Tue May 12, 2015 8:15 am

Thanxxx failurehappening for replying.
Error is fine , bt sudden stuck of the script is the in the midway is the issue

failurehappening · Post by **failurehappening** » Tue May 12, 2015 8:19 am

It will continue to get stuck until the data is fixed either in the TM1 model or in the source.

From the log:

Code: Select all

"$NOOID","$NOOID","$NOOID","$NOOID","$NOOID","$NOOID","73302100","79123200","$NOOID","$NOOID","K0154433","50632351","50468493","$NOOID","$NOOID","122425508","$NOOID","K100914","109411494","0","0","77.81999999999999",Data Source line (50) Error: Data procedure line (10): Element "122425508" not found in dimension "MACO_Staging_Customer Ship-To"
"$NOOID","$NOOID","$NOOID","$NOOID","$NOOID","$NOOID","73302100","79123200","$NOOID","$NOOID","K0154433","50632351","50468493","$NOOID","$NOOID","122425508","$NOOID","K100914","109411494","0","0","77.81999999999999",Data Source line (50) Error: Data procedure line (69): Invalid key: Dimension Name: "MACO_Staging_Customer Ship-To", Element Name (Key): "122425508"

There seems to be a lot of $NOOID values, which I'm guessing mean that there is no data for that column in the database you are connecting to through your ODBC connection.

The errors in the log file that say
Data procedure line (69): Invalid key: Dimension Name: "MACO_Staging_Customer Ship-To", Element Name (Key): "122425508"
Data procedure line (10): Element "122425508" not found in dimension "MACO_Staging_Customer Ship-To"
means that he process is attempting to do something on line 10 and 69 of the data tab within the process. Have a look at the data tab - what is the process trying to do there?

BariAbdul · Post by **BariAbdul** » Tue May 12, 2015 8:49 am

As failurehappening correctly pointed out this error occurs when element doesn't exist in that particular dimension,Did you check in the dimension "MACO_Staging_Customer Ship-To'' manually whether 122425508" element is there,If not, you need to add it.Also check CellPutN in you data tab ,whether dimensions are defined exactly in the order as it exists in cube .Thanks

SBK88 · Post by **SBK88** » Tue May 19, 2015 9:12 am

Hello Everyone,

Need ur help, We are running a chore which is loading a huge amount of data.
The problem that we are facing is that one or the other process getting stuck everyday.

We have parameters there for our process, ( Update Month - "Month Name", Update Dimension - "Yes/No")
to reduce the load we set Update Dimension as No, then data load got failed coz it didn't get few elements to map n load the data for lot many dimensions.

when we went with "Update Dimension - Yes", processes were taking tooo much time and one or the other process getting stuck..

How can I skip that particular process though TM1TOP so that I can have my data load go smooth.

Kindly help...... facing the same issue since many days.

Thankss in advance.

TrevorGoss · Post by **TrevorGoss** » Tue May 19, 2015 1:25 pm

Hello,

If you have Multiple processes being called via the ExecuteCommand function, or you have the same process being called multiple times, you can always get the return value of the process, that way you can know what the process returend, successful, minor errors etc.

http://www-01.ibm.com/support/knowledge ... uteprocess

The above link is the documentation to the ExecuteCommand function, it describes how to create logs based off return values for TIs.

As for TM1Top, I havnt used it in a while as I am 10.2 but you can cancel TIs whilst they are running, but it is best to discover the Error and fix it before the TI is executed.

Thanks.

mattgoff · Post by **mattgoff** » Tue May 19, 2015 1:48 pm

SBK88 wrote:when we went with "Update Dimension - Yes", processes were taking tooo much time and one or the other process getting stuck..

My guess is your datasource is very slow. When you have code on the metadata and data tabs TM1 pulls the full dataset from the source system twice, once for each tab. If you use "no" on this parameter, the process probably bypasses all the code on the metadata tab and therefore only cycles through the data once. The options I see for you are:

Speed up the performance of your datasource, e.g. by improving the SQL query, adding indices, stored procedures, etc.
Cache the datasource locally, i.e. copy the datasource to the TM1 server, probably as a flatfile.
Handle metadata updates separately, e.g. in a separate process that runs at a separate time or (not recommended) by manually adding elements.
Figure out why you have two processes overlapping and fix them, e.g. add more buffer time between the chores or instead use ExecuteProcess to sequence the processes.

Sounds like you're "black boxing" these processes a bit. Were they written by a 3rd party? You might need to call them back in to resolve.

Matt

mattgoff · Post by **mattgoff** » Tue May 19, 2015 1:55 pm

Also, your two threads on the same topic have now been merged together. Please do not open multiple threads on the same topic.

SBK88 · Post by **SBK88** » Thu May 21, 2015 9:50 am

Hello Friends,

Need some urgent help from you guyzzz.....

I ran a chore ( a scheduled one ), in TM1TOP I am not getting any updates, and even the log files showing no updates......
Its jus showing the status of last process that ran as " finished executing normally, elapsed time 444.23 seconds ".
No error , No updates, What to do.

Even I tried to kill this thread, but even that is not happenning. ( screenshot attached ).

Kindly help, its very urgent.

Thanks....

: TM1TOP_Screenshot; 1.png (15.55 KiB) Viewed 15070 times

mattgoff · Post by **mattgoff** » Thu May 21, 2015 3:25 pm

Please a) stop opening new threads for the same problem and b) read the request for assistance guidelines.

SBK88 · Post by **SBK88** » Mon May 25, 2015 3:49 am

Need your help guyzz,

I am facing the same issue as "http://www.tm1forum.com/viewtopic.php?f=3&t=8795".

In reply of this post a link is provided "https://www-304.ibm.com/support/entdocv ... wg1PM57320" posted by tm1novice; which I am not able to open.

Can somebody post the solution here in my post.

My TM1 process getting stuck and showing state=RUN and not allowing other processes to run.
Need your help guyzz.

Thanks

TM1 Forum

TM1 Data Load Failing everyday

TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday

One or the Other Process getting Stucked

Re: One or the Other Process getting Stucked

Re: One or the Other Process getting Stucked

Re: TM1 Data Load Failing everyday

Chore - Process got stuck

Re: TM1 Data Load Failing everyday

Re: TM1 Data Load Failing everyday