TM1 Forum

Hi,

I'm loading a file with duplicate records. To load all records I need to identify each row with a unique identifier.

To solve this I can create a dimension for record count and create a new element for each line so that no record will be overwritten due to duplication.

I'm thinking to use the dimsiz function and create a loop to add new element and load the record for each new one.

Is this the right way to approach it?

Thanks!

Kenneth

Yes, you could foresee an initial set of lines, 10,000 for example. It depends on the size of the data set and on the dimensionality.
After that, in the Data tab of the process, you will write values to the next line. On line number 1 or 0, on a counter measure, you can increment the number of unique lines you have written to for that intersection of Year, Period, Company, Account, ... for example.
Then you add 1 and after formatting the result you will know the next line to write to.

Like,
L0000
L0001
L0002
...
L9999

and on L0000 you keep track to the linenumber through a measure called Counter. Use NumberToStringEx to format the numeric line number counter as L0000.

Do you need to keep track of each record individually? (In which case you need to introduce some kind of transactional dimension as per Wim's suggestion). Or is it possible to cummulate the duplicate cube address intersections? (In which case you could just use CellIncrementN not CellPutN)

lotsaram wrote: ↑Tue May 05, 2020 9:25 pm Do you need to keep track of each record individually? (In which case you need to introduce some kind of transactional dimension as per Wim's suggestion). Or is it possible to cummulate the duplicate cube address intersections? (In which case you could just use CellIncrementN not CellPutN)

Thanks. I forgot about this function! Will definitely try.

Kenneth

If you forgot about that function, there is still CellGetN to retrieve the value in the cube, then add the new value of the data source, then CellPutN of the sum of both to the cube (same intersection).

Correct. But CellIncrementN is definitely more convenient.

Thanks!

Wim Gielis wrote: ↑Wed May 06, 2020 1:00 pm If you forgot about that function, there is still CellGetN to retrieve the value in the cube, then add the new value of the data source, then CellPutN of the sum of both to the cube (same intersection).