Data Precision

[hbell]

Can anyone tell me whether there is a maximum number of digits (before or after the decimal point) that TM1 can hold in a cell?


thanks ........hugh

[David Usherwood]

http://www.tm1forum.com/viewtopic.php?p=13746 and similar postings should help.

[hbell]

David

... thanks for that. I had searched the database (and FAQs) but found nothing precise enough. Unless I misread, the one you pointed me to concedes that he has "no idea" where the decimal limit is. He is also talking only about the decimal limit rather than the whole number.

I can follow up with IBM, but was hoping for a quicker route here.

thanks anyway ............hugh

[David Usherwood]

Hugh, TM1 uses standard 32-bit floating point for numbers. The Wikipedia entries for floating point
http://en.wikipedia.org/wiki/IEEE_754-2008
should give you what you need.

[hbell]

... the answer, per IBM, is 15 digits. The reply has now been published as a Technote on the IBM website. So if you have 6 digits to the left of the decimal point, you can only have 9 on the right ....etc etc.

[qml]

For those interested, the technote is available here:
https://www-304.ibm.com/support/docview ... wg21515228

The technote goes:

TM1 Precision

Technote (FAQ)

Question
What is the decimal precision of TM1?

Answer
TM1 stores data in IEEE-754 Floating Point Double.
The maximum number of digits is 15+ (almost 16, but some 16 digit number will not fit with full precision).
TM1 will clip to 14 digits for display.
By default Architect will display 9 digits of precision after the decimal point. This can be modified by by defining a custom format for the cell. Of course, the same overall 15 digit limit still applies. If one has 10 digits to the left of the decimal point, there will be only 5 digits to the right.

[tomok]

... the answer, per IBM, is 15 digits. The reply has now been published as a Technote on the IBM website. So if you have 6 digits to the left of the decimal point, you can only have 9 on the right ....etc etc.

I would caution anyone who is expecting TM1 to maintain this level of precision. I have seen numerous instances when someone enters 5 in a cube and TM1 stores it as 4.99999999, or something to that effect. It's never been a big deal to me because I've never cared about anything beyond three of four decimal places but I see this all the time on both ENTERED values and CALCULATED values. I've never seen any rhyme or reason behind it.

[Steve Rowe]

Hugh, TM1 uses standard 32-bit floating point for numbers. The Wikipedia entries for floating point
http://en.wikipedia.org/wiki/IEEE_754-2008
should give you what you need.

As per David's comment, the weirdness you talk about Tomok is to do with the fact for some numbers it's not possible to store them with 100% accuracy in base 2 (32-bit floating point base 2 at least). If you input one of these numbers it gets approximated in base 2 and then when you read it back in base 10 it is not exactly the same as the number you entered.

This isn't a problem unique to TM1 though.

Cheers,

Steve

[John Hammond]

Will actually be 64bit, Double Precision, Floating Point for this level of numeric accuracy (see the Wiki Entry). I am nitpicking a bit since the principle of inexactness applies to all FP arithmetic which is the main point of the poster.

FP Makes for quick calculations but surprisingly there is no provision for exact numbers within TM1 which is interesting seeing as it is primarily used in accounting although 15.9 digits = x,xxx,xxx,xxx,xxx.xx ie 10 Trillion which is even enough to represent the Uk national debt after Gordon Brown has finished with it...

[garry cook]

Yay, I guessed right at 15

And indeed, although that's a high degree of granularity, some calculated allocation splits can fall foul of this when a very high dp calc'd %age is applied to a very large base number (as I indicated in the post linked to above) so it can be an issue occasionally.

Still, multiplying up and then dividing down gets round it so wasn't a huge issue.

And PS - Alistair "I'll read a book on economics for beginners" Darling was probably just as bad for "our" national debt (I'll avoid my Scottish rant on SNP/Labour/Tories given the context).