Large number of Decimals - unexpected

Post Reply
chewza
Regular Participant
Posts: 154
Joined: Tue Aug 17, 2010 11:51 am
OLAP Product: TM1
Version: 9.5
Excel Version: 7

Large number of Decimals - unexpected

Post by chewza »

Hi there

I am importing data from a SQL table into a cube, and using CellIncrementN. For some reason I am getting a really small number with a huge number of decimals e.g. 0.000000000000023

I have tested by running the source query in SQL and pasting to Excel. The 16 rows that are returned all display with 2 decimals, and aggregate to 0 (as they should). I have also used RoundP(Value,2) in my Call Increment. The problem still remains.

Any ideas?

Many thx!
User avatar
gtonkin
MVP
Posts: 1214
Joined: Thu May 06, 2010 3:03 pm
OLAP Product: TM1
Version: Latest and greatest
Excel Version: Office 365 64-bit
Location: JHB, South Africa
Contact:

Re: Large number of Decimals - unexpected

Post by gtonkin »

Will point you to one of Steve's posts which will point you to others.
Basically down to floating point imprecision.
An IBM post
chewza
Regular Participant
Posts: 154
Joined: Tue Aug 17, 2010 11:51 am
OLAP Product: TM1
Version: 9.5
Excel Version: 7

Re: Large number of Decimals - unexpected

Post by chewza »

Thanks George!

I have a rule which divides by the value. This then becomes a real issue as you get massive calculated numbers.
Worked my way around this in a rule.
Post Reply