Ascii Quote Fun and Games

Post Reply
User avatar
jim wood
Site Admin
Posts: 3951
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: PA 2.0.7
Excel Version: Office 365
Location: 37 East 18th Street New York
Contact:

Ascii Quote Fun and Games

Post by jim wood »

Guys,

I'm trying to move awkward data between cubes and this includes free comment fields. The fields contain all kinds of crap so I can't use the usual delimiters as I'm exporting it to file first. I have therefore gone for the Ascii quote character set as Char(186) which then looks like this:
Quote.PNG
Quote.PNG (329 Bytes) Viewed 4022 times
The process exports the data to file as expected without issue. The same process then loads the data in to the other cube. The problem is that the process doesn't seem to like the quote character. I get the following errors:
QuoteError.PNG
QuoteError.PNG (3 KiB) Viewed 4022 times
I have made sure that the source ascii quote is set correctly yet the process still doesn't seem to like it.

Any idea why guys?

Jim.
Struggling through the quagmire of life to reach the other side of who knows where.
Shop at Amazon
Jimbo PC Builds on YouTube
OS: Mac OS 11 PA Version: 2.0.7
User avatar
jim wood
Site Admin
Posts: 3951
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: PA 2.0.7
Excel Version: Office 365
Location: 37 East 18th Street New York
Contact:

Re: Ascii Quote Fun and Games

Post by jim wood »

Looks like it may have something to do with the extended Ascii character set. Looking at something else and will post back.
Struggling through the quagmire of life to reach the other side of who knows where.
Shop at Amazon
Jimbo PC Builds on YouTube
OS: Mac OS 11 PA Version: 2.0.7
User avatar
jim wood
Site Admin
Posts: 3951
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: PA 2.0.7
Excel Version: Office 365
Location: 37 East 18th Street New York
Contact:

Re: Ascii Quote Fun and Games

Post by jim wood »

It was 100% because I was using an extended Ascii character 0186. Once I lowered it to character 94 everything worked as expected. It's interesting that the process itself recognized the extended character but the data tab part of the process did not,

Jim.
Struggling through the quagmire of life to reach the other side of who knows where.
Shop at Amazon
Jimbo PC Builds on YouTube
OS: Mac OS 11 PA Version: 2.0.7
Wim Gielis
MVP
Posts: 3105
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Ascii Quote Fun and Games

Post by Wim Gielis »

I have seen strange cases in the past with extended Ascii characters and try to stay away from there.
Which includes French accents like é, è, ê too. Not sure if this is still an issue nowadays, I am talking about some 5 years ago.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
User avatar
jim wood
Site Admin
Posts: 3951
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: PA 2.0.7
Excel Version: Office 365
Location: 37 East 18th Street New York
Contact:

Re: Ascii Quote Fun and Games

Post by jim wood »

Wim Gielis wrote: Tue Feb 05, 2019 4:39 pm I have seen strange cases in the past with extended Ascii characters and try to stay away from there.
Which includes French accents like é, è, ê too. Not sure if this is still an issue nowadays, I am talking about some 5 years ago.
We're still on 10.2.2 so it may have been fixed since.
Struggling through the quagmire of life to reach the other side of who knows where.
Shop at Amazon
Jimbo PC Builds on YouTube
OS: Mac OS 11 PA Version: 2.0.7
User avatar
paulsimon
MVP
Posts: 808
Joined: Sat Sep 03, 2011 11:10 pm
OLAP Product: TM1
Version: PA 2.0.5
Excel Version: 2016
Contact:

Re: Ascii Quote Fun and Games

Post by paulsimon »

Hi Jim

You may already be doing this, but in the past when I had to write different languages (including chinese) to files and read them back, I did the following :

In the Prolog get the char set

Code: Select all

vCharSet =  CellGetS( vInfoCube, 'Any', 'Character Set' ) ;
When writing to a file in the Data Tab you need something that will track the very first record being written :

Code: Select all

IF( vDataRecCount = 1 ) ;

  # Before outputting anything to the main file, 
  # we need to set the character set immediately before
  # the first output statement.

  SetOutputCharacterSet( vPathFile , vCharSet ) ;
  textoutput( vPathFile , 'Element' , 'Attr'  , 'OldVal' , 'NewVal' ) ;

ENDIF;

When reading from a file in the Prolog you need something like

Code: Select all

SetInputCharacterSet( vCharSet ) ;
Regards

Paul Simon
Andy Key
MVP
Posts: 351
Joined: Wed May 14, 2008 1:37 pm
OLAP Product: TM1
Version: 2.5 to PA
Excel Version: Lots
Location: Sydney
Contact:

Re: Ascii Quote Fun and Games

Post by Andy Key »

If you're only writing the data out to a file so you can read it back in again, won't SetOutputEscapeDoubleQuote cover it? No need to use anything other than the standard double quote, although you can still choose something else if you want, it will double up whatever character you have as the ASCIIQuoteCharacter.

You're still stuck with the joys of things like embedded CR/LF though...
Andy Key
Post Reply