The AWS Cloud no good, very bad performance
- PavoGa
- MVP
- Posts: 616
- Joined: Thu Apr 18, 2013 6:59 pm
- OLAP Product: TM1
- Version: 10.2.2 FP7, PA2.0.9.1
- Excel Version: 2013 PAW
- Location: Charleston, Tennessee
The AWS Cloud no good, very bad performance
We have just moved our development from on-prem servers to the AWS Cloud. The performance is, to be kind, lacking. The processes that move data from one slice of a cube to another are taking roughly double or more the amount of time to perform the same tasks. Running TIs side-by-side for the two environments and the Cloud is like a three legged mule racing a thoroughbred.
Architect (I use ARC, but we had some holdouts) is virtually unusable, as I think has been noted in this Forum. ARC seems to be fine. PAW seems to be fine and the installation went well. The AWS server has the same memory and cores as the on-prem. We use AWS as our development workstations.
If anyone else is using AWS cloud, were performance issues observed? Any particular setting(s) to check?
Architect (I use ARC, but we had some holdouts) is virtually unusable, as I think has been noted in this Forum. ARC seems to be fine. PAW seems to be fine and the installation went well. The AWS server has the same memory and cores as the on-prem. We use AWS as our development workstations.
If anyone else is using AWS cloud, were performance issues observed? Any particular setting(s) to check?
Ty
Cleveland, TN
Cleveland, TN
-
- MVP
- Posts: 2831
- Joined: Tue Feb 16, 2010 2:39 pm
- OLAP Product: TM1, Palo
- Version: Beginning of time thru 10.2
- Excel Version: 2003-2007-2010-2013
- Location: Atlanta, GA
- Contact:
Re: The AWS Cloud no good, very bad performance
If the type and number of cores is the same I would suspect hard drive issues, i.e., SAN. Are you sure your drive space is in the same region as your server? Even in cube to cube copies TM1 is going to utilize drive space in the memory temp file.
As to Architect, at least from an Admin standpoint, can't you just use RDP to the TM1 server and run it from there? I have used AWS for other systems and RDP is how you get access to the server to do stuff (assuming you are on Windows). If this is Linux don't they have something similar.
As to Architect, at least from an Admin standpoint, can't you just use RDP to the TM1 server and run it from there? I have used AWS for other systems and RDP is how you get access to the server to do stuff (assuming you are on Windows). If this is Linux don't they have something similar.
- PavoGa
- MVP
- Posts: 616
- Joined: Thu Apr 18, 2013 6:59 pm
- OLAP Product: TM1
- Version: 10.2.2 FP7, PA2.0.9.1
- Excel Version: 2013 PAW
- Location: Charleston, Tennessee
Re: The AWS Cloud no good, very bad performance
Thanks, Tom. I'll follow up on this with the server team. I'm new to a cloud installation, literally every install I've ever worked on was on-prem.tomok wrote: ↑Mon May 09, 2022 4:07 pm If the type and number of cores is the same I would suspect hard drive issues, i.e., SAN. Are you sure your drive space is in the same region as your server? Even in cube to cube copies TM1 is going to utilize drive space in the memory temp file.
As to Architect, at least from an Admin standpoint, can't you just use RDP to the TM1 server and run it from there? I have used AWS for other systems and RDP is how you get access to the server to do stuff (assuming you are on Windows). If this is Linux don't they have something similar.
With regards to Architect, logging into the server with RDP is being taken away. We had it with the on-prem.
Ty
Cleveland, TN
Cleveland, TN
- gtonkin
- MVP
- Posts: 1192
- Joined: Thu May 06, 2010 3:03 pm
- OLAP Product: TM1
- Version: Latest and greatest
- Excel Version: Office 365 64-bit
- Location: JHB, South Africa
- Contact:
Re: The AWS Cloud no good, very bad performance
Got a client looking to go this way too, so jumping in to keep an eye.
The one thing I always look to is how your connection is made. From your corporate network vs home (Fibre/4/5G)
Have found that corporate IT often throttle traffic they do not know and limp it into lead class, quality of service wise.
May be an idea to do some more tests from different networks. Another thing to keep in mind is different technologies utilise different MTU settings. Users on 4G often report better Perspectives performance than fibre peers.
The one thing I always look to is how your connection is made. From your corporate network vs home (Fibre/4/5G)
Have found that corporate IT often throttle traffic they do not know and limp it into lead class, quality of service wise.
May be an idea to do some more tests from different networks. Another thing to keep in mind is different technologies utilise different MTU settings. Users on 4G often report better Perspectives performance than fibre peers.
- vovanenok
- Posts: 88
- Joined: Mon Jun 23, 2014 4:54 pm
- OLAP Product: TM1
- Version: 2.0.9
- Excel Version: Office 365
- Location: Toronto, Canada
- Contact:
Re: The AWS Cloud no good, very bad performance
I use AWS and even with small instances (like t3.medium) had no issues. Though I've never compared vs on prem. I use RDP to access Architect and other stuff on the server.
Assuming you have the same or similar enough server specs, I would look into running some stress test software to compare your AWS vs on prem, which would tell you about the disk performance as well
I remember I used Prime95 for Windows a while ago when we switched from physical servers to VMs and I wanted to compare both.
Assuming you have the same or similar enough server specs, I would look into running some stress test software to compare your AWS vs on prem, which would tell you about the disk performance as well
I remember I used Prime95 for Windows a while ago when we switched from physical servers to VMs and I wanted to compare both.
-
- MVP
- Posts: 3651
- Joined: Fri Mar 13, 2009 11:14 am
- OLAP Product: TableManager1
- Version: PA 2.0.x
- Excel Version: Office 365
- Location: Switzerland
Re: The AWS Cloud no good, very bad performance
What type of AWS instance are you using?
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
- PavoGa
- MVP
- Posts: 616
- Joined: Thu Apr 18, 2013 6:59 pm
- OLAP Product: TM1
- Version: 10.2.2 FP7, PA2.0.9.1
- Excel Version: 2013 PAW
- Location: Charleston, Tennessee
Re: The AWS Cloud no good, very bad performance
I am not conversant on AWS instance types, but does x1e.2xlarge answer your question?
Ty
Cleveland, TN
Cleveland, TN
- PavoGa
- MVP
- Posts: 616
- Joined: Thu Apr 18, 2013 6:59 pm
- OLAP Product: TM1
- Version: 10.2.2 FP7, PA2.0.9.1
- Excel Version: 2013 PAW
- Location: Charleston, Tennessee
Re: The AWS Cloud no good, very bad performance
The DEV connections are mostly made via corporate network in DEV. Prod is still on-prem and a lot of those users will be via home. I use an AWS instance from home, so I'm inside the corporate network.gtonkin wrote: ↑Mon May 09, 2022 5:46 pm Got a client looking to go this way too, so jumping in to keep an eye.
The one thing I always look to is how your connection is made. From your corporate network vs home (Fibre/4/5G)
Have found that corporate IT often throttle traffic they do not know and limp it into lead class, quality of service wise.
May be an idea to do some more tests from different networks. Another thing to keep in mind is different technologies utilise different MTU settings. Users on 4G often report better Perspectives performance than fibre peers.
I tested copying a 100MB file from an on-prem server using psh. To another on-prem server it took just over one minute (1:00.685) and copying the same file to the AWS server took over three (3:35.851). Once this file was loaded onto the servers, I ran a TI that did nothing more than read and count the lines. There was no significant difference On-Prem vs. AWS.
I then ran a TI reading approximately 23,750,000 numeric cells from a cube. Did nothing more than count the cells read. Cube was not populated identically in the two instances. Ran the process three times in each instance. Here there was a difference.
The AWS read 23,744,470 cells averaging 101.88 seconds and 235,567 records per second. Run times: 94.08, 117.28, 94.28.
The on-Prem read 23,796,470 cells averaging 70.74 seconds and 336,447 records per second. Run times: 71.67, 70.58, 69.96. Significantly faster.
Was actually surprised that reading a text file showed no significant difference.
Ty
Cleveland, TN
Cleveland, TN
-
- MVP
- Posts: 3651
- Joined: Fri Mar 13, 2009 11:14 am
- OLAP Product: TableManager1
- Version: PA 2.0.x
- Excel Version: Office 365
- Location: Switzerland
Re: The AWS Cloud no good, very bad performance
Yes that's what I meant.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
- PavoGa
- MVP
- Posts: 616
- Joined: Thu Apr 18, 2013 6:59 pm
- OLAP Product: TM1
- Version: 10.2.2 FP7, PA2.0.9.1
- Excel Version: 2013 PAW
- Location: Charleston, Tennessee
Re: The AWS Cloud no good, very bad performance
Further update:
Running a TI with a simple loop (1-15,000,000) takes about 45% longer, on average, on the AWS than the on-prem server.
Running a TI with a simple loop (1-15,000,000) takes about 45% longer, on average, on the AWS than the on-prem server.
Ty
Cleveland, TN
Cleveland, TN
- Steve Rowe
- Site Admin
- Posts: 2410
- Joined: Wed May 14, 2008 4:25 pm
- OLAP Product: TM1
- Version: TM1 v6,v7,v8,v9,v10,v11+PAW
- Excel Version: Nearly all of them
Re: The AWS Cloud no good, very bad performance
The AWS server must have slower cores in it then or being throttled in some way?
Technical Director
www.infocat.co.uk
www.infocat.co.uk
- PavoGa
- MVP
- Posts: 616
- Joined: Thu Apr 18, 2013 6:59 pm
- OLAP Product: TM1
- Version: 10.2.2 FP7, PA2.0.9.1
- Excel Version: 2013 PAW
- Location: Charleston, Tennessee
Re: The AWS Cloud no good, very bad performance
I'm not on the team working on the infrastructure, but I have asked that question as I think George mentioned it earlier. I'm told they do not believe there is any throttling occurring.Steve Rowe wrote: ↑Thu May 12, 2022 8:21 am The AWS server must have slower cores in it then or being throttled in some way?
One other thing: is the "not running virus protection on the DATA folder" still a thing? That is being checked on, but one of the team asked if it was and frankly, I have not seen anything on that recently that was yay or nay.
Ty
Cleveland, TN
Cleveland, TN
- PavoGa
- MVP
- Posts: 616
- Joined: Thu Apr 18, 2013 6:59 pm
- OLAP Product: TM1
- Version: 10.2.2 FP7, PA2.0.9.1
- Excel Version: 2013 PAW
- Location: Charleston, Tennessee
Re: The AWS Cloud no good, very bad performance
An update: pay attention to the processor speed. While the RAM and cores were equivalent, switching to another AWS with faster processors resolved the issues. Faster now than the on-prem server was. I was not directly involved with the setup, but did devise several tests measuring performance.
Thanks, George & Steve, for providing the ultimate hint. Thanks to everyone for their comments. Some did not pan out, but helped eliminate other potential issues. Your suggestions were passed along to the server team.
Thanks, George & Steve, for providing the ultimate hint. Thanks to everyone for their comments. Some did not pan out, but helped eliminate other potential issues. Your suggestions were passed along to the server team.
Ty
Cleveland, TN
Cleveland, TN
- gtonkin
- MVP
- Posts: 1192
- Joined: Thu May 06, 2010 3:03 pm
- OLAP Product: TM1
- Version: Latest and greatest
- Excel Version: Office 365 64-bit
- Location: JHB, South Africa
- Contact:
Re: The AWS Cloud no good, very bad performance
Glad to hear you got it sorted and we all have learnt something through your school fees.