Aurora vs MySQL

Amazon Aurora: The next generation of hosted database services put to the test

In this post we’re taking a closer look at Amazon Aurora to see how it fares against MySQL and if it delivers on the promise of a 5x performance boost. Recently reaching general availability, Aurora is Amazon’s home grown MySQL compatible database. Currently available on three AWS regions (Virginia, Oregon and Ireland) on top of RDS, Amazon’s Relational Database Service.

Aurora is basically the next stage in the evolution of Amazon’s hosted services. If now we’re used to using EC2 for our servers, and RDS was introduced for optimized database instances, Aurora is the next step in this offering. Fully compatible with any operation you’d run through MySQL so no code changes are needed.

To see a previous speed test we ran and discover how each of the regions compare to each other, you can check out this blog post right here.

Why Amazon is building a new MySQL compliant database?

Unlike hosting a database on RDS, where we have additional performance benefits over EC2, monitoring, and quick installation features, Aurora even has some newer tricks up its sleeve. Seems like a new database was required to make it easier for Amazon to further optimize performance and add features like backups and autoscaling database storage – Which is not available on the basic MySQL instances. In that sense, we’re promised that Aurora would act like MySQL if it was optimized to run on AWS from day 1.

A side effect of this makes Aurora available only on AWS. So for example, you’d not be able to use a local version for testing and would have to completely rely on AWS instances. On the bright side, migrating to Aurora from an existing database instance is just a matter of a few clicks.

Benchmark Results

To have more of a hands on feel of how Aurora stands up to MySQL performance-wise we’ve set up a simple benchmark covering a write-heavy scenario using sysbench. To do this, we’ve used a similar configuration from a previous Aurora performance assessment ran by Amazon. In this version of the benchmark, we ran 4 different instances: The latest MySQL 5.6 on EC2 and on RDS, Aurora on RDS, and the release candidate version of the upcoming MySQL 5.7 on EC2. The exact setup we used and the raw results are available on the next section of this post. Let’s jump straight ahead and have a look at the findings:

Operations Per Second

Operations Per Second

Comparing operations per second for MySQL and Aurora on EC2 and RDS

1. MySQL 5.6 was 8% faster than Aurora – The new database didn’t provide the best results on this test, and far off from the 5x promise we’re used to hear from Amazon. Perhaps the biggest differences kick in on different setups with bigger DB sizes, but this just comes to show us the importance of trying it out with your specific use case.

2. RDS provides up to a 60% performance benefit over EC2 – A more significant difference is seen when we compare performance of databases on EC2 vs RDS. Using RDS provided considerable improvement over deploying the same database on EC2.

3. MySQL 5.7 RC provided slightly better results than MySQL 5.6 – The next version of MySQL shows some progress, even though the RC benchmark run contained a considerable amount of ignored errors (full results are available through the link below).

Find the Crap in Your Java App

Show me how >>

Fred

4. When taking 95th percentile response times into account, Aurora takes the lead – Aurora was 20% better than MySQL 5.6 on RDS, and MySQL 5.7 was 20% better than MySQL 5.6 on EC2. RDS performed about 3x better than EC2.

** [Edit] Please note that optimizing MySQL’s my.cnf configuration would have probably improved its results. Thanks @morgo!

Response times

Response time

Comparing 95h percentile response times for MySQL and Aurora on EC2 and RDS

The Groundwork

The benchmark included setting up 5 r3.large instances on AWS: An EC2 machine which will run the benchmark, and machines with the databases themselves. MySQL 5.6 on EC2, MySQL 5.7 RC on EC2, MySQL 5.6 on RDS and Aurora which is only available on RDS. Then, we’ve created a test database with 250 tables and a 25,000 table size on each of the tested machines:

./sysbench –test=tests/db/oltp.lua –mysql-host= –mysql-port=3306 –mysql-user= –mysqlpassword= –mysql-db= –mysql-table-engine=innodb — oltp-table-size=25000 –oltp-tables-count=250 –db-driver=mysql prepare

Afterwards we’ve ran a variation of Amazon’s write-heavy benchmark described here (300 sec instead of 600 and 125 threads instead of 1000):

./sysbench –test=tests/db/oltp.lua –mysql-host= –oltp-tables-count=250 –mysql-user= –mysqlpassword= –mysql-port=3306 –db-driver=mysql –oltp-tablesize=25000 –mysql-db= –max-requests=0 –max-time=300 — oltp_simple_ranges=0 –oltp-distinct-ranges=0 –oltp-sum-ranges=0 –oltporder-ranges=0 –oltp-point-selects=0 –num-threads=125 –randtype=uniform run

So basically, the benchmark had 5 minutes to throw all it’s got at each of the database instances, and report back on the results. The raw sysbench output is available right here:

MySQL 5.6 on EC2
MySQL 5.7 RC on EC2
MySQL 5.6 on RDS
Aurora on RDS

Final Thoughts

As far as managed services go, Aurora is definitely a step in the right direction. Taking on hosted solutions for databases to the next level. But was it really necessary to create a new database to support autoscaling? We feel that RDS proved that it’s the right way to go as far as hosted MySQL goes and it’s a good time to experiment with Aurora if you’re containing it to specific cases for testing in production. Like with any new technology that gets so close to the data at the core of your business, you’ll need to proceed with caution. With that said, a main pain point we didn’t address here is pricing and this is where Aurora holds the greatest promise against traditional full blown commercial databases.

 

fredjava

15 tools to use when deploying new code to production – Become a better developer

 

Java 8

Java 8 exceptions have never been so beautiful – Try Takipi for Java 8

email
Yoda

Looking for more posts like this?

Join our force of more than 30,000 Java Jedi masters!

Watch a live demo
Yoda
Some kind of monster @ OverOps, GDG Haifa lead.
  • Jamie Cressey

    Really informative post, it would be nice to also see those results against Mariadb, which is also available within RDS now.

    • http://www.scrollstory.com Abhisek Padhi

      MariaDB is faster then typical RDS with MySQL 5.6 Implementation. Lots of blogs have benchmark tests.

  • Anurag@AWS

    I’d recommend testing with an r3.8xlarge. Vanilla MySQL is decent at smaller instance sizes, but struggles to take advantage of larger cores/ram/bandwidth. You can easily reproduce our numbers using our performance white paper.

  • evcharger

    Results depend heavily on application and also how you configure EBS (GP2, provisioned iops, plain magnetic disks etc). In addition, ,there is i2 instances with local SSD storage which can be very fast. Overall, on our deployment on aws, aurora came in dead last after extensive testing. So be sure to benchmark your use cases directly before jumping to any conclusions.

  • https://franciskim.co Francis Kim

    Great work! Good to see a perspective on performance other than Amazon’s.

  • https://www.alfrednutile.info/about Alfred Nutile

    thanks for the article!