A response to Percona’s 2026 MySQL ecosystem benchmark: useful data, but not a realistic MariaDB comparison

Percona’s new 2026 benchmark report is interesting because it puts several MySQL-family releases on the same graphs and shares a public repository for the test harness. That openness is welcome. But after reading both the article and the published scripts, I do not think the post supports broad conclusions about “ecosystem performance,” and I especially do not think it represents an adequately tuned MariaDB, worthy of a neutral comparison.

Edit

Since the first publication of this article on MariaDB.org’s blog, the discussion thread with Percona has been open. I also recommend reading the comments on the original post.
At the end, all this is to keep improving everything 😉

Versions

The versions chosen for MariaDB are already a subject of debate. MySQL 9.6.0 (the latest Innovation Release) and MariaDB 12.1.2 (a rolling release) were used.

It would have been fairer to use MariaDB 12.3.1, the last release candidate. It was public that 12.1.2 had some performance issues.

Variables

Another notable issue is that the environments were not really comparable either. MySQL, Percona Server, and MariaDB 11 were using 4G of InnoDB log files, while MariaDB 12 was using half that amount. Indeed, in recent MariaDB, only one file is used, and the value of innodb_log_files_in_group is just ignored.

There are also other questionable variables, but let’s skip them for now.

My goal

So my initial goal with this post was to verify whether, under comparable configuration settings, MariaDB was indeed worse.

This is a copy of the graph from Percona:

We can see that MariaDB performs better up to 256 threads, and that even when MariaDB 12.1.2 uses half the InnoDB log size, the results are similar to the previous version of MariaDB.

The Environment

I then decided to run benchmarks on my system using only MariaDB Server, this time with comparable settings.

My machine has 16 cores and 32GB of RAM, but I limited the Buffer Pool to 2 GB as in one of Percona’s benchmarks. I didn’t want to redo them all.

The machine

========================================
 Percona pt-summary System Info
========================================
Platform:            Linux
Release:             Fedora release 43 (Forty Three)
Kernel:              6.19.9-200.fc43.x86_64
Architecture:        CPU = 64-bit, OS = 64-bit
Processors:          physical = 1, cores = 8, virtual = 16, 
                                            hyperthreading = yes
Models:              16xIntel(R) Core(TM) i7-10700 CPU @ 2.90GHz
Memory Total:        31.0G
========================================

Here are the lines used in my.cnf:

innodb_log_files_in_group=2
innodb_buffer_pool_size=2G
innodb_buffer_pool_instances=1
max_connections=4096
max_prepared_stmt_count=1000000
innodb_io_capacity=2500
innodb_io_capacity_max=5000
table_open_cache=200000
table_open_cache_instances=64
character_set_server=utf8mb4
innodb_doublewrite=1
innodb_flush_log_at_trx_commit=1
innodb_flush_method=O_DIRECT
innodb_log_buffer_size=64M
sync_binlog=1
binlog_format=ROW
binlog_row_image=MINIMAL

This is equivalent to the values in the scripts Percona shared.

I made 3 changes on the different MariaDB versions I used:

For MariaDB 11.8.6:

innodb_log_file_size=2G
innodb_log_files_in_group=2

For MariaDB 12.1.2 and 12.3.1:

innodb_log_file_size=4G

As the value of innodb_log_files_in_groups is ignored, and we want to have the same value, we need to set innodb_log_file_size to 4G.

I also made an extra run of MariaDB 12.3.1, but enabling the new InnoDB-based Binlog:

binlog_storage_engine=innodb

The Test

I used sysbench’s provided read/write lua script (oltp_read_write.lua) and run for 1, 4, 16, 32, 64, 128, and 512 threads. The test executed as many transactions as it could for 2 minutes per thread.

The data (100 tables with 5k rows each) is as large as the Buffer Pool.

I kept the total transactions and graphed them:

We can see that indeed MariaDB 12.1.2 is the least performant. But the curve follows Percona’s. The peak is earlier.

The drop we see with MariaDB is due to a mutex contention that still needs to be fixed: MDEV-37924.

So let’s add MySQL 8.4.8 to the picture now:

We can see that MySQL is worse at the start, but performs better once 128 threads are reached. But before that, MariaDB is much better!

But once again, this is a bad assumption: MySQL 8.4 has binary logs and the Performance Schema enabled by default, which is not the case for MariaDB.

So, let’s set the record straight by enabling this in MariaDB:

log_bin="binlog"
performance_schema=on

And also for another run, let’s see how the new InnoDB-Based Binary Logs are performing too:

performance_schema=on
binlog_storage_engine=innodb

Here we see another story, with traditional binary log files, and with this configuration, MariaDB performs a bit less well. But I am happy to see that the InnoDB-Based Binlog yielded good results. But it also decreases, and it seems even worse than the traditional binary log when there are many concurrent threads, like 128.

Of course, this is something for us to analyze further.

Another nice point to be completely fair, is that almost up to 128 threads, MariaDB without binlogs or with InnoDB-Based binlogs performs better than MySQL 8.4.8 without binary logs:

As the settings, like the InnoDB capacity, were maybe not the best for the versions and did not match my hardware, I decided to test the innodb_dedicated_server variable and used the same generated settings for MariaDB.

This created a 23GB buffer pool and 8G of InnoDB Redo Log capacity.

I also want to notify you that, as innodb_flush_method is also deprecated since MariaDB 11.0, I used the new variables with values providing similar behavior:

• innodb_data_file_buffering=OFF
• innodb_data_file_write_through=OFF
• innodb_log_file_buffering=ON
• innodb_log_file_write_through=OFF

The config is now the following:

innodb_log_file_size=8G
innodb_buffer_pool_size=23G
innodb_buffer_pool_instances=1
max_connections=4096
max_prepared_stmt_count=1000000
table_open_cache=200000
table_open_cache_instances=64
character_set_server=utf8mb4
innodb_doublewrite=1
innodb_flush_log_at_trx_commit=1
#innodb_flush_method=O_DIRECT   -- obsolete
innodb_log_buffer_size=64M
sync_binlog=1
binlog_format=ROW
binlog_row_image=MINIMAL
binlog_storage_engine=innodb
innodb_data_file_buffering=OFF
innodb_data_file_write_through=OFF
innodb_log_file_buffering=ON
innodb_log_file_write_through=OFF

And the result is the following:

We can also observe that degradation is much lower at 128 threads when Binary Logs are not used, even when using InnoDB-Based ones:

We can see that the drop is less significant with a larger number of threads compared to the previous configuration.

All the tests were made with data that could almost fit entirely in the Buffer Pool, but when this is not the case, we can see that MariaDB seems to have more predictable results, which I didn’t observe in the Percona benchmarks:

Binaries

Why is MySQL behaving so poorly with the same settings and on the same hardware? I don’t know. Could it be related to the way it was compiled? But I downloaded it directly from the mysql.com website.

After having discussed with my colleagues, they suggested that I also publish the list of binaries I used. And as I was using the minimal MySQL package, I decided to rerun the test with the full package in case there was an issue with the one I used.

So here are the packages I used:

414M Mar 27 17:00 mariadb-11.8.6-linux-systemd-x86_64.tar.gz
414M Mar 27 17:02 mariadb-12.1.2-linux-systemd-x86_64.tar.gz
436M Mar 27 17:01 mariadb-12.3.1-linux-systemd-x86_64.tar.gz
 63M Mar 24 17:01 mysql-8.4.8-linux-glibc2.28-x86_64-minimal.tar.xz
878M Mar 31 09:04 mysql-8.4.8-linux-glibc2.28-x86_64.tar.xz

Perception

I would also like to draw attention to the interpretation presented in the data.

On my hardware, my test shows that performance drops at 128 threads for MariaDB. Which was not the case in Percona’s results:

And here, it’s not about the validity of these numbers as we saw that some settings were not accurate for a comparison, but more on how these numbers are presented to the reader:

Everything is in red, and if the user doesn’t really pay attention to the chart’s title—just as some people ignore warning messages—their perception might be skewed 😉

Observations & Thoughs

I appreciate Percona publishing the scripts and data, but the benchmark should be read much more narrowly than the article suggests.

The main point is that the benchmark is not very relevant (neither is mine) because the settings used don’t reflect real-world conditions.

Some variables are useless (ignored) in the latest version of MariaDB, as the warning in the error log clearly mentions it, and no replacements or equivalent changes have been made:

2026-03-28 16:13:21 0 [Warning] 'innodb-buffer-pool-instances' was
removed. It does nothing now and exists only for compatibility with old 
my.cnf files.

Another example is the IO capacity, which is 4 times lower than the default MySQL value. Way too low on new hardware, where even a USB 3.0 disk can provide 10k IOPS. Why? MySQL already set the default to 10000. Maybe something to change in MariaDB for the future…

It’s important that you don’t just accept every benchmark at face value; look closely, take the time to consider the details, and question everything. Then the best thing to do is to test it yourself with your own data and setup, and honestly share your results with others (experts or not).

So if we go back to Percona’s benchmark, we can see that, to be fair, some configuration values should have been adjusted to match the other setups, and maybe another, more recent version would have been more appropriate to compare against. I expected more rigor from experts like them.

I would also be curious to see how much overhead containerization adds in their tests.

Another point is that, depending on the hardware, version, and configuration, your benchmark can tell a completely different story, which is why, once again, you should be very careful about drawing conclusions. And it’s not an easy task to be totally fair when comparing different products.

Of course, benchmarking is very, very important, especially in the database field. It helps identify where issues and contention occur, and whether there are regressions or improvements. And that’s why we also developed and released the Test Automation Framework (TAF by Jeb), and it’s open to contributions for other and better Lua scripts too 😉

To also avoid situations where variables change and warnings are ignored, I proposed this feature: MDEV-39252. Let me know what you think.

Final Words

My conclusions are simple:

• the versions were badly picked
• basic InnoDB parameters were wrongly set
• I would have loved it if Percona had reached out to review or discuss those numbers before publishing (as I did for this blog)
• ⁠I think it’s a bit embarrassing for Percona not to set InnoDB properly (at least for MariaDB)
• MariaDB therefore performed much better in the tests I conducted.