Magento upgrades have a bad reputation, and they often deserve it. You read the release notes, review your overrides, test, ship, and the next day something breaks somewhere far away from anything you touched.
This summer we had to take Adobe Commerce from 2.4.7-p10 to 2.4.9. Version 2.4.9 requires PHP 8.5 and MySQL 8.4, and it replaces its cache engine. In other words, the whole stack was moving at once: the language, the database, the cache, the application server, and some of the third-party modules whose version Magento now dictates.
We made one simple rule: never change two layers at the same time. Between late June and early September, each piece went to production on its own, with its own rollback, except for the last step: 2.4.9 requires PHP 8.5 and Adyen 10, so they shipped together. 2.4.9 came last, on 7 September, on a stack that had already been renewed.
What had to move
| Layer | Before | After | In production |
|---|---|---|---|
| Database | MySQL 8.0 | MySQL 8.4 | 29 July |
| Cache and sessions | Redis OSS 7.0 | Valkey 8.1 | 3 August |
| Application cache | local files + Redis | Redis only | 27 July |
| Platform | Adobe Commerce 2.4.7-p10 | 2.4.9 | 7 September |
| Language | PHP 8.2 | PHP 8.5 | 7 September |
| Payments | Adyen module 9 | Adyen 10 | 7 September |
Payments were not in the original plan. Version 9 of the Adyen module does not boot with the logging library that 2.4.9 imposes, so upgrading Magento meant a major Adyen upgrade too. That is typical of this kind of project, and it is exactly why we wanted everything else out of the way first.
The order of operations
The idea is to start from the bottom: data first, then cache, then the application. Every step has to be shippable, observable for a few days, and reversible without touching the others.
Prune before you upgrade
The first step upgraded nothing at all: it deleted. A scheduled customer-group change module nobody used anymore, an old export feed, leftovers of a storefront theme that going headless had made useless. 3,452 fewer lines, in production on 21 July. Every deleted line is a line you don’t have to port, review or test.
One cache, in Redis
Magento offered a two-tier cache: a local file copy on every server, synchronised with Redis. On Kubernetes, with pods coming and going, that local tier mostly added complexity and possible inconsistencies between pods. On 27 July we removed it: the default cache now lives in Redis only. The configuration lost about thirty lines, and since 2.4.9 was waiting with a brand-new cache engine, we wanted to hand it the simplest possible configuration.
MySQL 8.4 with blue/green
For the database we used RDS blue/green deployments. AWS creates a copy of the production database on the new version, keeps it in sync through replication, and switches traffic over in one go when you decide.
Preparing took longer than switching. We had to rebuild a MySQL 8.4 parameter group from the values actually deployed, not the ones we believed were deployed. Move storage to gp3 before the switchover, so as not to mix two changes. Rehearse everything on the test environment from 30 June.
On 29 July, early in the morning, the switchover took about 70 seconds. A detail for anyone running infrastructure with Terraform: a blue/green switchover creates new instances with new internal identifiers, and our Terraform state still pointed to the old ones. We reconciled it by hand (removing then importing the primary and the replica) until the plan came back empty against production.
After a database switchover, we now check the state of Magento’s indexer views. They track change tables through a version number, and a database changing underneath them can leave them ahead of what is left to process: the index stops updating, without any error.
Valkey without the applications knowing
On 3 August, ElastiCache moved from Redis OSS 7.0 to Valkey 8.1 through a new replication group. The tricky part was elsewhere: our internal DNS names contained the engine version. We renamed them in two phases. First, old and new names pointed to the same target. Then the new groups took the final, engine-neutral names. No application ever had to know the engine’s name, or restart at the right moment.
The upgrade itself
On 19 August, 2.4.9 landed in preproduction. The upgrade commit touches 5,666 files: most of it is Magento core, which is deployed in the repository, but 205 files in our own modules had to be adapted.
The database migration was too heavy for a regular deployment. 2.4.9 converts tables to utf8mb4: 328 table rebuilds, for 78 GB of data. Impossible while the site is selling. We ran it ahead of time on a copy of the database, then switched traffic over to it in 66 seconds.
Acceptance testing in preproduction lasted two and a half weeks, until 4 September. It found ten defects. Most of them looked nothing like what we expected.
The application server: when state outlives the request
Our GraphQL API runs on Magento’s application server. It is the deepest change for a PHP developer, and it is worth a closer look.
With PHP-FPM, every request boots Magento, handles the request, and the process throws away all its memory. Slow, but safe: nothing survives from one request to the next. The application server keeps workers alive that chain requests. Booting Magento once saves a lot of time, but every shared object now has to know how to clean itself between requests. Magento provides an interface for that, ResetAfterRequestInterface, and its _resetState() method. In our own code, 28 classes implement it.
2.4.9 showed us both ways to break that contract.
Not resetting enough. During acceptance, add-to-cart failed now and then, with no reproducible pattern. The core customer model reset itself, except for two properties: the address cache and the address collection. A worker kept the previous customer’s state for the next request. The fix is two lines:
// Magento\Customer\Model\Customer::_resetState()
$this->_origData = null;
$this->storedData = [];
$this->_data = [];
+ $this->storedAddress = [];
+ $this->_addressesCollection = null;
Resetting too much. A third-party promotions module cleared, in _resetState(), a list of validators that the dependency injection container only hands it once, when the object is built. After the first request, the worker ran without validators. The fix was to remove that reset, through a Composer patch on the module.
The rule we took away: a property filled during a request must be cleared, a property injected at construction must never be. Neither mistake throws an exception. They only produce behaviour that changes from one request to the next, depending on which worker answers.
Silent failures
The most expensive defects to find have one thing in common: they don’t say anything.
An ignored cache declaration
The new 2.4.9 cache engine no longer recognised the way we declared Redis, by class name. No error: it fell back to a file cache, local to each server. During acceptance, the product order of a category was lost one time out of two, depending on which pod answered. The fix is one line of configuration:
'frontend' => [
'default' => [
- 'backend' => '\\Magento\\Framework\\Cache\\Backend\\Redis',
+ 'backend' => 'redis',
A Redis connection shared through fork
Our indexers fork processes to parallelise sending the catalogue to Elasticsearch. After pcntl_fork(), each child inherits the connections the parent opened, socket included.
Several processes talking over the same Redis socket mix up their exchanges: one process reads the reply meant for another. The result: documents lost from the index, with no visible error. We patched the framework’s connection provider to tie its pool to the process that opened it:
$currentPid = getmypid();
if ($this->connectionPoolPid !== null && $this->connectionPoolPid !== $currentPid) {
// keep the parent's sockets referenced: closing them here would close them for the parent too
$this->inheritedConnections[] = $this->connectionPool;
$this->connectionPool = [];
}
$this->connectionPoolPid = $currentPid;
The same patch suffixes persistent connection identifiers with the pid, so a child can never reattach to its parent’s persistent socket.
A cache that never lets go
One last example: the tag indexes of the new cache never expired. Redis memory crept up slowly, with no alert possible as long as it stayed under the threshold. An upstream fix added a time-to-live to those indexes; we applied it before going to production.
The tally: 26 defects
2.4.9 produced 26 defects in total. Fifteen were stopped before production: ten during acceptance testing, two in preproduction right after, and three API contract changes neutralised in advance, so that the site and our other applications would see no difference. Eleven reached production, and were fixed in the days and weeks that followed.
Sorting them, we found five mechanisms:
- The platform shifts under the code: cache, application server, indexing, database encoding. It is the leading source of defects, and no file in our code points to it.
- The core adds a behaviour that kicks in on our data, without a single line of our code changing.
- Third-party modules move with Magento and change their formats.
- Our copies and workarounds become wrong: a copied template that used a removed library, a workaround for a defect the core has since fixed.
- Silent failures delay detection: a fallback with no error, a row deleted without a log line.
The lesson is uncomfortable. The classic method, reviewing our overrides of the core, only covers the fourth family. Most breakages come from what moves around our code.
What we’ll do differently next time
We turned this tally into a procedure.
- Compare what the new version changes outside our code: for every core module we use, the plugin, observer and cron declarations between both versions, and both
composer.lockfiles. That would have revealed most of the 2.4.9 breakages. - Review our hooks into the core: our overrides, our 115 Composer patches, and every disable-by-name. A plugin disabled by a name that has since changed silently comes back.
- Measure the database migration on a copy, and run it ahead of time when it exceeds the deployment window.
- Check invariants on day 0, day 1 and day 3: rules that must always hold (stock, orders, indexing, back-office grids, cache, API status codes), compared with the previous days. Alerting on errors is not enough when there is no error.
PHP 8.5, to wrap up
On 30 September, the last step: Rector 2 with the PHP 8.5 rule set on our modules, and their Composer metadata aligned with the new version. A deliberately separate step: once the platform already runs on 8.5, modernising the code becomes a style change, not a risk.
What we learned
- One layer at a time. Six separate production releases, only one of which bundled several layers because 2.4.9 required it: six simple rollbacks. On 2.4.9 day, the underlying stack had been stable for a month.
- Delete before you upgrade. The cheapest code to migrate is the code that no longer exists.
- The application server changes the nature of PHP. State lives as long as a worker, not as long as a request. Every
_resetState()must clear what the request filled, and nothing else. - The most expensive defects are silent. A silent fallback costs more than an exception. We now look for invariants, not just errors.
- Migrate the database on the side. Blue/green for the MySQL version, a migration rehearsed on a copy for the schema: either way, about a minute of switchover.
The next upgrade will start with the same question: what moves around our code, and how will we notice if it breaks without saying a word?