The search box gets the most deliberate click on the whole site. A shopper who types “leather jacket” knows what they want, and they judge the brand on what shows up in the next second.
For years that box ran on Algolia, as a SaaS. It worked. But the engine, its settings and its key all lived somewhere else, and every relevance tweak happened in a console our codebase knew nothing about.
This summer we built our own search on Meilisearch, hosted in our own cluster. Swapping the search engine on a fashion site without measuring is playing roulette with revenue, so we put both engines head to head in an A/B test across all nine storefronts.
The verdict after two months: a tie. Which is exactly what we were hoping for.
Why leave Algolia
Algolia is a very good product. The problem was the dependency:
- relevance did not live in our code. Ranking rules, synonyms and typo tolerance were tuned by hand in a UI, with no review and no history shared with the rest of the project;
- the cost tracks traffic, and our search traffic only goes up;
- search requests left the browser with a public key, straight to a third-party service;
- above all, we wanted to plug our own signals into ranking, which a closed engine makes awkward.
We had one advantage: our product records already exist as a flattened projection of the catalogue, the very one that feeds Algolia. A second engine could receive exactly the same documents, field for field. The comparison would be fair.
First, make the engine swappable
Before writing a single line of Meilisearch code, we spent a few days in June hiding Algolia behind an interface. The storefront no longer talks to Algolia; it talks to a SearchProvider that returns normalised products.
export const useSearchProvider = (): ((arm: SearchProviderId) => SearchProvider) => {
const algolia = useAlgoliaSearchProvider()
const meilisearch = useMeilisearchSearchProvider()
return arm => (arm === 'meilisearch' ? meilisearch : algolia)
}The detail that matters: this function picks an implementation, it never decides the arm. The caller decides, once, and keeps the same engine for an entire result list, first page and “load more” pages included. A “load more” that switched engines halfway would blend two rankings.
An engine that cannot replicate itself
The open-source edition of Meilisearch is a single-writer engine: it cannot form a cluster or replicate. For a site selling in nine markets, a single instance means a search outage every time a node restarts.
We did not try to bend the engine. We put high availability in the architecture instead:
- two fully independent instances, one per availability zone, each holding a complete copy of every index;
- the indexer does the replication: Magento writes every change twice, once per instance;
- the consumer fails over: the storefront queries a fixed primary and falls back to the other one when it fails.
Each instance is deliberately boring: a single-node Meilisearch with nothing around it. All the distributed behaviour lives in the indexer loop and in the proxy’s failover.
Three rules keep the whole thing robust:
- Fail-open per instance. If an instance does not answer, the indexer skips it, logs it and carries on with the other one. A dead instance never blocks its twin, nor the indexing queue it shares with Algolia: a search index running a bit late catches up, a frozen indexing queue stalls the entire catalogue.
- Pods are disposable. No persistent volume: state lives on local disk, and a sidecar ships a snapshot to S3 every six hours. A pod restarting on another node reloads the latest snapshot within seconds. The S3 key embeds the engine version: after an upgrade, the new version finds no compatible snapshot, boots empty and gets refilled by a reindex, instead of importing a file it cannot read.
- A full reindex every night, with no downtime: we build a temporary index, fill it, then swap it atomically with the live one. An instance that missed updates converges within 24 hours. And if the rebuild produces zero documents, the swap does not happen: we keep the live index rather than serve an empty catalogue.
The life of a query
The browser never talks to Meilisearch. The engine is only reachable inside the cluster, and a single storefront route knows it exists: a Nitro proxy that holds the search keys server-side.
The proxy does few things, but it does all of them:
- it validates the request with Zod: 200 characters at most, bounded pagination, a store taken from a list derived from our country configuration, so that no index name can be forged client-side;
- it builds the filter server-side. The client picks a mode from a closed enum (
serporarchive), never a filter expression: product visibility is not up for negotiation from the browser; - it queries a fixed primary, not a load balancer. The two instances are eventually consistent copies, a few seconds apart while indexing: a shopper bouncing between them would see their results shift from one page to the next;
- it fails over to instance B when A fails, with a circuit breaker (three consecutive failures, a thirty-second pause) and two time budgets, two seconds per attempt and seven in total. The total stays under the client’s eight-second timeout, so the server exhausts its failover before the browser gives up.
If both instances are down, the proxy returns a generic 502 and the client-side provider falls back to Algolia. That query then carries a fallback flag and the measurement drops it: an outage must never be able to skew the experiment.
Relevance, tuned in code
Meilisearch’s first win is not the engine itself: all of relevance becomes reviewed code. Index settings live in the Magento module, a base file plus one override per store, editable in the back office and pushed to the engine without a full reindex.
The ranking order is an assumed sandwich:
"rankingRules": [
"stock.is_in_stock:desc",
"words",
"typo",
"exactness",
"attributeRank",
"ordered_qty_store_30d:desc",
"wordPosition",
"proximity"
]Stock first: an unavailable product never ranks above one you can buy. Then textual relevance. Then, at equal relevance, sales over the last thirty days in that particular store: what sells in Madrid is not what sells in London.
Typo tolerance is switched off on the attributes where it does more harm than good (composition, categories, special labels). Algolia’s synonyms were migrated, then extended with inflections generated from the catalogue.
Which leaves the case that matters most in practice: the overly precise query. The proxy first requires every word, and only relaxes when nothing comes back:
let response = await search('all')
let degradation: SearchDegradation = 'strict'
if ((response.estimatedTotalHits ?? 0) === 0) {
response = await search('last', await resolveWordDropQuery())
degradation = 'word-drop'
}
if ((response.estimatedTotalHits ?? 0) === 0 && lastWord !== trimmedQuery) {
response = await search('last', lastWord)
degradation = 'last-word'
}Relaxation drops words from the end, which is where French puts the qualifier (“sac rock noir”). English does the opposite: in “burgundy bag”, the noun comes last. On the English-speaking stores concerned, we reverse the word order before relaxing, so the noun survives. That setting is per store, like everything else.
And everything that never needed an engine
The migration was also a chance to ask what really deserved a search engine. FAQ search, for instance, covered a few dozen answers. We replaced it with 80 lines of TypeScript: accent folding, Damerau-Levenshtein distance (swapping two adjacent letters counts as one typo, as in Algolia), the same tolerance thresholds. No network call, no index to maintain.
Archive search became a simple proxy mode, and the sales app used in our boutiques followed onto the same indexes.
Measuring without fooling ourselves
The search mode is set per store, in the Magento back office: algolia, meilisearch or ab. It is pushed live to the storefront with no deployment (we detail the mechanism in our article on per-market feature flags). We opened the test in waves: the United States on 22 July, Germany and Belgium on the 27th and 28th, five more markets on 3 August, France on 1 September.
In ab mode, assignment uses the _ga cookie identifier: even for Algolia, odd for Meilisearch, frozen for the whole visit. Two rules we learned along the way:
- server-side rendering never assigns. An arm picked during SSR would end up in the HTML cached by the CDN and be served to everyone. A brand-new visitor’s very first search, before the cookie exists, therefore goes to the default engine and is left out of the measurement;
- we label the engine actually displayed, not the one we thought we had picked.
The hardest part was reading the test. On European markets, only visitors who accept measurement cookies have a _ga, so only they get tested: the conclusion covers that population. A visitor can also show up in both arms (new device, cleared cookie): we count them in both, numerator and denominator alike, rather than attaching them to the arm where they searched the most, which would mechanically be the arm that fails the most.
And we had to lock the definitions. Our first reports, put together by hand week after week, did not strip fallback traffic and counted two-arm visitors asymmetrically. They showed Meilisearch ahead by more than 30% in Europe. Recomputed with frozen definitions, the lead for that same week was 16%.
The results
Data as of 27 September 2026, each market over its own window since launch:
| Europe excluding France (7 markets) | United States | |
|---|---|---|
| Window | 55 to 62 days | 67 days |
| Split of searchers | 50 / 50 | 50 / 50 |
| Buyers per arm | about 1,300 | about 2,000 |
| Conversion, relative gap Meilisearch vs Algolia | +1.9% (not significant) | −1.5% (not significant) |
| Add to cart | identical (11.0% on both sides) | identical within 0.1 point |
| Revenue per searcher, plausible gap (95% CI) | −8% to +14% | −11% to +8% |
No single market shows a significant conversion gap. France, the last to join, is read separately over 26 days: a gap in Algolia’s favour, not significant.
The most instructive chart is the trajectory:
In mid-August, Europe had Meilisearch winning by 16%, with a significant gap. On the same day, the United States had Algolia winning by 11%, significant too. Had we stopped the test that day, we would have drawn two opposite conclusions, each with a confidence interval to back it. Six weeks later, both curves had met around zero.
Relevance, slice by slice
The zero-result rate first told a lovely story: in Europe, 9.5% on Meilisearch against 18.2% on Algolia. Split by word count, it tells a different one.
| Queries (Europe, 17 to 27 Sept) | Algolia | Meilisearch |
|---|---|---|
| 1 word | 11.9% | 13.0% (significant gap) |
| 2 words | 20.9% | 2.4% |
| 3+ words | 34.2% | 3.1% |
On long queries, Meilisearch almost never returns zero results because it relaxes, whereas our European Algolia indexes did not. On single-word queries, the only slice where neither engine relaxes, Algolia fails slightly less often. In the United States, where both engines relax, Algolia does better on every slice. The overall rate was measuring a difference in relaxation strategy, not relevance. That is where our work lies: single-word queries.
Next up: Personalize
Since 29 September, the first results page on the Meilisearch arm is re-ranked by Amazon Personalize, and from the next day that re-ranking has happened server-side, before the page leaves: rendering waits at most 250 ms for it, after which the engine’s own order is served as is. On the same day, the category grid went into an A/B test between Personalize and Nosto, on the same assignment. From that date, the test therefore compares “Meilisearch + Personalize” with “Algolia + Nosto”, no longer two bare engines. The figures above deliberately stop on 27 September: they measure the engine and nothing else. We tell the rest of the story in our article on Personalize.
What we learned
- A tie is a win. The test had to prove we could take back control of search without hurting revenue. It rules out a commercial gap beyond roughly ten percent in either direction. What comes next is an architecture and ownership decision, no longer a bet.
- Never read an A/B test too early. Both significant leads from mid-August vanished. A run of non-significant results does not prove equality, but an early significant lead does not prove much either.
- Freeze definitions before publishing a number. Two defensible readings of the same spec produced gaps as large as the one we were trying to measure.
- High availability can live in the architecture. Two boring instances, an indexer that writes twice, a proxy that fails over: nothing exotic.
- Make it swappable before you swap it. The
SearchProviderinterface cost a few days in June. Without it, the A/B test would have been a fork of the storefront.
The search box looks like a detail. Changing it properly took an architecture, a measurement protocol and some humility in front of the numbers.