Evidence Education

Email deliverability claims: what do the percentages actually prove?

A vendor says 97%. Another test says 91%. A third platform promotes a different benchmark entirely. Those numbers may look comparable even when the tests behind them are not. Before declaring a deliverability winner, inspect what was measured, how it was measured, and whether the comparison is genuinely matched.

Guide type Evidence methodology
Core issue Unmatched percentages
Ranking policy No universal winner inferred
Commercial links None on this guide
Start with the vocabulary

Delivery, deliverability, and inbox placement are not interchangeable.

A percentage becomes meaningful only after the outcome being counted is clearly defined.

Delivery

Was the message accepted?

Delivery generally concerns whether a message avoided an immediate failure such as a bounce. Acceptance by the receiving system does not necessarily tell you where the message appeared afterward.

Inbox placement

Where did the message land?

Inbox-placement testing attempts to distinguish messages that appear in an inbox from messages routed elsewhere, such as a spam or junk area.

Engagement

What happened after delivery?

Opens, clicks, replies, conversions, and other recipient actions are separate outcomes. They can be influenced by far more than sending infrastructure alone.

A “98% delivery rate” is not the same claim as “98% inbox placement.” Before comparing numbers, verify that the numerator, denominator, and measured outcome mean the same thing.
The comparison problem

Two accurate percentages can still create an invalid comparison.

If the underlying tests differ, ranking the products from the final percentages alone can create false precision.

Different audiences One test may use highly engaged recipients while another uses colder or less active addresses.
Different sending domains Domain history and reputation may differ before the platforms are ever compared.
Different authentication Configuration differences can contaminate what appears to be a platform comparison.
Different message content A plain operational message and a promotional campaign are not equivalent test inputs.
Different sending volume Small tests and large production sends may create different operating conditions.
Different mailbox mix A test weighted toward one mailbox provider may not resemble another test's recipient distribution.
Different time periods A result from one period should not automatically be treated as equivalent to a result from another.
Different definitions “Delivered,” “inbox,” “success,” and “placement” may not use the same measurement method.
The mistake is not using percentages. The mistake is treating percentages produced under different conditions as though they came from one controlled experiment.
A stronger comparison

Better evidence holds important variables as constant as practical.

No real-world email test is perfectly controlled, but comparison quality improves when competing products are evaluated under similar conditions.

Same type of audience Compare similar recipient populations rather than unrelated lists.
Same message Keep subject, body, links, and formatting as similar as the methodology allows.
Same timing Run comparable sends close enough together that major environmental changes are reduced.
Same authentication standard Avoid comparing a carefully configured sender with an incompletely configured one.
Same mailbox mix Use comparable destination providers so the measured populations resemble each other.
Same measurement method Define delivery, inbox, spam, missing, and other outcomes consistently.
Even matched testing answers a narrower question than “Which ESP is best?” It tells you how those systems performed under the particular tested conditions. Generalization beyond those conditions still requires care.
Example

Why 96% versus 92% may not mean what it appears to mean.

Hypothetical comparison

Platform A reports 96%. Platform B reports 92%.

If Platform A's number came from a recent vendor-controlled test using one recipient mix, while Platform B's number came from a third-party study using another list, different content, a different time period, and a different definition of success, subtracting 92 from 96 does not produce a defensible four-point platform advantage.

Both percentages may accurately describe their own tests. The comparison becomes weak when we pretend the tests were equivalent.

What a deliverability claim can prove

A percentage can be useful evidence when its boundaries are preserved.

It can describe a specific test. A documented percentage can tell us what happened under the stated conditions.
It can expose a serious problem. Very poor outcomes in a well-designed test can justify additional investigation.
It can support trend analysis. Repeated measurements using a stable methodology can reveal changes over time.
It can compare matched conditions. The more variables held constant, the stronger the relative evidence can become.
It can validate instrumentation. Small controlled tests can confirm that sending, delivery-related reporting, and tracking are functioning.
It can generate questions. A result can identify areas worth investigating without being treated as a final universal verdict.
What it does not prove

One percentage should not carry more weight than the methodology can support.

It does not prove permanent superiority. Infrastructure, filtering systems, sending populations, and conditions can change.
It does not guarantee your result. Your audience quality, domain history, message mix, volume, and behavior may differ from the test.
It does not replace authentication. Choosing a platform with a strong benchmark does not remove the need for correct sender configuration.
It does not replace list quality. A software provider cannot turn a poor-quality audience into a healthy permissioned sending program.
It does not prove conversion performance. Inbox placement and business outcomes are related questions, not identical measurements.
It does not prove overall product fit. Pricing, automation, policy, usability, integrations, migration, and support still matter.
What affects real-world results

The ESP is only one part of the sending system.

Deliverability should be evaluated as an interaction between platform infrastructure and the sender's own practices and audience.

Sender identity The domain and identity associated with the sending program matter to the overall system.
Authentication Correct domain and sender configuration forms part of a responsible production setup.
Audience quality Permission, address quality, and subscriber expectations influence the environment in which messages are sent.
Engagement patterns Recipient behavior can provide signals about whether communication is wanted and useful.
Content Message structure, subject matter, links, and promotional characteristics can differ across campaigns.
Sending pattern Frequency, volume, sudden changes, and consistency can make one sending program unlike another.
Mailbox provider Different receiving systems may classify the same sending program differently.
Provider infrastructure The ESP's infrastructure and operational systems remain an important part of the equation.
Time A result is a measurement from a period, not a guarantee that conditions will remain unchanged indefinitely.
Controlled tests

Small tests are useful — but only for the questions they can answer.

StackEvidence uses controlled accounts and test contacts to verify product behavior where practical. That evidence has real value, but it should not be stretched into claims the test was never designed to prove.

Can we send? A controlled test can verify that the account and message flow function.
Was the message accepted? Delivery-related reporting can help verify the mechanics of the test.
Did links work? Controlled clicks can verify link behavior and some tracking functions.
Did automation respond? Test contacts can validate branches, waits, tags, and follow-up actions.
Was mobile rendering usable? Preview and controlled inbox checks can verify the tested message layout.
Is this a deliverability benchmark? No. A handful of controlled messages does not establish production-scale inbox performance.
StackEvidence evidence boundary: our controlled email-platform tests verify specific workflows and observed results. We do not convert a few test sends into a claim that one platform has universally better deliverability.
Evaluate a deliverability claim

Use five questions before allowing the number to influence a buying decision.

1
Define What exactly does the percentage measure?
2
Source Who ran, funded, or published the test?
3
Method How were audience, message, timing, and placement measured?
4
Match Are competing numbers actually comparable?
5
Scope How far beyond this test can the conclusion reasonably extend?
Should deliverability decide the winner?

Deliverability matters, but weak comparative evidence should not dominate the entire product decision.

If a platform cannot support responsible production sending, that is obviously important. But once products meet the basic requirements, the final choice still needs to account for the wider operating system.

Policy fit Can the business legally and contractually use the service for its intended content and workflow?
Workflow fit Can the product support the required campaigns, segmentation, and automation?
Economics What does the usable plan cost as contacts, sends, and requirements grow?
Integrations Can the ESP exchange data with the rest of the business stack?
Portability What can be exported, and what must be rebuilt if the business leaves?
Evidence quality Which conclusions are directly tested, documented, vendor-reported, or still uncertain?
Better research language

Strong conclusions describe the evidence without pretending uncertainty disappeared.

Too strong

“Platform A has the best deliverability.”

This implies a broad winner even when the evidence may come from one unmatched test or vendor-controlled statistic.

Better

“Platform A scored higher in this specific test.”

This keeps the conclusion attached to the evidence that actually produced it.

Strongest

Describe method, result, and limitation.

State what was tested, what happened, and why the result should or should not be generalized beyond that experiment.

What buyers should do

Use deliverability evidence as one input, then test your own setup responsibly.

1
Shortlist Choose products that satisfy the actual business requirements.
2
Configure Set up sender identity and required authentication correctly.
3
Test Verify messages, links, workflows, and controlled inbox behavior.
4
Observe Monitor the real production program rather than relying only on marketing benchmarks.
5
Improve Correct list, configuration, content, or workflow problems as evidence appears.
Do not choose an ESP solely from a leaderboard. A benchmark can inform the decision, but the system still has to work for your audience, content, policy requirements, budget, integrations, and operating process.
StackEvidence standard

Deliverability claims receive the same evidence treatment as pricing and policy claims.

We separate what was directly observed, what a vendor reports, what a third party tested, and what can reasonably be concluded from that evidence. We do not create a universal platform ranking from unmatched percentages.

Define the metric Delivery, inbox placement, engagement, and conversion are different outcomes.
Identify the source Vendor-controlled and independent evidence can both be useful, but provenance matters.
Inspect the method Audience, content, authentication, timing, mailbox mix, and sample size affect interpretation.
Match before ranking Percentages from different methodologies should not be treated as one shared experiment.
Preserve limitations Controlled testing can verify observed behavior without promising future production results.
Commercial transparency

Affiliate economics should never turn weak evidence into a deliverability winner.

This page contains no affiliate links. If StackEvidence later has commercial relationships with email software providers, commission differences will not change the evidentiary standard used for deliverability claims or comparisons.

Our position: when the available evidence cannot support a matched ranking, the correct conclusion is that a universal deliverability winner has not been established.