Industry Insights

What Counts as a Sample, and Why Nobody Agrees

November 27, 2024

A group of friends talking over beers around a bar counter.

Two sampling programs can report the same number and mean different things by it. There's no shared definition of a sample in this category, and the absence makes every comparison between vendors approximate.

At least four definitions are in circulation

A pour. A drink was made and handed over. This is what most people picture, and it's what a brand ambassador counts at the end of a shift.

A registration. Someone signed up for a sample. They may or may not have collected it. Funnel-based programs sometimes report this number because it's the largest one available.

A redemption. A voucher, code or confirmation was used. This establishes that a transaction completed, though not necessarily that a drink was consumed by the person who registered.

A verified pour. A specific identified person received a specific drink, recorded at the point of sale. The narrowest definition and the most expensive to produce, because it requires the guest to do something.

Each is defensible. The gap between the largest and smallest on the same night can be substantial, and nothing obliges a vendor to tell you which one a proposal is quoting.

Why this isn't pedantry

Because cost per sample is the number brands compare, and the denominator is undefined.

A program quoting $4 a sample against registrations and a program quoting $9 a sample against verified pours may be the same program. Comparing them on price alone produces the wrong answer, and the cheaper-looking one is frequently the one measuring the loosest thing.

It also affects what you can claim afterward. A count contains no people — fifty poured can't tell you whether fifty people tried the product or twelve people tried it four times. Any claim about trial, repeat or conversion requires that someone was identified at the moment of the pour, which only the narrowest definition supports.

The wider measurement problem this sits inside

This isn't unique to sampling. Event Marketer's 2019 measurement benchmarking report, drawing on more than 250 responses from event marketing executives and agencies, states plainly that there's still no standard event measurement approach and that most marketers combine several methods.

The same survey found 61% of consumer brands measuring with on-site staff using tablets and check-in software — which is to say the measurement chain in a large share of experiential programs still ends with a person recording something during an event.

What to ask

One question, in writing, before any proposal is comparable.

What exactly are you counting, and what has to happen for one to count?

Then ask what the historical gap is between registrations and completed pours in their own programs. A vendor who has run this at scale knows that number. A vendor who hasn't will tell you it rarely comes up.

Our own number has soft edges and we'll point at them. Ask what we count and you'll get the definition rather than the headline.

Book a demo
For brands