Measuring whether an offer simply discounts an inevitable purchase

A redeemed order cannot reveal whether that customer would have bought at full price. For a group, estimate the issue: assign comparable eligible customers to a targeted offer or the usual treatment, then compare all purchases and contribution over the same period. The usual-treatment group estimates purchasing that would occur without that targeted offer, subject to uncertainty.

Define what the offer changes

Set eligibility using information available before assignment: relevant prior purchase, product group, a plausible next buying opportunity and contact permission if a message is involved. Randomly assign at customer level and retain the assignment. Keep public prices and other marketing as similar as practical, and record offers that either group can obtain elsewhere.

The control receives what customers would ordinarily receive, which need not be silence. If both groups get a standard reminder and only one gets a targeted discount, their difference estimates the added effect of the discount with that reminder. If only the offer group receives both a new message and a discount, the comparison measures the package and cannot isolate the price benefit.

Define qualifying completed orders, the start date and the follow-up cut-off before sending. Include relevant orders made without the code and through sales channels covered by the test. A redemption report alone omits purchases made by offer recipients without redemption. Comparing redeemers with non-redeemers also selects people by a choice made after assignment.

Compare contribution per assigned customer

For each group, report assigned customers, completed purchasers, orders, quantities, sales after discounts and refunds, and contribution, using one tax and cost basis. Subtract product cost, relevant fulfilment, payment and delivery costs, and offer contact costs. Allocate costs incurred for customers who never buy to their assigned group too. Divide each group's total by all customers assigned to it.

The decision measure is the difference in contribution per assigned customer between offer and usual treatment. More orders in the offer group can still mean less contribution if the discount reduces what remains on purchases that would have happened anyway. The comparison estimates an average effect for the tested group; it cannot identify which individual purchases were inevitable.

Allow for shifted purchases and incomplete costs

Review cumulative orders and contribution through the next plausible buying opportunity, not only the offer period. A customer may buy a larger quantity early and need less later. Show later regular-price purchases and returns when they become observable. If the follow-up ends too soon, describe the timing question as unresolved.

Check what a cost report covers, and identify missing costs rather than treating them as zero.

Report group sizes, exposure overlap, the observed contribution difference and its uncertainty. If contribution falls despite more orders, the tested offer did not cover its measured cost in that group and period. If contribution rises, the result is promising for that setting, subject to the precision and completeness of the estimate.