Timing reminders by usage, not dates: Use product pack size and customer-selected date to trigger reminders; Compare repeat purchases within 60 days after each reminder type; Hold offer, channel and frequency constant across both test groups
Image: Retention Marketing Desk

Repeat Purchase

Part of Replenishment retention

Testing reminders around product usage rather than fixed dates

Design a fair test of product-informed reminder timing versus a fixed delay, with consistent eligibility, suppression and outcome windows.

Lifecycle Retention Desk · · 3 min read

Compare a reminder timed from available product-use signals with one sent a fixed number of days after purchase. Keep eligibility, message and follow-up period consistent, so the comparison chiefly tests timing. Neither rule reveals when a customer will actually run out.

Define the signal and both rules

Actual consumption is usually unknown unless a customer supplies it. Specify the available information: product, pack size, quantity, a customer-selected date or a repeat-order pattern. Document how those inputs produce a send window, including a default when records are incomplete.

The fixed rule might send the same reminder after a set delay for every qualifying order. The usage-informed rule might wait longer for a larger pack or use the customer's chosen date.

These are test designs, not evidence that one is better. Exclude returned orders and customers with a relevant subscription delivery already planned.

Usage-Based vs Fixed-Timing Reminders: Key Differences in Design

  • Timing TriggerProduct usage signals (e.g. pack size, customer-selected date, repeat-order pattern)
  • Eligibility CriteriaConsistent across both groups; exclude returns and existing subscriptions
  • Send Window FlexibilityAdapts to product size or user input (e.g., larger packs = longer window)
  • Send Window FlexibilityFixed for all orders (same delay regardless of pack size)
  • Default HandlingUses fallback logic when data is incomplete

Allocate customers before either send

Randomly assign eligible customers to one timing rule before the first possible reminder, and keep each customer in that rule throughout the test. Apply the same checks for a later relevant purchase, opt-out or subscription delivery in both groups.

Hold the product shown, offer, channel and frequency steady. Use a random split before the timing difference begins; a 50/50 allocation is an even test, though random assignment may not produce exactly equal group sizes.

Choose one primary outcome in advance, such as relevant repeat purchase within a common period starting at eligibility. Give both groups enough follow-up time to observe a purchase after the later send.

Analyse everyone assigned to each rule, including people whose reminder was suppressed under the shared rules. Track opt-outs, complaints and duplicate sends as guardrails.

If the question is whether any reminder adds purchases, use a separate no-reminder control. Two send-time groups can show which timing performs better in the test; they cannot establish that either improves on sending nothing.

Klaviyo's global holdout is broader than a single replenishment flow and is available only to accounts meeting its profile threshold, so its scope should not be assumed to fit this test.

How to Conduct a Fair Test Between Reminder Timing Rules

  1. Define EligibilitySet consistent criteria (e.g., first-time purchase, no active subscription)
  2. Random AllocationAssign customers randomly to either usage-based or fixed-timing rule before any send
  3. Maintain ConsistencyKeep message, offer, channel and frequency unchanged across both groups
  4. Track OutcomesMeasure primary outcome (e.g. repeat purchase), including opt-outs and complaints

Read beyond the first response

A reminder may move an order earlier without increasing eventual repeat purchasing. Compare cumulative purchases across the full follow-up period, not just the days after each message.

Check contribution if discounts, delivery costs or order mix differ. Product and quantity breakdowns can help explain the result, but small subgroups should remain exploratory.

Key Performance Indicators to Monitor in Reminder Timing Tests

Cumulative Purchases
Total purchases over full follow-up period (not just post-message)
Opt-Out Rate
Percentage of recipients who opt out after reminder
Complaints
Spam complaints reported via ATO or email provider
Duplicate Sends
Number of customers receiving multiple reminders

More from Repeat Purchase