Services

Experiments as a Service (EaaS)

Our Experiments as a Service (EaaS) makes running high-quality experiments simple. Our team handles everything, from experiment design and pre-testing to execution, analysis, and reporting.

Consulting

We provide expert consulting to help brands, agencies, and measurement firms integrate randomized controlled trials (RCTs) into their measurement strategies. RCTs are the only scientifically sanctioned method to measure true incremental effect and ergo ROI, making them essential for accurate media performance analysis.

Training

Our training workshops teach advertisers, agencies, and researchers the fundamentals of RCTs and ROI measurement. Learn why traditional methods like marketing mix models (MMM), attribution models, and quasi-experiments fall short when it comes to making million-dollar decisions. We cover which types of experiments work best across media channels and KPIs, and how RCTs can complement MMM and multi-touch attribution (MTA).

MPE: Models Plus Experiments

Models Plus Experiments (MPE) is the next evolution beyond MMM and MTA. By incorporating RCT results into MMM calibration, advertisers can fine-tune their model coefficients and enhance accuracy. We work with MMM developers to integrate experimental data into marketing models for better decision-making.

Experimental Design Methods

Experiment Designer (ExD) supports a number of campaign designs out of the box, with additional methodologies available through our consulting services. ExD and consulting frameworks include:

  • Geo Experiments: Randomize media exposures by DMAs, Cable Zones, or ZIP Codes.

  • User & Household-Level: Randomize exposures by user, device, account, or household units.

  • Multi-Armed Designs: Test 3 to 8 randomized groups to explore various conditions.

  • Block & Stratified Randomization: Ensure even distribution of experimental groups based on key criteria.

  • A/B Tests: Compare two creative treatments or tactics for optimal ROI.

  • Factorial Design: Measure combinations of factors to assess interaction effects and individual performance.

  • Intensity Tests: Test different media weights to find the optimal frequency for MMM calibration.

  • Crossover Design: Alternating test and control conditions within the same audience

  • Rolling Thunder: Multi-armed geo-experiment with cascading, overlappingg test groups and one control

  • Ghost Ads: Design, pre-testing, documentation and benchmarking for “sales lift” tests by programmatic media.

  • Multivariate Analysis: Enhance experiments with statistical methods that include MANOVA, regression, and Difference-in-Differences.