Why ordinary A/B tests can lose power when outcomes are networked
Standard A/B testing is built for direct effects. Randomize units, compare treated with control, estimate the lift. That logic gets strained when the target is spillover: how my outcome changes when my neighbors, suppliers, classmates, or customers are treated.
In the setup studied by Kirill Borusyak, Peter Hull, and Evan Munro, each outcome unit has an exposure measure. A simple version is the fraction or number of connected intervention units assigned to treatment. If outcome unit i is linked to intervention units k, exposure is a weighted sum of their treatment assignments.
Independent randomization can be weak for this purpose. If a person has many treated-or-control contacts, independent assignment may make their exposure hover near the average treatment rate. The spillover variable has little variation. That is bad news for power.
There is a second problem. Exposure is not cleanly individualized. Two outcome units may share the same treated supplier, teacher, village, advertiser, or friend group. Their exposure variables move together. If their unobserved outcomes also move together, the estimator can get noisy fast. Ordinary regressions on raw exposure can also lose a clean causal interpretation when the network itself is not as-good-as-random. The paper's answer is to design the randomization and estimator around the exposure variable directly.
The core trade-off: stronger signal versus more diffuse variation
The paper's main intuition is simple and useful.
If several intervention units feed into the same outcome unit's exposure, it can help to assign those intervention units in a correlated way. Treat them together more often. Control them together more often. That makes the outcome unit's exposure swing more. The spillover signal gets stronger.
But correlated assignment also concentrates variation. Instead of many small independent shocks, the experiment starts to look like a smaller number of group-level shocks. That can be costly when outcome errors are correlated within the same parts of the network. You have a bigger first stage, but fewer effectively independent comparisons.
So the design problem is not just "make exposure variance large." It is "make exposure variance large without putting all the identifying variation in a few network directions." The authors call this second goal isotropy. In plain English, spread the useful variation around the sample rather than letting it pile up in the same clusters or high-degree nodes.
That trade-off explains why neither individual randomization nor cluster randomization is always right. Individual randomization can produce too little spillover movement. Full cluster randomization can produce strong exposure movement but too little independent variation. In some settings, the best design mixes the two.
The proposed fix: minimax randomization plus a recentered IV estimator
The paper assumes the researcher knows the exposure mapping before the experiment. That is a serious assumption, but also a common one in network experiments. You need to know which links define spillover exposure, and how they are weighted.
Given that mapping, the authors choose a treatment-assignment distribution and an estimator to minimize worst-case asymptotic variance over a broad class of possible unobserved outcome errors. The researcher does not need to specify the exact covariance matrix of the errors. Instead, the method uses a parsimonious parameter for how much dependence, heteroskedasticity, and network-related error structure the planner wants to allow.
The design side chooses how treatment assignments should be correlated across intervention units. The estimator side uses a recentered instrumental variables approach. The instrument uses the random part of spillover exposure induced by the assignment rule, after subtracting the exposure's expectation under that rule. The estimator can also reweight the data so the identifying variation is less concentrated. That may reduce raw signal, but it can make the variation more useful for estimating the spillover parameter.
A nice feature is that familiar designs appear as special cases. With no spillovers, independent randomization is the answer. With cluster-level exposure, cluster randomization emerges naturally. With leave-one-out cluster exposure, the optimal design mixes individual and cluster assignment. For more general networks, including bipartite designs where intervention units and outcome units differ, the paper gives computational approximations rather than pretending there is a closed-form answer for every graph.
The authors report large standard error reductions in semi-synthetic exercises based on development economics applications, including insurance diffusion and deworming spillovers. The exact gain will depend on the network, the exposure rule, and the benchmark design, but the point is clear: design choices can matter a lot when adding field units is expensive.
What is actually new relative to clustered and network-experiment designs
This is not just "randomize by cluster." The paper's contribution is broader and more surgical.
Earlier work on partial interference often starts with clusters and asks how much treatment saturation to assign within each cluster. Work on bipartite experiments has studied clustered assignments for particular network structures and estimators. Those are useful cases, but they restrict the design class up front.
Here the network can be general. The intervention and outcome units can be the same or different. The design is not forced to be individual-level or cluster-level. The estimator is not fixed. The randomization and the recentered IV are optimized together.
The recentering point also matters. If exposure is correlated with baseline traits through the network, a raw exposure regression may not estimate a causal spillover parameter. Recentered estimators, following the logic in Borusyak and Hull, use only the random part of exposure induced by the assignment rule. This paper folds that causal identification step into the power calculation.
Why we could not backtest this on our data
We could not do a credible backtest with our existing data because the method is a design-stage tool. To evaluate it properly, we would need the full pre-randomization exposure network, the original assignment constraints, and the ability to simulate alternative randomization schemes before outcomes were realized.
A post-hoc comparison using observed treatment take-up and outcomes would not answer the right question. It would mix the proposed design with the design we actually ran. That would be unfair to the paper and not very informative for us.
Where this is useful for practitioners, and where it is not
This is most useful when spillovers are a primary estimand and the organization has real control over assignment. Think referral programs, supplier interventions, classroom or village experiments, marketplace treatments where sellers affect buyers, or ads where treated users influence connected users. If the network is measured before launch and the exposure rule is credible, the method gives a principled way to spend the same experimental budget more efficiently.
It is less useful when the network is missing, unstable, or itself affected by treatment. It also helps less when the direct treatment effect is the only business question. Operational constraints can bind too. Product teams may not be allowed to correlate assignments across connected users, legal teams may require simple randomization, and platform systems may not support custom joint assignment rules.
The practical burden is front-loaded. You need to define the exposure mapping, compute the assignment distribution, implement it correctly, and use the matching recentered IV estimator. If those pieces are not in place, this is not a plug-in variance fix after the fact.
Still, the paper's message is useful even before adopting the full algorithm: if spillovers are the target, independent A/B randomization may be spending variation in the wrong place.