Working papers
Nonparametric Identification of Demand without Exogenous Product Characteristics, with Kirill Borusyak, Peter Hull, and Lihua Lei
We show that $J$-dimensional exogenous variation in prices is sufficient for identifying counterfactuals in prices nonparametrically with recentered instruments.
Abstract
We study the identification of differentiated product demand with exogenous supply-side instruments, allowing product characteristics to be endogenous. Past analyses have argued that exogenous characteristic-based instruments are essentially necessary given a sufficiently flexible demand model with a suitable index restriction. We show, however, that price counterfactuals are nonparametrically identified by recentered instruments---which combine exogenous shocks to prices with endogenous product characteristics---under a weaker index restriction and a new condition we term faithfulness. We argue that faithfulness, like the usual completeness condition for nonparametric identification with instruments, can be viewed as a technical requirement on the richness of identifying variation rather than a substantive economic restriction, and we show that it holds under a variety of non-nested conditions on either price-setting or the index.Compound Selection Decisions: An Almost SURE Approach, with Lihua Lei, Timothy Sudijono, Liyang (Sophie) Sun, and Tian Xie
We extend Stein’s unbiased risk estimate (SURE) to compound selection decisions.
Abstract
This paper proposes methods for producing compound selection decisions in a Gaussian sequence model. Given unknown, fixed parameters $\mu_ {1:n}$ and known $\sigma_{1:n}$ with observations $Y_i \sim N(\mu_i, \sigma_i^2)$, the decision maker would like to select a subset of indices $S$ so as to maximize utility $\frac{1}{n}\sum_{i\in S} (\mu_i - K_i)$, for known costs $K_i$. Inspired by Stein's unbiased risk estimate (SURE), we introduce an almost unbiased estimator, called ASSURE, for the expected utility of a proposed decision rule. ASSURE allows a user to choose a welfare-maximizing rule from a pre-specified class by optimizing the estimated welfare, thereby producing selection decisions that borrow strength across noisy estimates. We show that ASSURE produces decision rules that are asymptotically no worse than the optimal but infeasible decision rule in the pre-specified class. We apply ASSURE to the selection of Census tracts for economic opportunity, the identification of discriminating firms, and the analysis of $p$-value decision procedures in A/B testing.Testing Monotonicity in a Finite Population, with Jonathan Roth and Jann Spiess
The standard definition of identifiability in design-based, finite-population settings suggests that the data are more informative than they actually are. We illustrate with inference on whether all treatment effects are of the same sign.
Abstract
We consider the extent to which we can learn from a completely randomized experiment whether everyone has treatment effects that are weakly of the same sign, a condition we call monotonicity. From a classical sampling perspective, it is well-known that monotonicity is untestable. By contrast, we show from the design-based perspective -- in which the units in the population are fixed and only treatment assignment is stochastic -- that the distribution of treatment effects in the finite population (and hence whether monotonicity holds) is formally identified. We argue, however, that the usual definition of identification is unnatural in the design-based setting because it imagines knowing the distribution of outcomes over different treatment assignments for the same units. We thus evaluate the informativeness of the data by the extent to which it enables frequentist testing and Bayesian updating. We show that frequentist tests can have nontrivial power against some alternatives, but power is generically limited. Likewise, we show that there exist (non-degenerate) Bayesian priors that never update about whether monotonicity holds. We conclude that, despite the formal identification result, the ability to learn about monotonicity from data in practice is severely limited.Certified Decisions, with Isaiah Andrews
We connect statistical inference with statistical decisions by thinking of inference as providing guarantees for decisions. This turns out to be essentially without loss—certified decisions implicitly conduct inference. Such certified decisions allow downstream decision-makers safety guarantees.
Abstract
Hypothesis tests and confidence intervals are ubiquitous in empirical research, yet their connection to subsequent decision-making is often unclear. We develop a theory of certified decisions that pairs recommended decisions with inferential guarantees. Specifically, we attach _P-certificates_---upper bounds on loss that hold with probability at least $1-\alpha$---to recommended actions. We show that such certificates allow "safe," risk-controlling adoption decisions for ambiguity-averse downstream decision-makers. We further prove that it is without loss to limit attention to P-certificates arising as minimax decisions over confidence sets, or what Manski (2021) terms "as-if decisions with a set estimate." A parallel argument applies to E-certified decisions obtained from e-values in settings with unbounded loss.Reinterpreting demand estimation
I translate two models of demand estimation (Berry and Haile, 2014, 2024) to the Neyman–Rubin model and show a Vytlacil (2002)-style equivalence result.
Abstract
This paper bridges the demand estimation and causal inference literatures by interpreting nonparametric structural assumptions as restrictions on counterfactual outcomes. It offers nontrivial and equivalent restatements of key demand estimation assumptions in the Neyman-Rubin potential outcomes model, for both settings with market-level data (Berry and Haile, 2014) and settings with demographic-specific market shares (Berry and Haile, 2024). The reformulation highlights a latent homogeneity assumption underlying structural demand models: The relationship between counterfactual outcomes is assumed to be identical across markets. This assumption is strong, but necessary for identification of market-level counterfactuals. Viewing structural demand models as misspecified but approximately correct reveals a tradeoff between specification flexibility and robustness to latent homogeneity.Empirical Bayes shrinkage (mostly) does not correct the measurement error in regression, with Jiaying Gu and Soonwoo Kwon
Abstract
In the value-added literature, it is often claimed that regressing on empirical Bayes shrinkage estimates corrects for the measurement error problem in linear regression. We clarify the conditions needed; we argue that these conditions are stronger than the those needed for classical measurement error correction, which we advocate for instead. Moreover, we show that the classical estimator cannot be improved without stronger assumptions. We extend these results to regressions on nonlinear transformations of the latent attribute and find generically slow minimax estimation rates.Potential weights and implicit causal designs in linear regression
I introduce a simple and generic diagnostic for design-based causal interpretation of regression estimands.
Abstract
When we interpret linear regression as estimating causal effects justified by quasi-experimental treatment variation, what do we mean? This paper characterizes the necessary implications when linear regressions are interpreted causally. A minimal requirement for causal interpretation is that the regression estimates some contrast of individual potential outcomes under the true treatment assignment process. This requirement implies linear restrictions on the true distribution of treatment. Solving these linear restrictions leads to a set of implicit designs. Implicit designs are plausible candidates for the true design if the regression were to be causal. The implicit designs serve as a framework that unifies and extends existing theoretical results across starkly distinct settings (including multiple treatment, panel, and instrumental variables). They lead to new theoretical insights for widely used but less understood specifications.Optimal Conditional Inference in Adaptive Experiments, with Isaiah Andrews
We study batched bandit experiments and identify a small free lunch for adaptive inference. For $ \varepsilon $-greedy-type experiments, we characterize the optimal conditional inference procedure given history of bandit assignment probabilities.
Abstract
We study batched bandit experiments and consider the problem of inference conditional on the realized stopping time, assignment probabilities, and target parameter, where all of these may be chosen adaptively using information up to the last batch of the experiment. Absent further restrictions on the experiment, we show that inference using only the results of the last batch is optimal. When the adaptive aspects of the experiment are known to be location-invariant, in the sense that they are unchanged when we shift all batch-arm means by a constant, we show that there is additional information in the data, captured by one additional linear function of the batch-arm means. In the more restrictive case where the stopping time, assignment probabilities, and target parameter are known to depend on the data only through a collection of polyhedral events, we derive computationally tractable and optimal conditional inference procedures.Resting / subsumed papers
Mean-variance constrained priors have finite maximum Bayes risk in the normal location model
Mostly Harmless Machine Learning: Learning Optimal Instruments in Linear IV Models, with Daniel Chen and Greg Lewis
[Accepted at NeurIPS 2020 Workshop on Machine Learning for Economic Policy] [arXiv]
Causal Inference and Matching Markets
Undergraduate thesis advised by Scott Duke Kominers and David C. Parkes. Awarded the Thomas T. Hoopes Prize at Harvard College.
[Simulable Mechanisms] [Cutoff Mechanisms] [Regression discontinuity with endogenous cutoff]