Compound Selection Decisions: An Almost SURE Approach, with Lihua Lei, Timothy Sudijono, Liyang (Sophie) Sun, and Tian Xie
We extend Stein’s unbiased risk estimate (SURE) to compound selection decisions.
Abstract
This paper proposes methods for producing compound selection decisions in a Gaussian sequence model. Given unknown, fixed parameters $\mu_ {1:n}$ and known $\sigma_{1:n}$ with observations $Y_i \sim N(\mu_i, \sigma_i^2)$, the decision maker would like to select a subset of indices $S$ so as to maximize utility $\frac{1}{n}\sum_{i\in S} (\mu_i - K_i)$, for known costs $K_i$. Inspired by Stein's unbiased risk estimate (SURE), we introduce an almost unbiased estimator, called ASSURE, for the expected utility of a proposed decision rule. ASSURE allows a user to choose a welfare-maximizing rule from a pre-specified class by optimizing the estimated welfare, thereby producing selection decisions that borrow strength across noisy estimates. We show that ASSURE produces decision rules that are asymptotically no worse than the optimal but infeasible decision rule in the pre-specified class. We apply ASSURE to the selection of Census tracts for economic opportunity, the identification of discriminating firms, and the analysis of $p$-value decision procedures in A/B testing.The purpose of an estimator is what it does: Misspecification, estimands, and over-identification, with Isaiah Andrews and Otavio Tecchio
We review results on estimation under model misspecification and provide a new result on the interpretation of J-statistics. Prepared for the 2025 World Congress of the Econometric Society.
Abstract
In over-identified models, misspecification---the norm rather than exception---fundamentally changes what estimators estimate. Different estimators imply different estimands rather than different efficiency for the same target. A review of recent applications of generalized method of moments in the American Economic Review suggests widespread acceptance of this fact: There is little formal specification testing and widespread use of estimators that would be inefficient were the model correct, including the use of "hand-selected" moments and weighting matrices. Motivated by these observations, we review and synthesize recent results on estimation under model misspecification, providing guidelines for transparent and robust empirical research. We also provide a new theoretical result, showing that Hansen's J-statistic measures, asymptotically, the range of estimates achievable at a given standard error. Given the widespread use of inefficient estimators and the resulting researcher degrees of freedom, we thus particularly recommend the broader reporting of J-statistics.Reinterpreting demand estimation
I translate two models of demand estimation (Berry and Haile, 2014, 2024) to the Neyman–Rubin model and show a Vytlacil (2002)-style equivalence result.
Abstract
This paper bridges the demand estimation and causal inference literatures by interpreting nonparametric structural assumptions as restrictions on counterfactual outcomes. It offers nontrivial and equivalent restatements of key demand estimation assumptions in the Neyman-Rubin potential outcomes model, for both settings with market-level data (Berry and Haile, 2014) and settings with demographic-specific market shares (Berry and Haile, 2024). The reformulation highlights a latent homogeneity assumption underlying structural demand models: The relationship between counterfactual outcomes is assumed to be identical across markets. This assumption is strong, but necessary for identification of market-level counterfactuals. Viewing structural demand models as misspecified but approximately correct reveals a tradeoff between specification flexibility and robustness to latent homogeneity.Empirical Bayes shrinkage (mostly) does not correct the measurement error in regression, with Jiaying Gu and Soonwoo Kwon
Abstract
In the value-added literature, it is often claimed that regressing on empirical Bayes shrinkage estimates corrects for the measurement error problem in linear regression. We clarify the conditions needed; we argue that these conditions are stronger than the those needed for classical measurement error correction, which we advocate for instead. Moreover, we show that the classical estimator cannot be improved without stronger assumptions. We extend these results to regressions on nonlinear transformations of the latent attribute and find generically slow minimax estimation rates.Certified Decisions, with Isaiah Andrews
We connect statistical inference with statistical decisions by thinking of inference as providing guarantees for decisions. This turns out to be essentially without loss—certified decisions implicitly conduct inference. Such certified decisions allow downstream decision-makers safety guarantees.
Abstract
Hypothesis tests and confidence intervals are ubiquitous in empirical research, yet their connection to subsequent decision-making is often unclear. We develop a theory of certified decisions that pairs recommended decisions with inferential guarantees. Specifically, we attach _P-certificates_---upper bounds on loss that hold with probability at least $1-\alpha$---to recommended actions. We show that such certificates allow "safe," risk-controlling adoption decisions for ambiguity-averse downstream decision-makers. We further prove that it is without loss to limit attention to P-certificates arising as minimax decisions over confidence sets, or what Manski (2021) terms "as-if decisions with a set estimate." A parallel argument applies to E-certified decisions obtained from e-values in settings with unbounded loss.Potential weights and implicit causal designs in linear regression
I introduce a simple and generic diagnostic for design-based causal interpretation of regression estimands.
Abstract
When we interpret linear regression as estimating causal effects justified by quasi-experimental treatment variation, what do we mean? This paper characterizes the necessary implications when linear regressions are interpreted causally. A minimal requirement for causal interpretation is that the regression estimates some contrast of individual potential outcomes under the true treatment assignment process. This requirement implies linear restrictions on the true distribution of treatment. Solving these linear restrictions leads to a set of implicit designs. Implicit designs are plausible candidates for the true design if the regression were to be causal. The implicit designs serve as a framework that unifies and extends existing theoretical results across starkly distinct settings (including multiple treatment, panel, and instrumental variables). They lead to new theoretical insights for widely used but less understood specifications.Optimal Conditional Inference in Adaptive Experiments, with Isaiah Andrews
We study batched bandit experiments and identify a small free lunch for adaptive inference. For $ \varepsilon $-greedy-type experiments, we characterize the optimal conditional inference procedure given history of bandit assignment probabilities.
Abstract
We study batched bandit experiments and consider the problem of inference conditional on the realized stopping time, assignment probabilities, and target parameter, where all of these may be chosen adaptively using information up to the last batch of the experiment. Absent further restrictions on the experiment, we show that inference using only the results of the last batch is optimal. When the adaptive aspects of the experiment are known to be location-invariant, in the sense that they are unchanged when we shift all batch-arm means by a constant, we show that there is additional information in the data, captured by one additional linear function of the batch-arm means. In the more restrictive case where the stopping time, assignment probabilities, and target parameter are known to depend on the data only through a collection of polyhedral events, we derive computationally tractable and optimal conditional inference procedures.Resting / subsumed papers
Mean-variance constrained priors have finite maximum Bayes risk in the normal location model
Mostly Harmless Machine Learning: Learning Optimal Instruments in Linear IV Models, with Daniel Chen and Greg Lewis
[Accepted at NeurIPS 2020 Workshop on Machine Learning for Economic Policy] [arXiv]
Causal Inference and Matching Markets
Undergraduate thesis advised by Scott Duke Kominers and David C. Parkes. Awarded the Thomas T. Hoopes Prize at Harvard College.
[Simulable Mechanisms] [Cutoff Mechanisms] [Regression discontinuity with endogenous cutoff]