Designing and analyzing randomized experiments: Application to a Japanese election survey experiment

Yusaku Horiuchi, Kosuke Imai, Naoko Taniguchi

Research output: Contribution to journalArticle

52 Citations (Scopus)

Abstract

Randomized experiments are becoming increasingly common in political science. Despite their well-known advantages over observational studies, randomized experiments are not free from complications. In particular, researchers often cannot force subjects to comply with treatment assignment and to provide the requested information. Furthermore, simple randomization of treatments remains the most commonly used method in the discipline even though more efficient procedures are available. Building on the recent statistical literature, we address these methodological issues by offering general recommendations for designing and analyzing randomized experiments to improve the validity and efficiency of causal inference. We also develop a new statistical methodology to explore causal heterogeneity. The proposed methods are applied to a survey experiment conducted during Japan's 2004 Upper House election, where randomly selected voters were encouraged to obtain policy information from political parties' websites. An R package is publicly available for implementing various methods useful for designing and analyzing randomized experiments.

Original languageEnglish
Pages (from-to)669-687
Number of pages19
JournalAmerican Journal of Political Science
Volume51
Issue number3
DOIs
Publication statusPublished - 2007 Jul
Externally publishedYes

Fingerprint

election
experiment
information policy
political science
website
Japan
efficiency
methodology

ASJC Scopus subject areas

  • Sociology and Political Science

Cite this

Designing and analyzing randomized experiments : Application to a Japanese election survey experiment. / Horiuchi, Yusaku; Imai, Kosuke; Taniguchi, Naoko.

In: American Journal of Political Science, Vol. 51, No. 3, 07.2007, p. 669-687.

Research output: Contribution to journalArticle

@article{c5ca152c04244439b1b337d959a1b4b7,
title = "Designing and analyzing randomized experiments: Application to a Japanese election survey experiment",
abstract = "Randomized experiments are becoming increasingly common in political science. Despite their well-known advantages over observational studies, randomized experiments are not free from complications. In particular, researchers often cannot force subjects to comply with treatment assignment and to provide the requested information. Furthermore, simple randomization of treatments remains the most commonly used method in the discipline even though more efficient procedures are available. Building on the recent statistical literature, we address these methodological issues by offering general recommendations for designing and analyzing randomized experiments to improve the validity and efficiency of causal inference. We also develop a new statistical methodology to explore causal heterogeneity. The proposed methods are applied to a survey experiment conducted during Japan's 2004 Upper House election, where randomly selected voters were encouraged to obtain policy information from political parties' websites. An R package is publicly available for implementing various methods useful for designing and analyzing randomized experiments.",
author = "Yusaku Horiuchi and Kosuke Imai and Naoko Taniguchi",
year = "2007",
month = "7",
doi = "10.1111/j.1540-5907.2007.00274.x",
language = "English",
volume = "51",
pages = "669--687",
journal = "American Journal of Political Science",
issn = "0092-5853",
publisher = "Wiley-Blackwell",
number = "3",

}

TY - JOUR

T1 - Designing and analyzing randomized experiments

T2 - Application to a Japanese election survey experiment

AU - Horiuchi, Yusaku

AU - Imai, Kosuke

AU - Taniguchi, Naoko

PY - 2007/7

Y1 - 2007/7

N2 - Randomized experiments are becoming increasingly common in political science. Despite their well-known advantages over observational studies, randomized experiments are not free from complications. In particular, researchers often cannot force subjects to comply with treatment assignment and to provide the requested information. Furthermore, simple randomization of treatments remains the most commonly used method in the discipline even though more efficient procedures are available. Building on the recent statistical literature, we address these methodological issues by offering general recommendations for designing and analyzing randomized experiments to improve the validity and efficiency of causal inference. We also develop a new statistical methodology to explore causal heterogeneity. The proposed methods are applied to a survey experiment conducted during Japan's 2004 Upper House election, where randomly selected voters were encouraged to obtain policy information from political parties' websites. An R package is publicly available for implementing various methods useful for designing and analyzing randomized experiments.

AB - Randomized experiments are becoming increasingly common in political science. Despite their well-known advantages over observational studies, randomized experiments are not free from complications. In particular, researchers often cannot force subjects to comply with treatment assignment and to provide the requested information. Furthermore, simple randomization of treatments remains the most commonly used method in the discipline even though more efficient procedures are available. Building on the recent statistical literature, we address these methodological issues by offering general recommendations for designing and analyzing randomized experiments to improve the validity and efficiency of causal inference. We also develop a new statistical methodology to explore causal heterogeneity. The proposed methods are applied to a survey experiment conducted during Japan's 2004 Upper House election, where randomly selected voters were encouraged to obtain policy information from political parties' websites. An R package is publicly available for implementing various methods useful for designing and analyzing randomized experiments.

UR - http://www.scopus.com/inward/record.url?scp=34250686881&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34250686881&partnerID=8YFLogxK

U2 - 10.1111/j.1540-5907.2007.00274.x

DO - 10.1111/j.1540-5907.2007.00274.x

M3 - Article

AN - SCOPUS:34250686881

VL - 51

SP - 669

EP - 687

JO - American Journal of Political Science

JF - American Journal of Political Science

SN - 0092-5853

IS - 3

ER -