Adaptation

• The air force example:
  • When students are mixed with low/high/medium achievers, low groups tend to perform better while medium and high groups are not affected.
  • When students are grouped with low/high achievers, low groups tend to perform worse.
  • When students are group with medium achievers only, they tend to perform better.
• Explanation:
  • The researchers identified peer effects in particular setting.
  • Though they thought the peer effects are the deep parameter, the peer effects are not universal.
    • Deep parameter: a fundamental parameter that describes a relationship.
  • Instead, they measured a particular relationship that depended on social circumstances.
• The general problem:
  • Many phenomena that we are interested in testing and measuring are going to differ in different settings.
  • The general problem is adaptation.
  • Take into account adaptation:
    • Alter the kinds of questions that we ask
    • The kinds of analyses that we conduct
  • Careful quantitative analysis requires thinking about adaptation and equilibrium responses.
• Problems of Adaptation:
  • Studying the Wrong Outcome
    • e.g. mental detectors at airports significantly decreased hijackings.
      • Research design isn't great because nothing was randomized.
      • However, even if we fix the research design, the research question is still wrong.
      • The right question is to look at all terrorist attacks (weighted by the social cost).
  • Extrapolating from a Narrow Quantitative Result
    • e.g. the air force example.