Paul Pfleiderer
About
Economist who discussed model misuse on a prior EconTalk episode
Cast within
No topic-region cast yet — this appears once Paul Pfleiderer's compiled claims are aligned into a topic region's argument tree.
Claims by Paul Pfleiderer (18)
Friedman's As-If Defense Misused
Friedman's 'as if' argument—that assumptions need not be realistic so long as agents behave as if they were—should not be used to shield chameleon models from criticism of their assumptions when those assumptions have no intersection with the real world.
Triple-A Rating Was Gamed In Securitization
Unlike a corporation (e.g. Hewlett Packard) whose ability to adjust bond risk is third-order, issuers of securitized products in 2006-2007 had both the incentive and fine-grained control to engineer Triple-A ratings, so deals clustered right at the rating margin and carried more risk than equivalently rated corporate bonds.
Chameleon Models Straddle Two Worlds
A 'chameleon model' illegitimately straddles two worlds: it is presented as having real-world policy implications, but when criticized for unrealistic assumptions it retreats to being 'just a logical exercise,' thereby gaining an ontological status it doesn't deserve without passing through the filter of testing its assumptions.
Risk Models Get Gamed Under Incentives
A risk measure like Value at Risk may be fine when used passively to observe someone's risk, but becomes far more problematic once it is set as the benchmark for someone who has an incentive to take risk, because very smart agents in industries like finance will game the measure.
Strong Priors Resist Falsification Via Tweaks
When a prediction fails, a sufficiently strong prior lets a modeler add a new assumption or tweak the model to explain away the contradiction and preserve cherished assumptions; doing this every time a large natural experiment fails violates scientific method and means the view is unfalsifiable.
Any Result Follows From Chosen Assumptions
For almost any result that doesn't rely on a logical contradiction, you can reverse-engineer a set of assumptions that will produce it; this 'theoretical cherry-picking' means a model matching a desired result tells you nothing about whether its assumptions are true.
Optimization Assumption Hides Modeling Difficulty
Modeling becomes deceptively easy if you assume agents optimize like physicists, because then everything reduces to math; but realistic modeling must also describe how boundedly-rational agents muddle through via heuristics and trial-and-error with few trials, which is far harder and is often avoided—'looking for the keys under the lamppost.'
Matching Reality Doesn't Validate Assumptions
Because a researcher can pick from many alternative assumption sets and calibrate them with enough degrees of freedom, successfully reproducing an observed phenomenon B gives no license to conclude the chosen assumptions are correct or to transport them to explain other phenomena.
Natural Experiments Driven By Availability
Empirical economics increasingly relies on natural experiments, but these are scarce and often not aimed at the big questions; the research process can be inverted—finding an available natural experiment first and then choosing the analysis—though this is partly too cynical a characterization.
Critical Analytical Thinking As Discipline
The remedy for misused models is 'critical analytical thinking': approach every issue with skepticism, check that arguments are logical, and constantly ask where the evidence is—recognizing that if a key premise is unsupported, the whole argument is unsupported.
As-If Works Only With Repeated Fast Feedback
The 'as if' argument works for the pool player and the fly-ball catcher because they get a million-plus repetitions with near-instantaneous feedback in a structured environment; it fails for decisions like corporate capital structure where the decision is rare, the feedback is slow and ambiguous (requiring the stock market itself to be solving the equations), and experts can't even agree on the model.
Economics Is Non-Stationary Because People React
Unlike chemistry or physics, economics is highly non-stationary: laws change because people think and react to what economists say, so a predicted behavior can be undone once people read the prediction and act differently, making the science fundamentally harder.
Greek Letters Add False Patina
Elaborate models with many variables, tables, graphs, and Greek letters carry a patina of scientific truth—especially to lay persons—and therefore more weight in debate than they deserve, even though the underlying assumptions may render the result vacuous.
The Problem Is Collective Lack Of Humility
Beyond individual overconfidence, the deeper issue is a collective lack of humility in the economics profession about what it can and cannot say about the world, even though properly recognizing limitations and putting correct error bounds on results is itself the scientific thing to do.
Incentives Reward Modeling Complexity And Confidence
PhD programs put a huge premium on developing and solving complicated models, and confident modelers who claim their calibrations reveal truth get more credit than they deserve because people want scientific-seeming answers, so the reward structure pushes against the humility the profession needs.
Ten Experts Yield Ten Different Models
For genuinely tractable physical problems ten physicists would produce essentially the same model of how a pool player or fielder behaves, whereas ten economists modeling something like capital structure produce ten different models—revealing that the underlying problem is not actually well understood.
No Alternative To Imperfect Scientific Method
Despite the limitations of models and evidence in economics, there is no alternative to formulating well-grounded models, doing empirical work, and seeking natural experiments to learn as much as possible; abandoning that would leave only a shouting match where anything goes.
My Notes
Loading notes...