Background Image

...

Expert Estimates Of Probability Are Often Off By Factors Of Hundreds Or Thousands. [...] I Used To Be Annoyed When The Margin Of Error Was High In A Forecasting Model That I Might Put Together. Now I View It As Perhaps The Single Most Important Piece Of Information That A Forecaster Provides. When We Publish A Forecast On Fivethirtyeight, I Go To Great Lengths To Document The Uncertainty Attached To It, Even If The Uncertainty Is Sufficiently Large That The Forecast Won't Make For Punchy Headlines.

-Nate Silver

...

Please Wait....
Translating....

Expert Estimates Of Probability

Nate Silver

...

Expert Estimates Of Probability Are Often Off By Factors Of Hundreds Or Thousands. [...] I Used To Be Annoyed When The Margin Of Error Was High In A Forecasting Model That I Might Put Together. Now I View It As Perhaps The Single Most Important Piece Of Information That A Forecaster Provides. When We Publish A Forecast On Fivethirtyeight, I Go To Great Lengths To Document The Uncertainty Attached To It, Even If The Uncertainty Is Sufficiently Large That The Forecast Won't Make For Punchy Headlines.

Views: 5

Topic

More From Nate Silver

The Key To Making A Good Forecast Is Not In Limiting Yourself To Quantitative Information.

Limiting Yourself

When A Possibility Is Unfamiliar To Us, We Do Not Even Think About It.

Caesar Recognized The Omens, But He Didn't Believe They Applied To Him.

Walk Rate Is Probably The Area In Which A Pitcher Has The Most Room To Improve, But A Rate That High Is Tough To Overcome.

I Have To Think About How To Not Spread Myself Too Thin. It's A Really Great Problem To Have.

Actually, One Of The Better Indicators Historically Of How Well The Stock Market Will Do Is Just A Gallup Poll, When You Ask Americans If You Think It's A Good Time To Invest In Stocks, Except It Goes The Opposite Direction Of What You Would Expect. When The Markets Going Up, It In Fact Makes It More Prone Toward Decline.

Any One Game In Baseball Doesn't Tell You That Much, Just As Any One Poll Doesn't Tell You That Much.

Basically, Books Were A Luxury Item Before The Printing Press.

Voters' Memories Will Fade Some.

I Think Punditry Serves No Purpose.

The Public Is Even More Pessimistic About The Economy Than Even The Most Bearish Economists Are.

We Must Become More Comfortable With Probability And Uncertainty.

To Be A Very, Very Minor, Eighth-tier Celebrity, You Realize, 'hey, Celebrities Are Just Like Us.'

Midterm Elections Can Be Dreadfully Boring, Unfortunately.

Distinguishing The Signal From The Noise Requires Both Scientific Knowledge And Self-knowledge: The Serenity To Accept The Things We Cannot Predict, The Courage To Predict The Things We Can, And The Wisdom To Know The Difference.

People Have Different Ways Of Interpreting History.

When You Get Into Statistical Analysis, You Don't Really Expect To Achieve Fame. Or To Become An Internet Meme. Or Be Parodied By 'the Onion' - Or Be The Subject Of A Cartoon In 'the New Yorker.' I Guess I'm Kind Of An Outlier There.

We Speak For Them. We Imbue Them With Meaning.

I Was Looking For Something Like Baseball, Where There's A Lot Of Data And The Competition Was Pretty Low. That's When I Discovered Politics.

Economy Is Not Baseball, Where The Game Is Always Played By The Same Rules.

You Don't Want To Treat Any One Person As Oracular.

All I Know Is That I Have Way More Stuff That I Want To Write About Than I Possibly Have Time To.

It's A Little Strange To Become A Kind Of Symbol Of A Whole Type Of Analysis.

People Still Don't Appreciate How Ephemeral Success Is.

A Lot Of Things Can't Be Modeled Very Well.

I Love South American Food, And I Haven't Really Been Down There. I Really Need A Vacation.

I Have To Make Sure That I Make Good Choices And That If I Put My Name On It, It's A High-quality Endeavor And That I Have Time To Be A Human Being.

I've Become Invested With This Symbolic Power. It Really Does Transcend What I'm Actually Doing And What I Actually Deserve.

The Problem Is That When Polls Are Wrong, They Tend To Be Wrong In The Same Direction. If They Miss In New Hampshire, For Instance, They All Miss On The Same Mistake.

If I Had A Spreadsheet On My Computer, It Looked Like I Was Busy.

We Need To Stop, And Admit It: We Have A Prediction Problem. We Love To Predict Things—and We Aren’t Very Good At It.

I Think People Feel Like There Are All These Things In Our Lives That We Don't Really Have Control Over.

In Politics People Build Whole Reputations Off Of Getting One Thing Right.

The Signal Is The Truth. The Noise Is What Distracts Us From The Truth.

I'm Not Trying To Do Anything Too Tricky.

People Gravitate Toward Information That Implies A Happier Outlook For Them.

Every Day, Three Times Per Second, We Produce The Equivalent Of The Amount Of Data That The Library Of Congress Has In It's Entire Print Collection, Right? But Most Of It Is Like Cat Videos On Youtube Or Thirteen-year-olds Exchanging Text Messages About The Next 'twilight' Movie.

I Don't Think That Somebody Who Is Observing Or Predicting Behavior Should Also Be Participating In The 'experiment.'

We Look At All The Polls, Not Just The Gallup Poll. So, It's Kind Of Like If You Have, You Know, Four Out Of Five Doctors Agree That Reducing Cholesterol Reduces Your Risk Of A Heart Attack, Gallup Is Like The Fifth Doctor.

Finding Patterns Is Easy In Any Kind Of Data-rich Environment; That's What Mediocre Gamblers Do. The Key Is In Determining Whether The Patterns Represent Signal Or Noise

You Don't Want To Influence The Same System You Are Trying To Forecast.

Remember, The Congress Doesn't Get As Many Opportunities To Make An Impression With The Public.

The Protestant Reformation Had A Lot To Do With The Printing Press, Where Martin Luther's Theses Were Reproduced About 250,000 Times. And So You Had Widespread Dissemination Of Ideas That Hadn't Circulated In The Mainstream Before.

We're Living In A World Where Google Beats Gallup.

If The State Polls Are Right, Then Mr. Obama Will Win The Electoral College. If You Can't Acknowledge That After A Day When Mr. Obama Leads 19 Out Of 20 Swing-state Polls, Then You Should Abandon The Pretense That Your Goal Is To Inform Rather Than Entertain The Public.

A Lot Of News Is Just Entertainment Masquerading As News.

I Think A Lot Of Journal Articles Should Really Be Blogs.

Expert Estimates Of Probability Are Often Off By Factors Of Hundreds Or Thousands. [...] I Used To Be Annoyed When The Margin Of Error Was High In A Forecasting Model That I Might Put Together. Now I View It As Perhaps The Single Most Important Piece Of Information That A Forecaster Provides. When We Publish A Forecast On Fivethirtyeight, I Go To Great Lengths To Document The Uncertainty Attached To It, Even If The Uncertainty Is Sufficiently Large That The Forecast Won't Make For Punchy Headlines.

Every Four Years In The Presidential Election, Some New Precedent Is Broken.

If You're Keeping Yourself In The Bubble And Only Looking At Your Own Data Or Only Watching The Tv That Fits Your Agenda Then It Gets Boring.

I Have The Same Friends And The Same Bad Habits.

There's Always The Risk That There Are Unknown Unknowns.

A Lot Of Journalism Wants To Have What They Call Objectivity Without Them Having A Commitment To Pursuing The Truth, But That Doesn't Work. Objectivity Requires Belief In And A Commitment Toward Pursuing The Truth - Having An Object Outside Of Our Personal Point Of View.

Race Is Still The No. 1 Determinant In Every Election.

Data-driven Predictions Can Succeed-and They Can Fail. It Is When We Deny Our Role In The Process That The Odds Of Failure Rise. Before We Demand More Of Our Data, We Need To Demand More Of Ourselves.

Well The Way We Perceive Accuracy And What Accuracy Is Statistically Are Really Two Different Things.

People Don't Have A Good Intuitive Sense Of How To Weigh New Information In Light Of What They Already Know. They Tend To Overrate It.

You Can Build A Statistical Model And That's All Well And Good, But If You're Dealing With A New Type Of Financial Instrument, For Example, Or A New Type Of Situation - Then The Choices You're Making Are Pretty Arbitrary In A Lot Of Respects.

By Playing Games You Can Artificially Speed Up Your Learning Curve To Develop The Right Kind Of Thought Processes.

If You Aren't Taking A Representative Sample, You Won't Get A Representative Snapshot.

Representatives

I View My Role Now As Providing More Of A Macro-level Skepticism, Rather Than Saying This Poll Is Good Or This Poll Is Evil.

On Average, People Should Be More Skeptical When They See Numbers. They Should Be More Willing To Play Around With The Data Themselves.

I Think There's Space In The Market For A Half-dozen Kind Of Polling Analysts.

One Of The Pervasive Risks That We Face In The Information Age, As I Wrote In The Introduction, Is That Even If The Amount Of Knowledge In The World Is Increasing, The Gap Between What We Know And What We Think We Know May Be Widening.

To My Friends, I’m Kind Of Sexually Gay But Ethnically Straight.

I Prefer More To Kind Of Show People Different Things Than Tell Them 'oh, Here's What You Should Believe' And, Over Time, You Can Build Up A Rapport With Your Audience.

Plenty Of Pundits Have Really High Iqs, But They Don't Have Any Discipline In How They Look At The World, And So It Leads To A Lot Of Bullshit, Basically.

If There's A Major Foreign Policy Event, The President Gets On Tv, The Congress Doesn't.

I've Just Always Been A Bit Of A Dork.

A Lot Of The Time Nothing Happens In A Day.

Distinguishing The Signal From The Noise Requires Both Scientific Knowledge And Self-knowledge.

If You Have Reason To Think That Yesterday's Forecast Went Wrong, There Is No Glory In Sticking To It.

The Quest For Certainty In Forecasting Outcomes Can Be The Enemy Of Progress.

Enemy Of Progress

New Ideas Are Sometimes Found In The Most Granular Details Of A Problem Where Few Others Bother To Look.

First Of All, I Think It's Odd That People Who Cover Politics Wouldn't Have Any Political Views.

Almost Everyone's Instinct Is To Be Overconfident And Read Way Too Much Into A Hot Or Cold Streak.

You Get Steely Nerves Playing Poker.

I Actually Buy The Paper Version Of The New York Times Maybe Once Or Twice A Week.

Racism Is Predictable. It's Predicted By Interaction Or Lack Thereof With People Unlike You, People Of Other Races.

When Human Judgment And Big Data Intersect There Are Some Funny Things That Happen

In Baseball You Have Terrific Data And You Can Be A Lot More Creative With It.

I Know It's Cheaper To Fund An Op-ed Columnist Than A Team Of Reporters, But I Think It Confuses The Mission Of What These Great Journalistic Brands Are About.

I Don't Think You Should Limit What You Read.

To The Extent That You Can Find Ways Where You're Making Predictions, There's No Substitute For Testing Yourself On Real-world Situations That You Don't Know The Answer To In Advance.

Testing Yourself

We Are Living Our Lives More Online And You Need To Have Different Ways To Capture That.

The Thing That People Associate With Expertise, Authoritativeness, Kind Of With A Capital 'a,' Don't Correlate Very Well With Who's Actually Good At Making Predictions.

Shakespeare's Plays Often Turn On The Idea Of Fate, As Much Drama Does. What Makes Them So Tragic Is The Gap Between What His Characters Might Like To Accomplish And What Fate Provides Them.

Whenever You Have Dynamic Interactions Between 300 Million People And The American Economy Acting In Really Complex Ways, That Introduces A Degree Of Almost Chaos Theory To The System, In A Literal Sense.

I Don't Play Fantasy Baseball Anymore Now Because It's Too Much Work, And I Feel Like I Have To Hold Myself Up To Such A High Standard. I'm Pretty Serious About My Fantasy Football, Though.

I Guess I Don't Like The People In Politics Very Much, To Be Blunt.

Success Makes You Less Intimidated By Things.

People Attach Too Much Importance To Intangibles Like Heart, Desire And Clutch Hitting.

Not Only Does Political Coverage Often Lose The Signal—it Frequently Accentuates The Noise.

Accountability Doesn't Mean Apologizing.

We're Not That Much Smarter Than We Used To Be, Even Though We Have Much More Information - And That Means The Real Skill Now Is Learning How To Pick Out The Useful Information From All This Noise.

Precise Forecasts Masquerade As Accurate Ones.

Data Scientist Is Just A Sexed Up Word For Statistician.

We Want To Get 80%-85% Of Predictions Right, Not 100%. Or Else We Calibrated Our Estimates In The Wrong Way.

Trending Author

Carrie Brownstein

Robert Anderson

Category