addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupsimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1outlookpersonJoin Group on CardStartprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

Confidence Calibration Exercises

  • Nov 13, 2012 · 7:00 PM
  • Zvi and Laura's Apartment

People tend to be ludicrously overconfident about things. Let's fix this!

From the wikipedia article on the overconfidence bias:

"If human confidence had perfect calibration, judgments with 100% confidence would be correct 100% of the time, 90% confidence correct 90% of the time, and so on for the other levels of confidence. By contrast, the key finding is that confidence exceeds accuracy so long as the subject is answering hard questions about an unfamiliar topic. For example, in a spelling task, subjects were correct about 80% of the time when they were "100% certain." Put another way, the error rate was 20% when subjects expected it to be 0%. In a series where subjects made true-or-false responses to general knowledge statements, they were overconfident at all levels. When they were 100% certain of their answer to a question, they were wrong 20% of the time."

Calibrating your confidence is an important rationality skill - a building block for many other skills you may want to develop. This Tuesday, we'll be doing a series of exercises. A moderator will ask trivia questions that have a numerical answer. Participants will write down a 50% and 98% confidence internal (i.e. a numerical range that you are 50% confident the answer lies within). Over the course of the evening we'll see how often our confidence was calibrated correctly, and hopefully adjust accordingly.

Join or login to comment.

  • Raymond A.

    A good experience - a lot of the questions were fun to try and answer, and it was very informative to see how overconfident many of us were.

    November 14, 2012

11 went

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy