Aumann’s Agreement Theorem says two perfect rationalists can’t “agree to disagree”. Therefore, when two people disagree, a good question is one that makes either the asker or askee change their minds. Some examples of bad questions are:

– Why don’t you agree that abortion kills innocent babies?
– Why don’t you support welfare programs that help children with cancer?
– Isn’t it obvious that Politician Bob is corrupt?
– Do you deny supporting the poaching of baby seals?

These aren’t really questions. They’re more like attacks, with a question mark tacked on at the end. Instead, a good question tries to roll back a chain of inferences. A person might support E because they support A, and they also believe in the chain of arguments A (therefore) B (therefore) C (therefore) D (therefore) E. A good question tries to bring the debate about E back to a debate about D, and ultimately all the way back to A.

Some thoughts on how to ask better questions:

– Try to be quantitative. For example, someone might say “I think idea X is too expensive.” So you might reply, “About how much do you think X will cost?”. Sometimes this is a bit more difficult, like if your co-worker said “I think Bob would be a terrible person to hire”. But you can still be semi-quantitative; eg., “On a scale from 1 to 10, where 1 is the worst candidate and 10 is the best, where do you think Bob is?”.

– Ask for examples. Your friend might say, “Policy X has always been a disaster.” So a good next question might be, “Can you talk about some times when people tried X, and it turned out really badly?”. A lot of words are vague enough that two people will hear them, and imagine totally different things in their heads, often without realizing it. So examples can help clear up semantic misunderstandings.

– Talk about probabilities. Eg., a lot of people will say that X is a serious threat, for various different values of X. But sometimes X is very unlikely to happen; the most famous examples are media sensations like terrorist bombings, shark attacks, and stranger abductions. So a good question might be, “If I did X, about how likely do you think it is that <bad thing Y> would happen?”.

– Investigate where ideas come from. Even when people are wrong, it’s rarely because they made something up out of their heads. Much more often, they’ll hear something from Alice, who heard it from Bob, who heard it from Carol, and so on, and the original (correct) idea got lost in a long game of “telephone”. (The Science News Cycle shows this process in action.) So if you can find the original source for an idea, you might both wind up agreeing with it.

– Ask what a supporter thinks about an idea’s downsides. Sometimes, they might disagree that a downside exists; sometimes, they might think a downside exists, but that it’s very small; and sometimes, they might think the downsides are large, but the benefits are so big that it’s worth it. So if eg. someone supports using a new chemical in agriculture, you might ask “How dangerous do you think chemical X is?”. (Don’t let this become rhetorical, though. A question like “But won’t idea X kill millions of puppies?” is back in attack territory.)

– Find comparisons to other examples. A person who really liked X might say things like “X is the best Y ever”. So you might ask, “what are some other Ys, and what makes X better than them?”. Luke Muehlhauser’s post on textbooks used this technique very successfully – people who liked a book also had to name two other books they thought were worse. Otherwise, people might recommend something just because it was the only book they’d read in the field.