Peer disagreement: Why it matters and what the proper response to it is.

Murali

Murali did his undergraduate degree in molecular biology with a minor in biophysics from the National University of Singapore (NUS). He then changed direction and did his Masters in Philosophy also at NUS. Now, he is currently pursuing a PhD in Philosophy at the University of Warwick.

Related Post Roulette

24 Responses

  1. I just want to come out from the woodwork to say I greatly enjoyed this post. The whole time I was reading I kept thinking of the one wise judge vs. twelve idiots question, but then towards the end, I found myself thinking about how this framework does and doesn’t apply to science. In a certain sense, the scientific method is a standard that aims to get around the problems raised here – there’s also the fact that science is an empirical pursuit, so we can see afterwards who’s right and who’s not (so far we’re pretty sure quantum theory is on the money, as cracked out as it is, because every experiment ever designed to disprove it has failed.) We can’t do this for ordinary experience, however, since we aren’t talking about ideal cases.Report

    • You’re going to get a bit of this in science as well. What does the result of a particular experiment really mean vis-a vis a particular hypothesis/theory? Also, every once in a while, even the greats among us slip up. Einstein famously thought that quantum mechanics couldnt possibly be true. What are we to do about such situations?Report

      • You can certainly claim the the scientific method is flawed or that it allows for flawed thinking or experiment design to get past whatever standard you set up, but it is a standard that exists outside subjectivity, that consciously eliminates the problem of subjectivity, so that whatever systematic problems do arise are the problems of the system and not the problems of its individual agents (at least theoretically speaking. This whole thing breaks down when we’re talking about “sciency” things like neo-social Darwinism and evopsych.).Report

        • You’re mistaking me. My claim is this:

          The line of reasoning from evidence (e.g. the result of an experiment) to conclusion is not straightforward. Not everyone can draw those conclusions. Similarly, even experiments require some interpretation. This is precisely why the scientific enterpise is and ought to be a social enterprise. The scientific enterprise is one that is conducted by specially trained communite who check each other and thereby reduce error. No single scientist can do it all on his own. Peers review eachother’s work to mak sure that such work adheres to accepted standards. Peers look at the evidence that individual scientists draw upon to reach their conclusions and ask themselves whether those conclusions are warranted. It is this social factor in scientific investigation which is often overlooked. But the social factor is key to the success of the scientif enterprise. I’m willing to bet that absent any kind of peer review, scientists would be drawing all sorts of unwarranted conclusions from their experiments and our state of scientific knowledge (to use rather imprecise terminology) would be much more backward and messy.Report

  2. Will H. says:

    I really like this post as well.

    I would take issue the statement that Einstein believed quantum mechanics to be untrue.
    My understanding is that he took issue with the concept of randomness, and its liberal (non-Rawlsian) application.
    And I see his point. True randomness would be difficult to prove. Some manner of pattern may yet be ascertained.Report

  3. BlaiseP says:

    Epistemic disagreement ought to lead the parties to propose better experiments to solve those disagreements.   See Bell’s InequalityReport

  4. This was an excellent piece, though my poor, Internet-addled brain required re-reading a few key paragraphs before I got what you were saying.  Thanks for writing it,

     Report

  5. Rose says:

    But what about the legitimacy of saying “I agree to disagree?” which is what started this conversation? Kelly’s view is not, as you say, a practical guide. As a practical matter, when peers disagree, one of us is right and one of us should be weighting the evidence differently. But I can’t always be sure it’s me (assuming an epistemic peer). Also, what if I believe my evidence is indeed weak? Then it seems legitimate to say to a disagreeing interlocuter, whom I believe to be using sound reasoning but perhaps slightly weaker evidence, that I am not willing to try to persuade her that I am right and she is wrong.

    Looking at abortion, which is what brought his up: I think all the personhood arguments for any particular stage of embryonic development are too weak to endorse (conception, heartbeat, viability, sentience, etc.). My own position is that there are no necessary and sufficient conditions for personhood, but that’s based on a relatively tentative and weak philosophical position of mine that we shouldn’t always be trying to muck around with necessary and sufficient conditions. So I know personhood when I see it, and I think I see it some time after conception, but pretty early on. How strong is my evidence for that? Not very! I think it’s warranted, but barely, and I want to respect the fact that there are people who think that there are indeed necessary and sufficient conditions for personhood.Report

    • Pierre Corneille in reply to Rose says:

      My own position is that there are no necessary and sufficient conditions for personhood, but that’s based on a relatively tentative and weak philosophical position of mine that we shouldn’t always be trying to muck around with necessary and sufficient conditions. So I know personhood when I see it, and I think I see it some time after conception, but pretty early on. How strong is my evidence for that? Not very! I think it’s warranted, but barely, and I want to respect the fact that there are people who think that there are indeed necessary and sufficient conditions for personhood.

      +1Report

      • Stillwater in reply to Pierre Corneille says:

        I think it’s warranted, but barely, and I want to respect the fact that there are people who think that there are indeed necessary and sufficient conditions for personhood.

        I’m not sure this is a case of agreeing to disagree, since you’re not agreeing that their view is true, just that it might be true. It seems to me that you’d be less inclined to ‘agree to disagree’ with someone who holds a robust view that since personhood begins at conception, abortion is murder, than you would someone who is only slightly further to the pro-life side than you are. Additionally, you might be inclined to agree to disagree with them, while they wouldn’t be inclined to offer you the same concession.

        Most of the disagreement, it seems to me, is based on the priors. If you discount priors and assume a restricted set of evidence E, then both of you should agree in your confidence of P. It’s the priors which cause the disagreement. And my guess is you disagree about those.

        Which gets to some of my worries about the framework Murali has outlined here: how do you discount priors from creeping into the evidence set? And if we can’t, then do we have to include priors as a necessary part of the evidence set? And if so, then doesn’t the thesis amount to the claim that any two people with the same evidence (and priors) and of the same intellectual abilities will necessarily ascribe the same confidence level to proposition P?

        But how is that interesting?Report

        • Murali in reply to Stillwater says:

          And if so, then doesn’t the thesis amount to the claim that any two people with the same evidence (and priors) and of the same intellectual abilities will necessarily ascribe the same confidence level to proposition P?

          Let’s look at the abortion issue. Are the priors that I have the same that you have?No? If not, can those priors be stated in such a way as that I am aware of them. i.e. is the prior just some indefinable something or does it actually count as a premise in your argument. Presumably, if our different conclusions boil down to different priors, then resolving the conclusion would require us to resolve the difference in priors. Are our priors the kinds of things that can be critically evaluated? If yes, can we critically evaluate whose priors are better? If priors are the kinds of things we cannot critically evaluate, how is it that we continue to believe in those things? If they cannot be evaluated, shouldnt we both be reserving judgement?Report

          • Stillwater in reply to Murali says:

            Murali, ahh good. I think I misunderstood what you wrote in the OP. Priors will certainly play a fundamental role in how we judge any other evidence set E, or if we even accept it. So even tho we can restrict E to a limited number of beliefs in a particular context, for practical purposes E will be the total set of (relevant) beliefs which contribute to our confidence in P. I also like your statement about priors: either they can be evaluated and subject to argument or they can’t be, in which case it’s rational to reject them.Report

        • Rose Woodhouse in reply to Stillwater says:

          FWIW, I think the position that all abortion is murder is a defensible one. But I do agree that most on both sides of the issue do not think there is any way to hold the other position defensibly!Report

          • Stillwater in reply to Rose Woodhouse says:

            I’m not so sure about that. On what grounds would you say that you agree with them, in what sense is their belief ‘defensible’ given your evaluation of the evidence? One way to make sense of it is to say that you agree with the justification for their belief, that is, that you agree with the argument for the claim that abortion is murder. If so, then it seems that you hold two contradictory beliefs: both that abortion is always murder and that abortion sometimes isn’t murder. Or at a minimum, that you believe the two claims – abortion is always murder and abortion sometimes isn’t murder – are both equally justified. But by hypothesis, you don’t think that: you hold the claim that abortion sometimes isn’t murder with a higher level of confidence than it’s alternative.

            It’s at this point that Murali’s main thesis comes into play. If both you and another person share the same intellectual abilities and the same evidence set E, and if the two of you disagree in the confidence of P (abortion is always murder), then the difference must be accounted for in some way. And I think the difference is accounted for by considering the antecedently held beliefs by which both of you evaluate and ascribe confidence levels to specific claims in E. If so, then you’re not necessarily agreeing to disagree. You’re saying that given the other person’s priors, his confidence level in P is justified.

            But do you agree with the priors? I think this is where it gets sticky. Murali’s comment above suggests that priors are subject to evaluation and argument just like any other beliefs. And I agree with that. So in my view, it’s not correct to say that two people agree to disagree about whether abortion is/is not  murder: what we’re in fact saying is that while I disagree with you, I agree to concede (usually for pragmatic purposes) the foundational premises which justify your view.

            Or in other words, if two people who ostensibly agree to disagree about P were to identify and evaluate their priors, there would be movement towards agreement in the confidence level of P. This rarely happens, of course, since most people either aren’t consciously aware of their priors, or hold those priors so closely that they won’t surrender them in any event. In both cases, tho, I think that means the ascribed confidence level in P isn’t warranted (or defensible), and the belief in P is to some degree irrational.

             Report

  6. Rose Woodhouse says:

    Also, Murali, I want to thank you for this post. I know shamefully little about epistemology – just enough to teach intro, or whenever it’s related to my area, or related stuff I’ve picked up by osmosis from my husband who has an AOC in phil sci. I’ve been meaning to beef up, like, forever.

    Are you a philosopher (professionally, that is)?

    And speaking of epistemic peers and disagreement, this cracked me up: http://fauxphilnews.wordpress.com/2012/02/22/psychologists-search-the-philosophical-mind-for-bullshit-detector-find-friendship-deterrence-system-instead/Report

  7. Rose Woodhouse says:

    Good luck with the PhD program! I remember what an anxious time that was.

    A friend of mine just did a dissertation defending reflective equilibrium in moral cases. And (speaking of peer disagreement), I agree with my friend – I think some reflective equilibrium is absolutely crucial in coming up with an ethical system. Just doesn’t make sense to me otherwise (in that, without reflective equilibrium, you can come up with a perfectly coherent moral system aimed at maximizing the amount of grape jelly in the world). Have you posted on the topic?Report

    • Murali in reply to Rose Woodhouse says:

      I actually disagree with you on that. I dont think reflective equilibrium gets us anywhere when it comes to moral theory (because we don’t really have any reason to think the starting points are any good in the first place).

      I do however think that I can start with  some self evident premises and then develop a moral theory from there.

      One premise is that ought implies can.

      Another premise is that at least the most fundamental principles are valid for all possible persons.

      Even though this is not fully worked out yet, the rough idea is that we cannot permissibly pursue a particular end E using a particular means M if pursuing E via M would be counterproductive if everyone who cared about E pursued it via M. This might need to be tweaked a bit here and there, and I’ll need to be extra careful, but this seems viable. (It is also quite reminiscent of Kant’s formula of Universal law)Report

  8. Rose Woodhouse says:

    But how do you determine that as the most plausible version of a basic moral principle instead of (say) some version of the mere means principle without some kind of reflective equilibrium?

    Also, how do you cash out “counterproductive” without something at least similar to reflective equilibrium?Report

    • Murali in reply to Rose Woodhouse says:

      But how do you determine that as the most plausible version of a basic moral principle instead of (say) some version of the mere means principle without some kind of reflective equilibrium

      Presumably, the principle can be deduced from the initial 2 premises. On finding that the conclusions are counterintuitive, I will not go back and tweak my premises so that they produce more “plausible” principles.

      Also, how do you cash out “counterproductive” without something at least similar to reflective equilibrium?

      I’m not sure what you mean.When I say an action is counterproductive, all I mean is that some action fails to bring the actor closer the end for which it was performed (or may even take the actor further away from it). I am not sure how reflective equilibrium would figure into it? Do you mean something very different from what I mean when I talk about reflective equillibrium.

      By reflective equilibrium, I refer to a process by which I tweak considered judgements and high level principles to fit with low level more fundamentl principles and vice versa. The end result is something which would tend to be fairly commonsensical. The presumed starting point is some set of pre-theoretical considered judgements about cases. Principles are constructed to fit these considered judgements. i.e application of these principles will reproduce those judgments fairly closely. Judgements which would not fit are discarded. Some more mutual adjustment takes place until the set of principles and judgements are in equilibrium.

      By contrast, I move in only 1 direction. My argument is roughly that if my premises are necessarily true, then any conclusions I properly deduce from the premises are necessarily true as well.Report

      • Rose Woodhouse in reply to Murali says:

        I was thinking you’d have to see if things were counterproductive really, but maybe that’s not the case.

        But I still think that maximization of grape jelly could follow from your two premises. Or at least, act-utilitarianism or something like the mere means principle.Report