Skip to content

Problems with the Scientific Method….

December 31, 2010

Here is an article that appeared in the New Yorker.

Interesting Quote:

“For many scientists, the effect is especially troubling because of what it exposes about the scientific process. If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved? Which results should we believe? Francis Bacon, the early-modern philosopher and pioneer of the scientific method, once declared that experiments were essential, because they allowed us to “put nature to the question.” But it appears that nature often gives us different answers.”
What say all of you?


10 Comments leave one →
  1. December 31, 2010 2:44 pm

    It is no surprise that the “treatment effect” in the general population is less than what might be found in a randomized clinical trial where all participants are carefully screened. This isn’t squishiness of any sort, just the laws of probability in action. It does show some of the difficulties involved in trying to predict the actual strength of a finding.

    Without going into a lesson on statistics and the value of randomized trials and replication, the best non-technical description of this process is “The March of Science” (and I would link this if only I could recall the name …). Science “marches” towards new ideas, but the steps are uneven. Sometimes there is a big step forward, but more often there are many small steps leading the way. Sometimes the steps go backwards, or off in a different direction, and a once promising idea might be abandoned entirely. It is a wobbly march, but that is how progress is made.

  2. December 31, 2010 3:07 pm

    Actually, I think they’re right, particularly in the realm of medical research. Many clinical studies nowadays turn out to be “pseudoscience,” as the article put it. Medical research declares something is bad for you, or that a new drug is a breakthrough, only to find out ten years later that they were gravely mistaken. Yeah, no thanks.

  3. December 31, 2010 4:30 pm

    Certainly there are problems – I’ve seen a few of them firsthand – but the fact remains that the randomized clinical trial is the gold standard of medical research. In most cases there simply is no better alternative. This article also ignores any example of findings that *have* been validated through multiple replications. I don’t think it is so much a matter of “decline” as it is process of weeding-out of results that were wrong or ill-defined in the first place.

    All new knowledge requires risk; There is no advancement without the possibility of error. To change our minds on anything, ever, is to risk the possibility that decision may be wrong. To never change our minds at all is even worse, for that assumes we already know all there is to know.

    Alternative medicine might be a good example (if a bit of a straw horse). Would you rather trust the result of a clinical trial that says a treatment works, or the unfounded claims of alternative medicine that a treatment works – especially when there is a clinical trial showing the alternative treatment *does* *not* work?

    Related and amusing: Sometimes there is a case to be made that clinical trials might be overkill.

  4. Scott permalink
    December 31, 2010 4:52 pm

    What a welcome respite! When our rigid absoluteness has softer edges it seems we are more a little more human(e).

  5. Chris permalink
    December 31, 2010 5:45 pm

    That’s a confusing statement. Why isn’t replication a part of psuedoscience? The fact is, there are plenty of flukes and chance occurences, but replication is what we use to determine the laws of science. It’s a matter of determining what can and will happen with reliable consistency. If replication isn’t used in psuedosciences, and if any chance occurences are accepted as fact, then it’s no wonder so many fringe sciences have not been accepted.

  6. Matt permalink
    December 31, 2010 7:15 pm

    You know what they say… you can take the science out of human failings, but you can’t take human failings out of science. Or at least, that’s what they should say. At the very least, this should be a caution against holding up scientific findings as some golden, infallible standard.

  7. December 31, 2010 8:04 pm

    Though human failings certainly contribute, the “decline” effect can also occur simply as a matter of chance, with no human error or malfeasance. Each finding needs to be judged on the strength of the evidence. The good ones persist through multiple replications and become very very close to golden, perhaps failing only because the open the way to further knowledge (Newton’s Law of Gravity?). Bad findings get replaced over time with better ones. New findings, especially those making big claims on the basis of small evidence, are the ones to view most skeptically.

  8. Knick Gaughts permalink
    January 7, 2011 1:13 am

    I came here on January 6, as directed, looking for the brand new Thomas Society. Specifically, I was looking for some dynamite insight into \”Culture.\” Alas, the relaunch has apparently been postponed.

    More importantly, consider this an open letter to KG, who I see has reappeared in an earlier thread on this forum. KG, this site needs you. You were the best thing about this place. You were the pin in Jon\’s balloon, the thorn in his side, the Jerry to his bumbling, fumbling Tom. Where is the KG that dubbed Jon \”THH\”? Where is the KG that crushed people for their ridiculous dissertations on magic? We need that KG back. On a site devoted to make-believe, we need a voice of reason.

    I realize that you most likely left because Cruz chose to censor you rather than suffer the humiliation of losing an argument. I fear, however, that my trolling may have contributed to your departure from the discussions on this board. As a plea for your reappearance, this will be the last post by Knick Gaughts. Please. Come back, KG.

    Otherwise, this site will not be interesting enough for Jon to garner enough hits to get any publishers interested in his \”Pirate Atheist\” book. Well, perhap$ Brio.

  9. Jonathan Weyer permalink*
    January 9, 2011 2:45 pm

    Knick Gaughts,

    Don’t you have anything better to do? Just wondering….

  10. January 18, 2011 6:04 pm

    Knick Gaughts,

    As you can see by the posts that were censored. KG was removed due to ‘lack of philosophical content’ in philosophical discussions. But have fun and enjoy the world of trolling.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: