Making decisions about your own medical care is tough. Making decisions about a child’s medical care is tougher still. But making decisions about care for your unborn child? Nothing is harder than that. And these days, there is no end to the decisions that must be made: genetic testing? Birth attendant? With or without medication? Hospital, birthing center or home birth?
These are some of the most agonizing decisions most people will ever make. Especially with a first child, most of the terrain is unfamiliar, opportunities for ghastly mistakes seem to be everywhere, and there’s often surprisingly little solid evidence to go on. Most of us listen to our doctors and ask our friends and family members. We maybe do some Internet research and read a few books, or even a study or two. And then, sooner or later, we make a decision and muddle through as best we can.
Carl Michal isn’t like the rest of us. He’s a physicist who spends his time studying the properties of biological materials like silk and hagfish slime. He’s a father. And he doesn’t seem to have much patience for muddling.
When Michal and his wife were expecting their first child six years ago, his wife wanted to give birth at home, not in a hospital. Michal was somewhat alarmed at what, for him, was an unfamiliar idea. So he did what any self-respecting scientist would do: a literature review. He found a study of 862 home births in British Columbia, where he and his family live. It concluded that home births attended by midwives were just as safe for mother and baby as hospital births. The paper eased Michal’s worries, and he and his wife went on to have their first and second children at home with the aid of a midwife.
Then last summer, while Michal’s wife was pregnant with their third child, he happened to hear about a new home birth study on the news. Distressingly, it claimed that babies born at home were two to three times more likely to die in their first month of life than those born in hospitals. Michal was perplexed: how could this new study have arrived at a conclusion that so contradicted his family’s experience–and his own extensive reading of the evidence? What should he and his wife do about the birth of their third child?
That’s the point at which most people throw up their hands in frustration or fall back on the advice of trusted family members or professionals. Being a physicist, Michal turned instead to a closer scrutiny of the numbers. The paper, which had been written by a group of doctors from Maine, was a meta-analysis, combining results from previous studies to calculate composite measurements of 21 outcomes, including newborn deaths. But Michal found that it was impossible to tell which previous studies had been included in or excluded from each composite result. So he wrote a computer program to try to reverse engineer the authors’ process. His program revealed that there was no combination of individual studies that could have given some of the results reported in the paper.
Michal then turned his attention to the statistical program that the Maine doctors had used to perform some of their analyses, which was posted online as part of a free Internet course. Michal realized that the statistical program was flawed; its authors had made errors in translating its formulas from a textbook into their online spreadsheet. Michal found that the errors could have changed some of the conclusions of the paper.
For Michal, these discrepancies were enough to restore his faith in the previous studies and in his own family’s experience. He wasn’t done though. He wanted other families facing the same decision–and the doctors counseling them–to know that the study was flawed. The publishing journal, the American Journal of Obstetrics and Gynecology, declined his demand that they retract the paper, but an internal investigation found mathematical errors in each of the three results from the paper it scrutinized, including the key finding that newborns are more likely to die after home birth. But the correct results didn’t change the paper’s conclusion that home birth is more dangerous for babies than hospital birth, so the journal isn’t retracting the study.
It may seem remarkable that it took a physicist working in his free time to find flaws with a statistical program used in a medical study. But many peer-reviewers consider it beyond the call of duty to check math and ground-truth procedural details.
One errant study does not invalidate the whole body of medical literature, of course. But especially in politicized areas of medicine, such as home birth, such discrepancies only open the door to more uncertainty. We’re told to avoid looking for medical advice on the Internet–a minefield of quacks and misinformation, and contentious blog posts making vitriolic arguments based on scant facts. But when one flaw in the peer-reviewed literature emerges, we’re left not knowing how many others are out there, undetected. We’re left with one more doubt as we try to arrive at some decision we can live with.
As for Michal, he continues to press the journal to retract the Maine study. Already, he estimates the effort has taken a hundred hours over eight months–time that he could have spent on teaching, research, or with own family. For earlier this month, Michal’s third child, a girl, was born, safely, at home.
**
Image: Wooden birthing forceps used in the 18th Century. Photographed by Dave Russ at the Hunterian Museum and Art Gallery/wikimedia commons.
What an incredible example of diligence. I always thought there should be a place in science for meta-analysis specialists who understood the statistics more deeply and could go around to each field and summarize it. I suppose I’m really describing a statistician, but I don’t see them integrated very well into the research publishing process.
I think it’s a mistake to confound a tale of real science with an anecdote of 3 successful births. But speaking of anedotes I have also recently heard that “at home birth” statistics tend to include children born at home accidentally, that is cases that were already medical emergencies.
Terrifically interesting post, Erika. What this suggests to me is that it’s probably time to overhaul the peer-review process at some journals. I think reviewers should be checking the math and other basics before signing off on a paper. Peer-review is the gold standard of scientific publishing; reviewers can’t afford to slough off.
Good for Michal. And I wish he and his family a wonderful time together with their new arrival!
Fascinating story, Erika. I love the fact that a physicist who studies fish probed a medical meta-analysis to find a flaw in a statistical program. That doesn’t happen everyday.
As an obstetrician for over 30 years, I have unfortunately seen the most routine labors turn in seconds to potentially catastrophic situations for both mother and baby. Without access to an operating room, modern anesthesia techniques and blood replacement, many of these situations would have led to disastrous outcomes.
Anecdotal stories are great Michal but if your wife happens to have had a bad outcome at home because of the unavailability of modern medical technology, statistical analysis means very little.
I applaud the detailed review of statistical methodology and I believe that peer review of complex statistical calculations is important by trained statisticians.
The majority of physician reviewers don’t have the background to analyze statistical calculations and when they themselves write articles, most of them rely on statisticians.
I would be interested to see what literature Michal reviewed to help him make his initial decisions re homebirth. The majority of the peer review literature that I have seen over the years have concluded that homebirth is a risky proposition unless you have a helicopter spooled up and sitting on your front lawn as one celebrity who encouraged homebirth had.
Dr. Michal may want to head back to the library. He seems to have read only a smattering of the homebirth literature and none of the state and national statistics.
First, Dr. Michal quotes the Janssen 2002 British Columbia study but appears to unaware that Janssen was forced to retract her conclusion because her data showed that homebirth had 2 perinatal deaths and hospital birth had none.
Second, Dr. Michal appears to be unaware that there are two types of midwives in the US and that their statistics are very different. Most midwives in the US are certified nurse midwives (CNMs), but most homebirth midwives are certified professional midwives (CPMs). CPMs are grossly undereducated and grossly undertrained. Indeed, they have less education and training than ANY midwives in the industrialized world. They are not eligible for licensure in The Netherlands, the UK, Canada, Australia or any other first world country.
A certified nurse midwife is a master’s level degree. All European, Canadian and Australian midwives have university degrees as well as comprehensive in hospital training. In contrast, the CPM is a post high school certificate, and there is no training in the prevention, diagnosis and management of pregnancy complications. The certificate programs are filled with nonsense courses like homeopathy, flower essences, vibrational healing and crystals.
There has only been one study of CPM attended births in the US and that is the Johnson and Daviss study (BMJ 2005) which claimed to show that homebirth with a CPM is as safe as hospital birth, but actually showed nothing of the kind. That’s because Johnson and Daviss (he is the former Director of Research for the Midwives Alliance of North America) compared homebirth with a CPM in 2000 to hospital birth from … a bunch of out of date papers extending back to 1969. When you compared homebirth with a CPM in 2000 to comparable risk hospital birth in 2000, you find that homebirth with a CPM has nearly triple the mortality rate of hospital birth.
Consider the statistics in the state of Colorado where planned homebirth with a licensed middwife has double the mortality rate of the entire state including high risk, and premature babies.
Most egregious of all in the fact that the organization that represents CPMs, the Midwives Alliance of North America (MANA) has collected a database of 18,000 planned CPM attended homebirths from 2001-2008. Although they repeatedly promised their members that the results would be released to demonstrate the safety of homebirth, they changed their minds. They refuse to release the death rate for those 18,0000 homebirths. The data is available ONLY to those who can prove they will use it for the “advancement of midwifery” and even then they must sign a legal non-disclosure agreement.
It doesn’t take a rocket scientist to suspect that MANA’s own data shows that homebirth increases the risk of neonatal death, and that they are hiding that information.
The Wax study is a red herring. It received a lot of press because it was promoted very heavily, but it was not a good study for a variety of reasons. However, there is copious data from a variety of sources demonstrating that American homebirth increases the risk of neonatal death and an ominous unwillingness of American homebirth midwives to disclose their own mortality rates.
MANA knows exactly how many of those 18,000 babies died at the hands of homebirth midwives. The American public deserves to know, too. That’s a story that ought to be investigated.
But in cases of high risk pregnancy, nobody would recommend home birthing. Home births would therefore include disproportionately more straightforward deliveries. Do these studies account for this form of statistical bias?
Everyone has an agenda, that is exactly what this shows. The scientific theory starts with a hypothesis, a guess, that predisposes the scientist to what he should find. When I first learned that in 4th grade, I found that incredibly odd. I’m not exactly sure what the alternative would be, but to bring a supposition from the beginning sets the stage for the “correct” result.
I’m certainly impressed with the physicist for going so far as to check the original math in a paper and wish more people did the same.
But if the math was wrong, what was the “correct” answer anyway? This article is devoid of that highly important bit of information.
From Nature.com
“On 28 February, Wax’s team posted the requested summary graphs on the journal website. The risk of newborn death and postmaturity among babies born at home is now higher than it was in the original paper, and the risk of prematurity is now lower. The document does not discuss whether or how Wax’s group erred in its original calculations, or what changes were made to produce the new results.”
So the revised numbers show an increase in mortality but this article leaves you with three anecdotes.
I had the exact opposite experience. I found that having a baby at home was much more high risk than having a baby at a hospital. The final clincher to my decision was when a Hassidic rabbi who worked in the temple next door to my work said that his wife (a midwife because their people only gave birth at home) asked him to tell me that having a baby at home was much more dangerous than in a hospital and I should have my baby at a hospital. I’m glad I listened to her because my otherwise very healthy/no complication pregnancy and early labor ended up having last minute serious complications that had I been at home, I may not have had time to be transported to the hospital before myself and my baby died.
Thanks, everyone, for your comments.
Douglas, you’re right – mothers who choose home birth are less likely to have certain risk factors, such as previous pregnancy complications, and the Maine study found this as well.
Actual Scientist: Thanks for reading my Nature story. You’ll have noted that epidemiologists I quoted in that story pointed out significant issues, in addition to the statistical problems, that could have affected the newborn death rate reported in the Maine study. In this piece, I wanted to focus on Michal’s story rather than reiterating all the points explored in my Nature piece.
Excellent writing about a fascinating situation.
To me it’s illustrative of the current problem of data glut in the sciences. OF COURSE nobody’s out there checking the math: there aren’t enough human-hours in the world right now to do it.
It might not be so bad if scientific studies weren’t restricted to journals behind paywalls. As it is now, the chances are slim-to-none that an enterprising mathematician would ever chance upon a paper badly in need of an analyst’s hand. We can only hope for the continued rise of free, public-access journals like PLoS.
Meanwhile, the ivory tower continues to misunderstand the value of internet-based discussion and scoff at science blogging as irrelevant… thus ensuring that the gargantuan task of proper peer review in this era of accelerating technological and scientific progress can NEVER be achieved.
Does it ever occur to some of these people that the “last minute complications” that almost caused them and their babies to die at the hospital might have been CAUSED by being at the hospital? Does it ever occur to the OB/GYNS that some of the horrible complications they see are CAUSED by their own hands? I don’t know the individual stories of people who have posted here, but I’ve heard more than once that “the doctor broke my water at 4 cm and I had [enter resulting cord prolapse/bradycardia problem here] and I had to have an emergency c section.” Or “I hemorrhaged after they pulled my placenta out and gave me lots of pitocin.” Or “My baby was blue at birth (which is normal) so they cut the cord right away and he had to be resuscitated and he almost died.” All of this would not happen at home NEARLY as often as it happens in hospital, and these “crises” are being caused by the very OBs who we are so glad to have there to “save us.”
Putting aside all of the biases reflected both by the physicians and non-physicians who have commented there are a couple of points that have to be made.
1) Being in a hospital is not an innocuous thing. The chances of a severe, antibiotic resistant infection, or medication error or other mishap is much higher in-hospital than at home. Whether this is higher than the risk of an adverse outcome in a home birth in a low-risk pregnancy that could be avoided if you are in the hospital is not at all clear.
2) The reason for #1 is that the statistical analysis in many if not most medical papers is poor. Only a handful of journals routinely have their papers reviewed by a statistician. In my mind the most suspect papers are meta-analyses because they are subject not only to statistical error but to author bias in the selection and weighting of the papers selected to be included. I would bet that if we let Dr. Michal loose on a random sample of the medical literature he would find many more errors.
I agree with Dr. Tutuer that any organization that will not release scientific data has to be viewed with extreme suspicion, but the truth is that this practice is rampant in much of the medical literature, particularly if it is industry-sponsored.
Given all of the above it is only fair to say that all of my children were born in the hospital and my bias is that hospital birth is safer, but I admit that I cannot prove that contention, and I think that anyone who claims that they can is deluding themselves.
nice job erika. a well-told story. it’s unfortunate that the ob/gyn’s who read your post could only perceive that their supremacy was being challenged. when that was not at all the point of the article. proving once again that people see what they are predisposed to see.
two further comments:
despite my disgust at their arrogance, i must admit that the NEJM was one of the first (the first?) medical journal to institute routine analysis of statistical methods.
jessa, do you know about the cochrane library?
I agree with Nik. Hospitals actually CAUSE so many of those “last minute emergencies” with all their interventions. They disrupt the whole natural labor process with all of their policies and procedures and interruptions and interventions, then they swoop in and “Save the day” with the emergency c-section and act like this would have been the case no matter what path had been taken to start. Grrr… don’t get me started.
I had a home birth that ended up being transferred to the hospital. OMG the hospital setting was HORRIFIC for trying to give birth naturally. I was constantly offered medication. I was hooked up to machines unnecessarily. Forced to wear an uncomfortable robe/hospital gown. Sure, normally people don’t care about this kind of stuff in a hospital, but when you’re in labor, everything matters, even the texture of your clothing.
I was given “attitude” from some of the nurses for even TRYING to give birth without meds. Even the doctor requested the monitoring systems be removed so I could labor in peace, but the NURSE reminded him “it’s hospital policy…” We took it off anyway when she left the room, what rebels we were, and luckily I finished up naturally with him mainly because my doctor was SO pro-natural birth and making sure I was comfortable enough to succeed with it. If I hadn’t had him, and had some average OB-GYN schmuck, I am certain I would be yet another sad “emergency C-section” statistic.
Hospitals are for sick people. Pregnant women are not sick. If they are (ie- if they’re high risk) then yes, get them to a hospital. But if they’re normal low risk, keep them AWAY from there. A 30% C-section rate is out of control – they are NOT ALL EMERGENCIES as much as some OB’s would like you to think.
“Hospitals actually CAUSE so many of those “last minute emergencies” with all their interventions.”
There is precisely zero scientific evidence to support that claim.
Great story Erika, Thank you Carl Michal for…
being good at math
following through
& trusting birth…
It is fascinating to read how many points of view this post has elicited.
For the prospective parent seeking information to make a decision about home birth, the answer will not come in the form of anecodes, be they pro or con. The strength of the evidence is very weak.
So what is a parent to do? The scientist/parent gravitates to the literature, and the answer is not there either. I agree with Christine who succinctly states that “to bring a supposition from the beginning sets the stage for the correct result.” That is beautifully illustrated in this post. The obstetricians who reviewed the metanalysis paper were probably predisposed to agree with the conclusion, and the data as presented supported it. End of story.
The parent, Dr. Mishal, was surprised at the conclusion of the paper, and had the dogged determination to solve why it didn’t fit with his prior reading of the literature. What detective work–it was a spreadsheet error! How human.
Should this have been flagged in the review process? Probably not. The reviewer is not expected to recalculate the statistics (unless the statistical methods or conclusions are in question). Should the error be acknowledged and corrected? Absolutely.
P.S. At the risk of keeping the home birth aspect of this post alive, but in response to William’s comment, there are a few evidence-based reviews on the topic in the cochrane library: http://onlinelibrary.wiley.com/o/cochrane/mainSearch?products=all&Query5=&Query4=&FromYear=&Query3=&Query2=&Query1=home+birth&ToYear=&mode=startsearch&zones5=tables&zones4=abstract&zones3=author&zones2=article-title&zones1=%28article-title%2Cabstract%2Ckeywords%29&opt4=AND&opt3=AND&opt2=AND&unitstatus=none&opt1=OR&
I have a few comments in response to Amy Tuteur’s post above.
While I respect everyone’s right to express opinions, we should all be clear on what is opinion and what is fact, and make every effort not to scramble them.
Dr. Tuteur states as fact that ‘Janssen was forced to retract her conclusion,’ but this claim is unsubstantiated.
That paper is listed on Pubmed here:
http://www.ncbi.nlm.nih.gov/pubmed/11868639 and while it clearly generated many comments, no retraction or correction notice is apparent. The National Library of Medicine does post such notices on Pubmed, as described here:
http://www.nlm.nih.gov/pubs/factsheets/errata.html
If the paper had been corrected or retracted, it would be noted.
Dr. Tuteur also appears to have misquoted the deaths from that paper, as anyone is free to check – the full text of the paper is available to all from the Pubmed link. The study looked at three groups: planned home births (attended by midwives), planned hospital births attended by midwives, and planned hospital births attended by physicians. There were three perinatal deaths in the first group, none in the second, and one in the third. Details of all three perinatal deaths in the home birth group are provided, and those interested can read for themselves to form opinions on whether those deaths might have turned out differently had the births been planned to take place in hospitals rather than homes.
I do not know what the data MANA has collected shows, but if Dr. Tuteur’s assessment of that data (which she claims not to have seen!) is as accurate as her characterization of the Janssen 2002 paper, it may well show the opposite of what she states.
In response to “An Actual Scientist,” we did not calculate the “correct” answer for Wax’s study because it just doesn’t make sense to combine these studies in the way Wax et al. did. The result would not be relevant to anyone because the conditions of practice for home-birth care providers varies so much between the different studies.
A complete description of our analysis of Wax’s paper can be found in an article just published on Medscape:
http://www.medscape.com/viewarticle/739987
(free registration required) An enormous number of errors, not just problems with the spreadsheet, are described. We show that even for the three results that Wax et al. “corrected,” they still have the wrong numbers.