How Reliable is Health Advice?

Via the Health Blog:

Just 11% of more than 2,700 established heart recommendations are backed by high-quality testing, says a study in the current issue of JAMA.

The American College of Cardiology and American Heart Association jointly issue guidelines to doctors, WSJ explains. The ones thought to have the highest level of evidence are based on multiple randomized clinical trials. Those considered weakest are backed by expert opinion or case studies.

According to the JAMA paper there are three type of sources:
1. Sources based on multiple randomized clinical trials (high quality testing)
2. Sources based on a single randomized clinical trial or observational study (moderate)
3. Sources based on case studies, expert opinion - etc (poor)

As an epidemiologist, I will probably spend most of my time dealing with observational studies in the moderate category. While not "high quality", these are still important since many randomized control trials can not be conducted because of ethical, feasibility, and financial reasons. For example, we can't ethically conduct a randomized control trial on whether smoking causes lung cancer - since there is substantial evidence that we would inflecting harm in our control arm.

However, randomized clinical trials (RCTs) should be used whenever possible because they are removing many biases and confounding from the relationship of interest. A classic case of the benefit of RCTs is Hormone Replacement Theory (HRT) in women. The Nurses Health Study and others found a protective effect on mortality for HRT in several of their observational studies. Their evidence looked so convincing that hundreds of thousands of women went on the therapy. That was until randomized clinical trial results found that it can increase breast cancer and disease risk.

Getting back to the JAMA article, it's pretty scary when you think about. Heart Disease is America's biggest killer, and quite possibly the most widely studied disease. What this says is that there is a lot of work to do in the health field and that there will be a lot of reversals, ala HRT, of doctor opinion in future recommendations.

On a positive to note, I'm happy to say that we're headed in the right direction with this $1.1 billion dollar allocation to compartive effectiveness research in the stimulus package. Maybe Billy Beane had Mr. Obama's ear?