Skip to content Skip to navigation

The Inadequacies of the Evidence

March 1, 2007
| Reprints
Flaws in research methodology may result in overselling of certain treatments for addiction

It is with growing dismay that we review much of the recent literature in the field of alcohol and other sedative dependence. Our current literature appears to suffer from either a lack of understanding as to what constitutes this addictive disease or from a perceived need to investigate issues that are irrelevant to the development of improved treatment.

This article will address four primary fallacies that are present in many recent studies.


For a study to properly reflect the potential value of treatment for a specific disease, the inclusion criteria must reflect the currently accepted definition of that disease. There may be numerous exclusions such that comorbidities do not interfere with the study, but one cannot reduce the disease population by adding new criteria not previously accepted as part of the disease definition.

A variety of published definitions of sedative dependence exist. They each include reference to the patient using a substance despite negative effects upon multiple aspects of his or her life.1,2 At no time in any of the accepted definitions are quantity or frequency of use indicated as being essential for diagnosis. Substance use itself does not define the illness. Quantity and frequency are not germane to the diagnosis, and brief or extended alteration of such parameters has not been demonstrated as having long-term impact upon morbidity or mortality.

One recent study required that participants not only be alcoholic, but also drink heavily and frequently.3 Another study required that patients be alcoholic, have abnormal GGT (gamma-glutamyl transpeptidase) and MCV (mean corpuscular volume), and have at least five days of abstinence.4 These studies investigated inadequately characterized subsets of alcoholics. It is difficult to state with validity and conviction that any such research is applicable to the broad population of patients with alcoholism.

Outcome measures

In order for disease improvement to be detected, one must measure relevant aspects of the disease. In current literature, the outcome measures under examination either have no demonstrated bearing upon long-term morbidity/mortality or, worse, are irrelevant to the fundamental nature of the disease under study. If we show, for example, that cough has decreased with methadone in the treatment of tuberculosis, we cannot state that methadone is a successful treatment for tuberculosis itself.

Reducing the frequency or quantity of the marker (the substance use itself) would be of value only if it could be demonstrated that such reduction leads to a significant long-term beneficial outcome. Some studies have focused on “percent days abstinent”3 or “number of days until next heavy use”3 or the “event rate of heavy drinking days”5 and then demonstrate an improvement in these parameters based upon a treatment mechanism. These studies are looking in the wrong place, or at least a place still not identified as being of value. These parameters are unrelated to the presence of disease, are not included within the definition of the disease, and are not as yet demonstrated to have significant impact upon long-term morbidity or mortality or upon eventual recovery rate.

Even more of a concern, if patients feel that a reduction in use is an acceptable goal, might these types of treatment simply foster denial, delay achievement of abstinence, and potentially increase long-term morbidity and mortality? Also, prescribing medication for alcohol dependence to patients who drink “too much” but are not necessarily alcoholic may bring stigma to such patients in the eyes of third parties who are aware of such prescriptions.

Before studies such as these, who would have thought that drinking less alcohol for three months was a significantly positive outcome to a person whose life has become unmanageable and on the verge of ruination from drinking? These studies make it seem as if decreasing alcohol intake each day is equivalent to reducing cholesterol or blood pressure, signs of illnesses in which stepwise improvement has been of demonstrated value. What makes alcohol intake worth the risk? Quantity of intake is measurable and decreasing such quantity has been theorized as cost-saving and quality-improving, but again studies have not demonstrated this with respect to long-term outcomes for patients.

One might hypothesize that since lowering average glucose levels in diabetes leads to improved morbidity/mortality, a decrease in substance use also would lead to such improvement within sedative dependence. This hypothesis had been promoted by some cigarette manufacturers in tobacco trials and clean air hearings, but safe cigarette use has become oxymoronic. Drinking less or controlled drinking or outcomes other than abstinence have been proposed in the counseling and psychology literature. As in diabetes studies, the benefit of such an approach would need to be demonstrated over an extended period with objective measures of substance intake incorporated into the study. Random assignment studies of at least five years duration would be preferred and such a five-year approach, with random objectively confirmed outcomes, should be the standard within our field. Otherwise, we will end up with more situations akin to what happened between the 1970s and 1980s.6,7