I come from a generation which listened to, but didn't necessarily do, what doctors prescribed. When my urologist suggested I start taking a drug called Proscar, because I might be having symptoms of benign prostate hyperplasia (BPH), I did what a physicist knows to do. I logged on to my work station to search the medical journals for what was known about Proscar. I am not a physician but have a half-century of experience analyzing data, which routinely uses statistical methods. More important, physicists are sensitive to the role of ``systematic'' errors. 48 abstracts turned up on my computer screen and I could readily select the few papers that described the clinical studies of Proscar. I especially studied the summary of the data published as a result of the Merck ``double-blind study''. (1)
What I read was unbelieveable.
First, the relative improvements in the test scores of the two groups were negligible. Second, the results showed wonderful evidence that the ``placebo effect'' was in full force, namely, the more serious the symptoms, the greater the improvement for both groups. Since the difference, after correcting for the placebo effect, was insignificant, I could not resist writing in the Journal of Neurourolgy and Urodynamics that the ``placebo was almost as good as the drug, cost nothing, did not need a doctor's prescription, and had no side effects''.
But that was only part of the story. The experimenters studied the effect on prostate size of 1 and 5 milligram doses of the drug. The shrinkage was observed to be the same. Yet those receiving the low dose appeared to show less symptom improvement and less increase in flow rate than those receiving the larger dose. The reported statistical errors supported the difference. Since it was the shrinkage in prostate size that was supposed to cause the improvement, what had happened to cause and effect? The authors made no comment on this evidence that non-statistical errors must be present.
After the first year the 5 mg group which showed the largest improvement in scores was retained and the double blind study halted.(2) The group that showed the same decrease in prostate size but did not show the larger improvements in scores were dropped. What systematic error entered here? About half the participants with 5 mg doses in the first 12 months dropped out. Why? Is this a biased sample?
Sudden improvement now appeared in the North American study of symptom score but not in the International Study. Sudden improvement appeared in the International Study of urinary flow but not in the North American study.
Even more significant was non-reporting of what is believed to be the answer to the question that patient and physician would want to know: Were improvements in flow and score correlated? Did a patient find improvement in both the symptom score and flow rate, or did they have nothing to do with each other. And what about the reported side effects? Were they independent? For example, if one become impotent, would one also grow breasts?
So I called Merck. They had the data. They had not published the correlations. They said that there were none! They would not let me see the data, calling it ``raw'', and saying it was ``proprietary''. Of course if the basic data were raw, how could they publish it? All that one would need for the study would be a simple result for each patient, namely, the change in the three numbers: flow rate, symptom score, and prostate size.
I decided I ought to write up my literature study of Proscar for my urologist friends. They urged that I publish my findings. Naturally I submitted it to the New England Journal of Medicine, which had reviewed, accepted, and published the original papers. The article was first rejected by the editor, Dr. R. Utiger, who wrote that determining the correlations from the data was ``in decision analysis not appropriate''. I gave his letter and my paper for study to the Chair of the Statistics Department at my university, who replied, saying that there were nothing incorrect with my analysis. I resubmitted the paper to the NEJM along with that letter. Now, another NEJM referee made the statement: ``By the current standards (italics mine) of reporting of therapeutic studies, failure to report such evidence does not represent a deficiency of reported studies. Rather, this represents an unexplored aspect of the data whose utility and informativeness must be demonstrated.'' To suggest, at the close of the 20th century, that the value of reporting correlations in data variables must be demonstrated, is incredible.
And finally the coup de grace by editor Utiger: ``Had you submitted some of this material as a letter to the editor when the article was published we might well have thought it worth publishing''. Apparently one cannot take bad science out of NEJM unless it is done immediately.
My article was published in ``Neurourology and Urodynamics''. I thought I would go peacefully back to my physics research when a letter of protest by a Merck proponent was sent to the journal accusing me of bias because I had only studied Proscar and not Hytrin. Since I did not know at the time I studied Proscar that there was even a drug called Hytrin, this seemed bizarre. But I soon found out what was going on when newspapers and television aired a battle in progress between Merck and Abbott because of the anticipated publication of a new double blind study by Dr. Lepor of New York University which was to compare Proscar and Hytrin. The rumor was that this study came to the same conclusion, which I had found simply by studying the literature.
(In this connection I remembered the famous letter that Voltaire sent to the explorer Maupenius. After Isaac Newton published his work on the force of gravity, Maupenius sailed to Peru and other places on the equator to test that the gravitational force was weaker at the equatorial bulge. Voltaire wrote: ``You have confirmed in these tedious places what Newton found out without leaving his room.'')
Delighted that the study supported my conclusions I sent Lepor a copy of my paper and suggested that, since he had the new data, he should publish the correlations he had found. I especially asked that he publish the correlations in side effects, since the published Hytrin data, which appears in the information sheet accompanying a bottle of Hytrin pills, showed that there were more side-effects with Hytrin than with Proscar. The chance of getting a headache with Hytrin was 16.2% and for getting dizzy was 19% (The new Lepor results on Hytrin quote 25.9% for dizziness.) It seemed to me that the patient and physician would want to know if a patient got headaches would the patient also get dizzy?
After receiving my letter, Lepor did not answer my fax or phone calls. He did not send a preprint. Thus I was not surprised to find that Lepor's paper in the New England Journal of Medicine did not reference my publication, that competent referees would not have required it, and that Dr. Utiger would refuse to print a notice of the omission when the lack was called to his attention. (My study of side effects and their correlations is now in print.(3))
Apparently in these urology studies the level of science and its reporting
and also the refereeing and editing in NEJM is below the level found in
the physical sciences. It need not be.