What Did DSM-III Solve? The Rosenhan Experiment and the Reliability of Psychiatric Diagnosis, 1955–1980

This is Part 7, the final part, of the series A History of Psychiatry. Part 6 followed Japan as psychiatric beds kept multiplying and confinement moved from the family home to the hospital. In the United States at the same time, the opposite was happening. Patients left the big state hospitals, and an in-patient population of more than half a million shrank to a fifth in thirty years. Then, in 1973, the journal Science published a paper claiming to test whether sane people admitted to psychiatric hospitals would be found out. Would a psychiatric diagnosis come out the same if a different doctor made it? The American Psychiatric Association's answer to that question was its 1980 diagnostic manual, DSM-III.

This series is a history of psychiatry, not advice on diagnosis or treatment. If you are struggling with your mental or physical health, please contact a medical service or a support line. The article describes violence against patients in hospital. It also contains diagnostic terms and expressions that are no longer used, including the former classification of homosexuality as an illness; they are kept as they appear in the historical sources.

450,000 people who left the hospitals (1955–1985)

The number of patients in American state and county mental hospitals peaked in 1955 at 558,922. From there it began to fall: 193,436 in 1975, 109,939 in 1985. Moving patients out of hospitals and back into the community became known as deinstitutionalisation. The sociologist David Mechanic and the political scientist David Rochefort traced its course in a 1990 review.

President Kennedy led the way. In a special message to Congress on 5 February 1963 he proposed a national programme of community mental health care, with the target of cutting the number of patients in custodial care by half or more within ten or twenty years. The Community Mental Health Centers Act passed the same year.

Deinstitutionalisation went faster and further than the target. But the places meant to receive patients did not keep up. By 1980 a little over 700 community mental health centres had been funded under the programme, roughly half the 1,500 thought necessary to cover the country. And many centres, Mechanic and Rochefort wrote, tended to underserve people with severe and long-lasting illness.

Why did the numbers fall? The usual answer is the new drugs, such as chlorpromazine from Part 5. The drugs did give hospital staff and families confidence that patients could be managed with less coercion. But the numbers tell a slightly different story. From 1955 to 1965 the in-patient population fell by only 1.75 percent a year on average; the steep fall came between 1965 and 1975, at about 8.6 percent a year. More than three quarters of the decline between 1955 and 1985 happened between 1965 and 1980. What expanded in those years was public health insurance, which shared the cost of nursing homes, and income support for disabled people. A 1985 study by Gronfein, which Mechanic and Rochefort describe, compared states and found that hospital declines went with Medicaid payments to nursing homes, while the spread of the drugs showed no statistical link.

Many of those who left went to nursing homes or to board-and-care homes that offered a bed and meals. When welfare budgets were cut in the 1980s, more people lost their housing. Some surveys found significant psychiatric symptoms in a quarter to a half of homeless people. Mechanic and Rochefort, however, found little evidence that deinstitutionalisation was the main cause of homelessness; it was one of many.

The same review has one line on Japan: private psychiatric hospitals there were growing rapidly and replacing informal care. That is what we saw in Part 6. In the same decades in which America emptied its hospitals, Japan filled them.

The same patient, a different diagnosis

Inside the hospitals and out, another problem had been building: psychiatric diagnoses depended on the doctor.

In 1962 the psychiatrist Aaron T. Beck and colleagues had 153 patients examined separately by two psychiatrists, each making a diagnosis in the standard nomenclature of the day. The two agreed 54 percent of the time.

Countries differed too. In a comparison of New York and London published in 1972, New York psychiatrists diagnosed about 62 percent of admitted patients as schizophrenic, against 34 percent in London. For depressive psychosis it was the other way round: under 5 percent in New York, 24 percent in London. When the patients were re-examined in the same way, most of the difference turned out to lie in the habits of the psychiatrists, not in the patients (as described by the sociologist Andrew Scull in 2023).

In 1974 the Columbia psychiatrist Robert Spitzer and the statistician Joseph Fleiss recalculated the earlier studies of diagnostic agreement on a single scale. The scale is a number called kappa: how far two raters agree, after removing the agreement you would expect by chance. One means perfect agreement; zero means no better than chance. Their verdict was harsh. Reliability was satisfactory for only three categories: mental deficiency, organic brain syndrome and alcoholism. It was no better than fair for psychosis and schizophrenia, and poor for everything else.

They opened the paper with a warning. A reliable classification is not guaranteed to be valid; but an unreliable one cannot be.

Sane people in hospital: the Rosenhan experiment (1973)

That debate stayed inside the specialist journals. What took it to the public was a paper in Science on 19 January 1973 by David Rosenhan, professor of psychology and law at Stanford: "On being sane in insane places".

According to the paper, eight people with no history of mental illness (three women and five men) went to twelve hospitals in five states on the East and West coasts. Besides Rosenhan himself they were a psychology graduate student, three psychologists, a paediatrician, a psychiatrist, a painter and a housewife. They complained of one thing only: hearing voices, unclear but seeming to say "empty", "hollow" and "thud". Apart from their names and jobs, everything they said about their lives and families was true.

All twelve were admitted: eleven with a diagnosis of schizophrenia, one with manic-depressive psychosis. Once on the ward, they stopped reporting symptoms and behaved as they normally would. Even so, their stays lasted from 7 to 52 days, 19 on average, and they were discharged with a diagnosis of "schizophrenia in remission".

The people who saw through them, Rosenhan wrote, were not the doctors but other patients. In the first three admissions, when counts were kept, 35 of 118 patients on the ward voiced suspicions: "You're not crazy. You're a journalist, or a professor." The eight were handed nearly 2,100 pills in all; they swallowed two, and nobody noticed what happened to the rest.

The paper reported a second experiment. A research and teaching hospital, having heard the results, doubted that such errors could happen there. Rosenhan told the staff that over the next three months one or more pseudopatients would try to get admitted. Of 193 patients judged in that period, 41 were rated with high confidence as pseudopatients by at least one member of staff, 23 were suspected by at least one psychiatrist, and 19 by a psychiatrist and one other staff member. Rosenhan had sent nobody.

His conclusion was that in psychiatric hospitals the sane cannot be distinguished from the insane.

The rebuttal, and the records found half a century later

The leading critic was Spitzer. In papers in 1975 and 1976 he borrowed an analogy from the psychiatrist S. S. Kety. If I drank a quart of blood, hid what I had done and came to an emergency room vomiting blood, the staff would treat me for a bleeding peptic ulcer; that would not show that medicine cannot diagnose a peptic ulcer. Doctors who believed a faked symptom were, after all, responding to what the patient told them.

Spitzer also looked at the discharge diagnosis, "in remission". At his own New York State Psychiatric Institute, the official record of more than 300 patients discharged with schizophrenia in the previous year showed not one coded as "in remission". When twelve other hospitals were asked, eleven said the term was never used, or used for only a handful of patients a year. If doctors had given this rare label to the pseudopatients, they must have noticed that the pseudopatients showed no symptoms in hospital. Rosenhan's own data, Spitzer argued, contradicted his conclusion.

Nearly fifty years later, doubts came from another direction. The journalist Susannah Cahalan obtained Rosenhan's papers and his own hospital records. Andrew Scull, a sociologist at the University of California, San Diego, reported on that material in a 2023 paper.

The surviving records show that Rosenhan was admitted to a Pennsylvania state hospital on 6 February 1969. At his intake interview he did not limit himself to three words. He said he was sensitive to radio signals and could hear what people were thinking; that he had put copper over his ears to shut out the noise; that he had been unable to work for six months; and that he had had suicidal thoughts. The doctor recorded him grimacing, twitching and speaking in a low voice. Had Rosenhan told the truth about how he presented himself, Scull writes, nobody would have been surprised that he was admitted or given the diagnosis he got.

As for the other pseudopatients, two years of searching by Cahalan turned up only two, both Rosenhan's graduate students. One stayed eight days; his chart has no note of "in remission", and the space for the reason for discharge is blank. The other was the ninth participant, dismissed in a footnote of the paper as having falsified his history. His ward was well staffed, his daily contact with doctors was well over an hour, and he described his impressions of the hospital as overwhelmingly positive. What is more, an earlier draft of the paper with nine participants and the published version with eight reported identical figures for staff contact with patients. Scull concludes that the study was a scientific fraud.

Yet the paper's force at the time is beyond doubt. According to Scull, less than two weeks after publication, on 1 February 1973, the board of trustees of the American Psychiatric Association held an emergency meeting. After long discussion it set up a task force to rework its diagnostic manual, and Spitzer was chosen to lead it.

A diagnosis removed by a vote (1973)

The same year brought another controversy over diagnosis: whether homosexuality was an illness.

The first edition of the APA's Diagnostic and Statistical Manual of Mental Disorders, DSM (1952), had classified homosexuality as a "sociopathic personality disturbance"; the second (1968), as a "sexual deviation". In the early 1970s gay activists protested at the association's annual meetings, and debate grew inside the profession. The subcommittee that examined the question was chaired by Spitzer. He concluded that, apart from homosexuality and perhaps some other sexual deviations, the mental disorders all regularly caused subjective distress or were associated with a general impairment in social functioning. He had, in effect, drawn the line around mental disorder with distress and impairment.

In December 1973 the board voted to remove homosexuality from the manual. Psychoanalysts who opposed the decision petitioned for a vote of the whole membership, and 58 percent of about 10,000 voting members upheld the board. What members voted on, the psychiatrist Jack Drescher of New York Medical College pointed out in 2015, was not whether homosexuality was an illness but whether to support the board's decision. Even so, Scull writes, the impression that a diagnosis had been settled by ballot invited public ridicule and fuelled further doubts about the reliability of psychiatric diagnosis. A new diagnosis, "sexual orientation disturbance", limited to people distressed by their orientation, took its place; in DSM-III it became "ego-dystonic homosexuality", which was removed only in the 1987 revision.

Diagnosis by checklist: DSM-III (1980)

The groundwork for the new manual had been laid at Washington University in St. Louis. Since the 1950s, when psychoanalysis dominated American psychiatry, Eli Robins, Samuel Guze and their colleagues had kept to classifying illness by following its course. In 1972, with the resident John Feighner as first author, they published diagnostic criteria for use in research. These spelled out which symptoms, how many and for how long, had to be present for a diagnosis, and which cases were excluded. Spitzer and colleagues built the Research Diagnostic Criteria on these "Feighner criteria", and the RDC became central to DSM-III.

The effect showed in the numbers. According to Spitzer, an early version of the RDC gave a kappa of 0.88 for the diagnosis of schizophrenia in 68 in-patients. In the DSM-III field trials reported in 1979 (281 adults), kappa for major diagnoses overall was 0.78 when two clinicians judged the same interview together, and 0.66 when they interviewed patients separately. Given that most earlier studies had been fair or poor, this was a large step forward.

DSM-III, published in 1980, ran to 494 pages; its predecessor had 134. The content changed too. Each disorder now came with a list of diagnostic criteria and a systematic description. A definition of mental disorder was written in, and the main diagnosis and conditions such as personality disorders were recorded on separate axes.

Here Part 2 comes back. Kraepelin, not knowing the causes, divided illnesses by their symptoms and course. The American line that began with Robins and Guze at Washington University later came to be called neo-Kraepelinian, and DSM-III too drew its lines mainly by observable symptoms and course. Wilson wrote in a 1993 history of DSM-III that DSM-III moved the centre of psychiatry from a broad clinical outlook, informed by psychoanalysis among other things, to a research-based medical model, and began a return to descriptive psychiatry. It also, he added, significantly narrowed psychiatry's clinical gaze.

What could be done by this point, and what could not

First, what was achieved. DSM-III answered the problem that diagnoses changed with the doctor by giving lists of symptoms. With the same criteria, a psychiatrist in New York and one in San Francisco were more likely to give the same patient the same diagnosis. Drug trials could now gather patients with the same diagnosis and compare them. The placebo-controlled trials of Part 5 at last had a shared yardstick.

In Part 1, Earle showed that at Worcester, as soon as the superintendent changed, the share of patients discharged as recovered fell from 43.3 percent to 22.2 percent. Change the person doing the judging and the numbers change. DSM-III was the first large attempt to tackle that century-old problem head on.

Then, what was not. First, how well clinicians agree depends on how you test it. In the 2013 field trials for the next edition, DSM-5, two clinicians interviewed each patient on separate occasions without knowing each other's diagnosis. Kappa came out at 0.46 for schizophrenia, 0.28 for major depressive disorder and 0.20 for generalized anxiety disorder. The DSM-5 report notes that in the DSM-III trials a third of patients were judged by two clinicians from the same interview, and many diagnoses had few patients. The two panels of the figure above cannot be compared directly.

Second, agreeing on a diagnosis is not the same as the diagnosis being right. As Spitzer and Fleiss put it, a classification that people cannot agree on cannot be valid, but agreement does not make it valid. The DSM-5 report says the same: classify by an easily measured sign such as fever, and you may get good agreement without a group that shares one underlying cause. Most psychiatric diagnoses are still defined by combinations of symptoms. Illnesses that were separated out by finding their cause, like general paresis in Part 2, remain few.

Third, Rosenhan's paper went on appearing in textbooks for nearly half a century. The study meant to expose how shaky diagnosis was had itself spread without being checked. Scepticism about numbers had to be turned on the people producing them as well. This article centres on DSM-III in 1980 and does not go into the content of later revisions.

Closing the series

This series has followed one thread: the gap between being able to treat or classify something and understanding why it works. Across seven parts, that gap never closed.

Moral treatment in Part 1 showed that patients could be managed without knowing the cause, but its "nine in ten cured" changed greatly with how the counting was done and who did the judging. In Part 2, only general paresis was traced to a cause, syphilis; other illnesses were divided by their course. Psychoanalysis in Part 3 began with a story of being cured by talking, but compared with people who had no treatment, its effect was unclear. Among the bodily treatments of Part 4, lobotomy won a Nobel Prize, and then its effect vanished in trials with a comparison group. The drugs of Part 5 were found by chance, and the explanations of why they work were built afterwards. In Japan, in Part 6, hospitals were built, but confining people and treating them were never clearly separated, and the same in-patients could be counted as 12 percent or 80 percent compulsory. And this time: diagnoses came to agree, but whether those divisions match real boundaries in illness is still not known.

What grew over two hundred years were tools for counting, comparing and standardising: recounting who recovered, comparing with people who went untreated, comparing with placebo, aligning the criteria for diagnosis. Each was made to reduce error even while the "why" stayed unknown. And the better the tools became, the more clearly they showed what remained unexplained. This brings the series A History of Psychiatry to a close. Thank you for reading to the end.