By 1900 overcrowding in New York City's psychiatric asylums had become a serious problem. There were several strategies implemented to deal with the escalating patient overload. One was to put the patients to work, farming in a relaxed setting on what was then rural Long Island. The new state hospitals were dubbed "farm colonies" because of their live-and-work treatment programs and emphasis on agriculture. However, these farm colonies, Kings Park State Hospital, (later named Kings Park Psychiatric Center) and Central Islip State Hospital (later named Central Islip Psychiatric Center), became overcrowded, like the institutions they were meant to replace.
New York State began making plans for a third farm colony, which was to become Pilgrim State Hospital, named in honor of the former New York State Commissioner of Mental Health, Dr. Charles W. Pilgrim. The state bought approximately 1,000 acres (400 ha) of land in Brentwood and began construction on the hospital in 1929. Pilgrim State Hospital opened on October 1, 1931 as a close-knit community with its own police and fire department, courts, post office, a LIRR station, power plant, swine farm, church, cemetery and water tower, as well as houses for staff and administrators. A series of underground tunnels were used for routing utilities. Each set of buildings were known as quads, a pattern of four buildings situated around a center building, where the kitchen was located.
The hospital continued to grow as the patient population increased. Eventually, the state of New York bought up more land to the southwest of the facility to construct Edgewood State Hospital, a short-lived facility that was a subsidiary of Pilgrim State Hospital. In fact, Pilgrim State Hospital was so large that it reached into four Suffolk towns: Huntington, Babylon, Smithtown and Islip, and had two state roads passing through its bounds.
During World War II, the War Department took control of Edgewood State Hospital, along with three new buildings at Pilgrim State Hospital, buildings 81, 82, and 83. The War Department constructed numerous temporary structures and renamed Edgewood State Hospital and buildings 81–83 "Mason General Hospital," a psychiatric hospital devoted to treating battle-traumatized soldiers. Renowned filmmaker John Huston, who received a special commission in the U.S. Army Signal Corps during World War II, made a documentary at Mason General Hospital called "Let There Be Light", which showed the effects of war on mental health. The film was highly controversial and was not seen by the public until 1981.
After World War II, Pilgrim State Hospital experienced an increase in patient population that made it the world's largest hospital, with 13,875 patients and over 4,000 employees. In the 1950s more aggressive treatments, such as lobotomy and electro-convulsive therapy (ECT) were implemented. The best known controversy about this surrounded the case of Beulah Jones, a patient there between 1952 and 1972 who received both such treatments and was left seriously impaired. However, Pilgrim State Hospital and the other state hospitals began to decline shortly afterwards with the advent of pharmaceutical alternatives to institutionalization.
Dr. Henry Brill served as the director of Pilgrim from 1958 to 1974 and presided over both the introduction of the new anti-psychotic medications and the large numbers of discharges related to good response to these medications.
As psychiatric medication and community care became an increasingly viable alternative to institutionalization, large mental institutions began to decline. Edgewood, the last psychiatric hospital to be built on Long Island, closed its doors in December, 1971, following decentralization. Kings Park and Central Islip remained open, while slowly downsizing. During this time Pilgrim was not exempt from downsizing either, with parts of the campus closing throughout the 1970s and 1980s. Buildings 81–83 were briefly used as a correctional facility in the 1980s, but after community protest they were renovated and reverted to psychiatric use. In the early 1990s, with declining patient populations in the three remaining hospitals, the New York State Office of Mental Health (formerly the Department of Mental Hygiene) began making plans to re-organize the Long Island hospitals, which were implemented in the fall of 1996, when Kings Park and Central Islip were closed, and the remaining patients from those facilities were transferred to Pilgrim or released into community care. Parts of Central Islip Psychiatric Center became a campus for the New York Institute of Technology, as well as a residential and commercial development. At Kings Park, two buildings housing community residences administered by Pilgrim remain open. A 52-acre (0.21 km2) portion of former parts of the Pilgrim campus has become the Brentwood State Park athletic field complex, while the rest sits unused.
Today, the much smaller Pilgrim Psychiatric Center is the last of the state institutions still operating on Long Island. The farm was sold, the land developed, and it became the Western Campus of the Suffolk County Community College in 1974.
Gerald Wolkoff bought 462 acres (187 ha) of the remaining property for $21 million in 2002 and announced a plan to build a $4 billion residential/office complex, called Heartland Town Center, on the site which borders his Heartland Business Park, which is to the west of the complex. In preparation, several former hospital buildings were demolished in 2003, however, rebuilding has not yet begun. In 2011, the former housing facilities for the hospital staff were demolished in further preparation of redevelopment. Demolition of the old medical/surgical building, male and female admissions buildings and the old administration building was completed in 2012.
On July 1, 2009 Dr. Anand D. Nadkarni M.D. retired after 27 years of state service. Dr. Nadkarni had worked as an Attending Psychiatrist from 1987 to 1994, when he moved to take on supervisory positions. He eventually became the Chief of Psychiatry. During his tenure at Pilgrim, Dr. Nadkarni also had his own Private Practice and worked for the Town of Islip. He currently serves as the Medical Director for ACCESS/ACCESSO, an Alcohol & Drug Counseling Center in Islip.
Another proposed project for the site is the Pilgrim Intermodal Freight Transportation Center, which would be a facility for transloading freight from trains to trucks. It would be built on a 120-acre (49 ha) tract owned by the state and use a rail siding of the Long Island Rail Road that previously carried freight and visitors to the hospital.
Pilgrim Psychiatric Center hosts the Long Island Psychiatric Museum, which displays items from Kings Park, Central Islip, Pilgrim, and Edgewood, such as photos, newsletters, relics from abandoned and demolished buildings, and other historical information. It is located in Building 45 and is open to the public.
In 1985 the movie "Murder: By Reason of Insanity", starring Candice Bergen, was filmed on the grounds of Pilgrim State Hospital, in Building 14. This movie was based on Adam Berwid, who had murdered his wife on a Day Pass from the hospital. Staff working at the facility were able to audition for small roles. The case was also the subject of a 1980 segment of CBS's 60 Minutes.
Allen Ginsberg's mother, Naomi Livergant Ginsberg, who suffered with schizophrenia throughout most of her life, died at Pilgrim State Hospital in 1956. "Pilgrim State" is mentioned in Ginsberg's poem "Howl."
Pilgrim State Hospital is mentioned in the 2009 documentary movie Cropsey, as having reportedly housed the mother of convicted child kidnapper Andre Rand. One of Andre's supposed victims, Jennifer Schweiger, was found buried in a shallow grave behind the grounds of the abandoned Willowbrook State School which was built under the same design as Pilgrim State Hospital.
Currently, Pilgrim Psychiatric Center is protected by members of the NYS Office of Mental Health Safety and Security Officers. OMH safety and security officers have New York State peace officer status which is granted under the Mental Hygiene Law (section 7.25), Public Health Law (section 455), and Criminal Procedure Law (section 2.10 subsection 12) which allows officers to issue summonses and effect arrests. Some of the duties performed by officers include, but are not limited to, enforcing state and local laws including vehicle and traffic laws, apprehending absconded patients, filling out motor vehicle accident reports and performing motor vehicle accident investigations within state grounds. Officers also are responsible for conducting fire service procedures which include conducting fire drills, fire extinguisher inspections, and building inspections.