Conversation with Douglas Dockery ScD

Douglas DockeryFor over a decade, Harvard University Professor Douglas Dockery has been conducting research with enormous implications for human health and for public policy. His work has focused on the potential for polluted air to cause a range of health problems, including cardiovascular disease, asthma and other respiratory ailments.  Dockery’s research has been at the center of the debate regarding what levels of particular pollutants are dangerous, and what limits the federal government should impose on sources of emissions to protect public health. 

Because it is the science that justifies tightening regulation, Dockery’s work has been subject to extraordinary scrutiny. Its quality and conclusions have withstood replication and reanalysis and he has become one of Harvard’s leading public health researchers. Nonetheless, the implicit and explicit attacks on his work have colored his experience as an academic, and they worry him: how will political processes change the norms and practice of science when controversial issues such as public health or the environment are at stake?  SKAPP’s Polly Hoppin and Dick Clapp had an opportunity to interview Dr. Dockery. The paragraphs that follow summarize the conversation.

Dockery’s training was in physics and meteorology. After graduating from MIT, he took a job at EPA, working on air pollution modeling and evaluations of environmental impact statements. He soon became frustrated with the work, feeling as if he was “cranking through numbers but not getting at the core questions of the health effects” of environmental exposures.  His decision to pursue a doctorate at the Harvard School of Public Health—with the goal of learning how to evaluate the health effects of air pollution—coincided with the start of the Six Cities Study, which looked at the effects on mortality of exposure to air pollutants.  Though he did not anticipate it at the time, the Six Cities Study would redefine the debate about air pollution and health.

The publication of the Six Cities Study in 1993 strengthened a well-established body of evidence on links between air pollution and health problems. Prior to Dockery’s work a range of studies had demonstrated that exposure to airborne contaminants was associated with changes in and accelerated loss of pulmonary function in children, increased asthma attacks, and respiratory illness more broadly. The Six Cities Study added the finding that exposure to air pollutants increased mortality rates, and shortened life. 

Intense scrutiny of the Six Cities Study did not begin until two years after the publication of the initial results, when, based in part on Six Cities, EPA proposed strict new National Ambient Air Quality Standards for fine particulates and to maintain strict standards for PM10 (particles less than 10 µm in median aerodynamic diameter).  Though the Clean Air Act explicitly states that standards should be set to protect health, and should not take costs into account, subsequent mandates require that EPA conduct cost-benefit analyses of proposed rules. Dockery says that the Six Cities finding of increased mortality suddenly made the likely cost of the new health-based standards “large and tractable.” Using well-established methods for calculating the monetary value of lost lives, cost-benefit analyses taking into account the Six Cities results concluded that the benefits of regulation outweighed cost by five to ten times.

Vigorous efforts to weaken the scientific case for regulation ensued. Some critics proposed alternative explanations for the associations Dockery and his colleagues observed. People living in the areas with high air pollution also had poor diets, or were not exercising adequately, they posited; the study failed to control for these variables, or for the hardness of the water, or for climatic variation.  Some of the critiques implied that Dockery and colleagues had misrepresented the associations between the exposures and the health outcomes. Others argued that the study results should not be accepted until they were independently validated. Some of this debate reflected the normal process of scientific discovery, in which an article is published in a peer reviewed journal, inviting response from others and ongoing exchange among scientists. But much of it, Dockery says, was particular to the regulatory setting, in which EPA and its Scientific Advisory Board review available evidence at an accelerated pace, and some interest groups attempted to discredit the body of evidence supporting a proposed rule by pointing out flaws in individual studies they think are crucial to the case for regulation. 

This was not the first time Dockery and his colleagues had been challenged, even accused of fraud. There were also investigations and reevaluations of their data during the PM10 standard setting process in the mid-1980’s.  Speaking about how these most recent challenges had affected his life and work, Dockery said: “I felt under extraordinary pressure, getting up very early to read the newspaper to see what was being written about us on any given day, and to spot new problems that were developing.” Though he and his colleagues anticipated this scrutiny from the outset, incorporating independent audits and verifications into the research plan as a result, the criticisms were relentless, often without a basis, and focused on data that he and his colleagues had collected 25 years earlier. Going to work, he said, wasn’t fun. The questioning raised the specter of some mistake; some “error lurking in the data that you had overlooked. You do your best, but you never know, deep down, whether there was some mistake or error that you overlooked.” Dockery did get tenure from Harvard during this period, but was acutely aware that some colleagues raised concerns about the quality and orientation of his work. He was surprised at times, he said, hearing indirectly about negative comments fellow faculty members had made about him.

As the debate over the quality of Dockery’s science grew more heated, EPA urged that a process be put in place to bring some order to the hundreds of comments made. The study authors and EPA agreed that independent replication of the study—often seen as essential in verification—was impossible in a short timeframe. They proposed instead that the Health Effects Institute, an independent research agency set up specifically to address science issues relevant to policy debates, review all the evidence relevant to the proposed rule. Dockery admits to some trepidation about this approach, but he was reassured by HEI’s success in conducting at least one relevant reanalysis (of studies examining associations between exposure to particulate matter and acute health effects.) He and his colleagues turned over their raw data to HEI, and agreed to provide input to the questions asked in the review or to the data analysis.

We asked Dockery whether or not he would recommend this approach for resolving other questions about science in the context of rulemaking. He thought the HEI process was a good solution in this particular situation, but was too difficult and expensive to consider applying to the science behind every proposed rule. One of the problems HEI confronted, for example, was the investigators’ assurance to the study subjects that they would not disclose their identities. Though they decided that it was not necessary for HEI to try to secure consent from the original participants a second time, HEI did have to negotiate and sign agreements with a suite of individual states and agencies to obtain mortality data that they could use to validate death certificates.

Ultimately, HEI’s reanalysis affirmed the quality of the Six Cities study data and comfirmed its results. In 1997, EPA enacted a new rule regulating particulate matter. Although the implementation of the rule was delayed years by litigation, the Supreme Court ruled to allow EPA to move forward with the new standards.

In this case, scrutiny of the science and political decision-making led to what Dockery thinks is the right result. In the longer term, he worries about the impact of this kind of scrutiny on scientific research.  Mechanisms like the Freedom of Information Act, the Shelby Amendment, and, more recently, the Information Quality Act, facilitate requests for data and challenges to them. The process can drag on for years, subjecting the researcher to seemingly endless exchanges in which few holds are barred about the quality of any particular study. "For people coming into this for the first time," Dockery says, "this could be an extraordinary shock." It could also change the norms of research; guarantees of confidentiality to study subjects will be replaced by clauses such as: "we will try to keep the information confidential, but under FOIA, we may be required to turn it over to the federal government."

Dockery is concerned about the growing political power of those who tend to discount risk and oppose regulation regardless of the strength of the evidence on hazards from current levels of emissions. Yet Dockery sees his biggest influence as providing scientific information, and not advocating one particular “side” or the other. The most important thing he provides to the debate, he says, is objectivity. On the other hand, “there are many people who feel a need to speak very strongly in favor of one position or another.” He respects that position: “it is something that individual scientists need to make a decision regarding where they align themselves in these debates.”

Is Dockery optimistic about the potential for good science to lead to policy that is protective of health and the environment? He says that “there is an expectation that science will provide the answers. We hear from many politicians that we want to make decisions on the best possible science. The problem is that science does not provide hard and fast answers.” He goes on: “People seem to think there is a definitive study. But all of studies have flaws and weaknesses.” Science can not conclusively demonstrate whether an exposure is safe or not safe; “those are political decisions.” “Science can provide information, but ultimately decisions are made by public policy makers based on what is deemed to politically acceptable levels of risk versus benefits to the population. We need to recognize that there are two separate judgment processes here; science contributes to public policy, but does not define public policy.”