2-9-18- RE: Comments on Review of Advances Made to the IRIS Process: A Workshop (PIN: DELS-BEST-17-03)

Printer-Friendly Version          Fact Sheet


February 9, 2018

Jonathan M. Samet, MD, MS (Chair)

Committee to Review of Advances Made to the IRIS Process

Board on Environmental Studies and Toxicology, Division of Earth and Life Studies

The National Academies of Science, Engineering, and Medicine

2101 Constitution Avenue Northwest

Washington, DC 20418


RE: Comments on Review of Advances Made to the IRIS Process: A Workshop (PIN: DELS-BEST-17-03)


Dear Dr. Samet:

I am writing to make several specific recommendations regarding the Integrated Risk Inventory System (IRIS) that would benefit EPA and the scientific community.  My office has been working on issues related to the IRIS since 2011.  Small businesses are very concerned with the accuracy of scientific determinations made under IRIS since these assessments are often used in regulations promulgated by the agency and other regulatory bodies.  Advocacy has several observations and recommendations for your consideration in your workshop critique of EPA’s post-2011 efforts to improve IRIS. 

Office of Advocacy

Congress established the Office of Advocacy under Pub. L. No. 94-305 to advocate the views of small entities before Federal agencies and Congress. As Advocacy is an independent body within the U.S. Small Business Administration (SBA), the views expressed by Advocacy do not necessarily reflect either the position of the Administration or the SBA.



Advocacy observed in late 2010 and early 2011, that EPA was having great difficulty addressing scientific issues for a variety of chemicals, including arsenic, formaldehyde, hexavalent chromium and perchlorate.  Advocacy invited attorneys and scientists to present these issues before a small business environmental roundtable discussion in May 2011.  There was a clear pattern – EPA scientific analyses were inadequately justified and lacked transparency, often resulting in a finding of additional risk that may not be justified under a more robust examination. 

Coincidentally, the Formaldehyde NRC report was released in April 2011, which found that EPA was having similar difficulties in the IRIS program.  The NRC sharply critiqued the IRIS program for persistent failures to provide objective scientific evidence to support its conclusions.[1]  The NRC noted problems with the objectivity, scientific accuracy and transparency needed to ensure high quality assessments.   Advocacy believes that more transparent and stronger science could emerge at IRIS.  Such a development could serve as a template for improvements at EPA and possibly other national and international bodies that perform scientific assessments.   Advocacy has worked closely with EPA since 2011 on IRIS implementation, and endorsed the excellent IRIS enhancements announced in July 2013 by Ken Olden, the previous Director of the National Center for Environmental Assessment (NCEA).[2]  Advocacy commends EPA’s accomplishments in implementing those revised procedures to date (hereinafter “IRIS 2.0”).


Discussion and Recommendations

As a close observer and participant in several IRIS-related proceedings over many years, Advocacy believes that EPA has made great progress in implementing IRIS 2.0.  In particular, as demonstrated at this recent NRC workshop,[3] EPA has created a world class system to perform systematic review for chemical hazard assessments, including identification of study quality and criteria for selecting key studies for quantitative analysis, as demonstrated by the discussion at the workshop, the draft systematic review protocols,[4] and at the presentations in the poster session.  Advocacy applauds EPA’s substantial achievements over this short period of time.

However, EPA has not yet demonstrated the scientific maturity and expertise to implement the final critical steps in the assessment addressed by the NRC recommendations, specifically the judgments needed to evaluate study quality, select key studies, utilize expert judgment evaluating  complex streams of evidence, and finally to derive sound toxicity values. EPA did reveal an understanding of the final steps needed to complete the assessment, but it hasn’t demonstrated that these steps can be successfully implemented.   Unfortunately, EPA did not provide any mature assessments under IRIS 2.0 to allow the NRC to do a review on this final key portion of the IRIS improvements.  Advocacy recommends that NRC prioritize review of the EPA IRIS handbook and IRIS 2.0 assessments, the key documents that would show how close EPA is to implementing the final elements of the IRIS recommendations. 

Advocacy believes that the most recent evidence regarding the EPA ability to implement these final scientific judgments is unsettling.  Public statements at the workshop by Dr. Sam Cohen, of the University of Nebraska Medical Center and Dr. Jessica Ryman-Rasmussen of API regarding TBA (tert butyl alcohol) and ETBE (ethyl tertiary butyl ether) indicate that, despite the lengthy and comprehensive discussion in the previous June 2016 Public Science meeting addressing the key science issues, EPA failed to account for the pathology-related criticisms that EPA was improperly using rat kidney tumor data to represent noncancer risk to humans. The subsequent September 2017 peer review discussion highlighted again the lack of pathology expertise available to EPA, which contributed to this confusion.[5]  One of the peer reviewers, Dr. Lorenz Rhomberg, a toxicologist from the consulting firm, Gradient, stated that at a minimum, EPA needed to acknowledge “a significant dissenting body of expert opinion.”[6]  

Tina Bahadori, the current Director of NCEA, did explain after the API testimony at the workshop, that EPA judged that it was too late to incorporate IRIS 2.0 procedures into the TBA and ETBE assessments.  However, the Agency could have brought EPA or non-EPA pathologists to engage the peer reviewers in the September 2017 peer review to at least partially remedy the known shortcomings of the assessment.   There was substantial public input from the earlier June 2016 public session which made the various inadequacies quite apparent to anyone in attendance, but the new draft 2017 assessments failed to correct the errors.   EPA needs to redouble its efforts at IRIS step one public meetings to discuss the key issues and, most importantly, to take these concerns more seriously.[7]  In addition, EPA proposed a derivation of a toxicity value for ETBE, despite the EPA protocol that such values should not be derived for chemicals with only “suggested evidence of carcinogenicity.”[8]  This workshop testimony on TBA and ETBE was reminiscent of the problems observed in the 2011 NRC report.

Our most recent experience with the perchlorate review by the EPA Office of Water last month is instructive.    Even with the help of EPA IRIS program, EPA failed to perform any quality review of the five key studies selected for quantitative analysis.  The agency selected only the five positive studies for quantitative analysis, and did not use the negative studies.[9]  EPA explicitly declined to perform a systematic review of the literature in the structured manner now adopted in IRIS.[10]  Although this activity was not intended to meet IRIS 2.0 standards, EPA’s effort in 2017, six years after the NRC report, appeared to fall far short of these standards.  A study that would inform the setting of a drinking water standard for the U.S. population warrants more expert judgment and precision. 



Advocacy has some specific recommendations to the NRC for a more robust review that would benefit EPA and the scientific community.   The Office urges the NRC to take five steps:

  1. Review the material provided to date by EPA, which include the systematic review protocols and the workshop posters.
  2. Review the recently released protocols after having the opportunity to review the public comments on those protocols (comment period ends in early March).
  3. Review the input from the IRIS Public Science Meetings (including TBA and ETBE) and how EPA responded to this input in the draft assessments.
  4. Review the completed portions of the nearly completed IRIS handbook.[11]  This key set of protocols would benefit from NRC review.  
  5. Review an IRIS 2.0 Assessment whenever EPA completes either an assessment for peer review (IRIS step four) or a final assessment (IRIS step seven).   Reviewing actual assessments would allow NRC to determine if EPA is institutionally capable of implementing the IRIS 2.0 reforms.  

Advocacy appreciates the opportunity to provide input on these important issues.  If you want to discuss these issues with my office, please contact Kevin Bromberg of my staff at kevin.bromberg@sba.gov or 202-205-6964.  





Major Clark

Acting Chief Counsel for Advocacy




Kevin Bromberg

Assistant Chief Counsel

Office of Advocacy


Copy to:          The Honorable Neomi Rao
Office of Information and Regulatory Affairs
Office of Management and Budget





























[1] Review of EPA Formaldehyde Committee to Review EPA’s Draft IRIS Assessment of Formaldehyde, National Academy of Sciences,  April 8, 2011, Washington, D.C.

[3] Review of Advances Made to the IRIS Process: A Workshop (PIN: DELS-BEST-17-03), February 1-2, 2018.

[5] See details in the Cohen and Ryman-Rasmussen comments.

[6] Dr. Rhomberg wrote in his September 2017 preliminary peer review comments: 


In public comments, some strong views, supported by analysis of a specifically convened PWG, are expressed regarding whether the kidney endpoints are separable, whether they are better considered as various aspects of Chronic Progressive Nephropathy (CPN), and whether they are relevant to processes that could occur in humans. Importantly, the endpoint chosen as critical, urothelial hyperplasia, is characterized by the PWG as a stage in CPN. In sum, the question of the validity and applicability of the endpoints analyzed for the oral RfD needs to be carefully examined.  Even if the decision is to use them, that use must be couched in prominent caveats that acknowledges a significant dissenting body of expert opinion.


Even if one decides to employ these endpoints, it has been said by knowledgeable public commenters that, because the endpoints are seen as a suite of CPN manifestations, not all appearances will necessarily be noted in pathological examination, and the counts (and denominators) may be inappropriate. This question needs a clear resolution if the data are to be taken as valid for analysis.


[7] Under the 2013 IRIS Enhancements, EPA holds a “step one” public meeting to discuss literature search, evidence tables, exposure-response figures, and key issues.  In our experience, these public meetings of experts in the field, including scientists selected by the NRC, have been of invaluable assistance in bringing significant new information to EPA.  It remains to be determined how well EPA will address the new information. 

[8] EPA Guidelines for Carcinogenic Risk Assessment, March 2005, p. 3-2; Ryman-Rasmussen Comments, January 26, 2018, pp. 4-5.

[9] NASA Comments, November 20, 2017, p. 4.

[10] EPA Draft Report: Proposed Approaches to Inform the Derivation of a Maximum Contaminant Level Goal for Perchlorate in Drinking Water Proposed Approaches, September 2017,  p. 5-1. 

[11] The handbook contains the instructions for completing systematic review and other protocols underlying the development of IRIS 2.0 assessments.