Transparency can play a big role in monitoring and tracking agency progress on goals. A new GAO study details the effectiveness of six agencies in reporting on the quality of their performance information.
In September 2015, the Government Accountability Office released a study on the transparency of public reporting. The 55-page report, Managing for Results: Greater Transparency Needed in Public Reporting on the Quality of Performance Information for Selected Agencies' Priority Goals, was one of a series responding to a statutory provision that GAO review how agencies are implementing the Government Performance and Results Modernization Act of 2010 (GPRAMA).
Specifically, GAO reported on the effectiveness of selected agencies in publicly reporting on the quality of the performance information they use to measure progress on agency priority goals. APGs are an agency's highest priority goals, and drive progress in important and complex areas, such as strengthening disaster preparedness and improving food safety.
GAO selected six agencies for its review: NASA and the Departments of Agriculture, Defense, Homeland Security, Interior, and Labor. Agency selections were based on responses to a question in GAO's 2013 federal managers' survey, reported in Managing for Results: Agencies' Trends in the Use of Performance Information to Make Decisions. The question asked managers to rank whether they had sufficient information on the validity of the performance data they use to make decisions.
GAO reviewed the information the six selected agencies had published on Performance.gov and in their annual performance plans and reports. Among the six agencies, GAO reviewed data related to 23 APGs.
GPRAMA Requirements
To improve transparency, GPRAMA requires agencies to publicly describe how they ensure the accuracy and reliability of the performance information they use to measure progress toward priority goals. GPRAMA has five transparency requirements for reporting on information quality:
- the means used to verify and validate performance data
- the sources of the data
- the level of accuracy required for the intended use of the data
- any limitations to the data at the required level of accuracy
- how the agency will compensate for such limitations (if needed) to reach the required level of accuracy.
The Office of Management and Budget—which is responsible for working with agencies to implement GPRAMA—is required to publish this information for each APG on Performance.gov. This public website consolidates information about the APGs and makes it readily accessible to external audiences, such as members of Congress, citizens, journalists, and researchers. In addition, agencies must address all five requirements for their APGs in their annual performance plans, which identify the planned level of performance for the current fiscal year and the next fiscal year. Finally, agencies need to report data in performance reports that summarize the actual level of performance achieved during the previous five fiscal years.
Transparency of Public Reporting
GAO found little information on Performance.gov about how these six agencies ensured the performance information they reported on their APGs was accurate and reliable. Although they used various sections of Performance.gov to discuss some of GPRAMA's performance information quality requirements for APGs, none addressed all five requirements. Moreover, while GAO found hyperlinks from Performance.gov to the selected agencies' performance plans and reports, there was no explanation of where to find performance information quality discussions in these plans and reports. Consequently, GAO recommends that all six agencies work with OMB to provide this information on Performance.gov for their APGs.
GAO's review of each agency's performance plans and reports found that all agencies provided explanations for how they ensured the overall quality of their performance information. For example, the USDA stated that its performance data were collected using a standardized methodology that has been vetted by scientists, policymakers, and the under secretaries of the respective mission areas. However, five of the six agencies' performance plans and reports failed to explain how they ensured information quality for individual APGs. In fact, the DHS was the only agency to address all five GPRAMA requirements for its three APGs. GAO recommended that all of these agencies, except DHS, more fully address these requirements in their performance plans and reports.
Distinct Approaches to Transparent Reporting
Agencies took different approaches to being transparent in their performance reporting. For example, for fiscal years 2013-2016, DHS published an appendix to its performance plans and reports that provided a detailed data-quality explanation. These appendices addressed the five performance information quality requirements for all of DHS's APGs. For each performance measure related to its three APGs, DHS's appendices described the related program, the scope of the data, the source and collection methodology for the data, and an assessment of data reliability.
Meanwhile, the USDA provided data quality explanations that helped readers better understand how it was ensuring the accuracy of performance information on a particular program or policy. For instance, to report how USDA measures progress on two of its five transparency requirements for the Reduce the Number of Foodborne Salmonella Illnesses APG, the agency detailed how it identifies a data source and a potential limitation.
The USDA, for example, identified the Centers for Disease Control and Prevention as the source of the data measuring the number of illnesses from products USDA's Food Safety and Inspection Service regulates. The USDA also noted that the CDC receives information from state and local health agencies concerning outbreaks of illnesses. The USDA acknowledges that the quality of the data can vary by reporting agency, which is an example of identifying a potential limitation.
Plans to Improve Transparency
Both OMB and the selected agencies GAO reviewed have plans to further improve the transparency of their public reporting on performance information quality for APGs. In June 2015, OMB updated its A-11 guidance that directs agencies on how to publish information on Performance.gov about how they ensure the quality of their APG performance information. In addition, the OMB A-11 Circular advises agencies to provide a hyperlink from Performance.gov to an appendix in their performance report that discusses the quality of their performance information.
Another effort to improve transparency comes from the Performance Improvement Council. GPRAMA established the council, which is chaired by OMB and comprised of agency performance improvement officers, to facilitate the exchange of useful practices to strengthen agency performance management. The council's executive director told GAO that it wanted to develop a collection of useful and leading practices on performance information quality that can be shared with agency officials.
In February 2015, the Performance Improvement Council established the Data Quality Cross-Agency Working Group. GAO reported that this group had identified several goals that will help with transparency efforts, including:
- improving the reliability and quality of performance information and of the reporting process
- setting standards and developing consistency across agencies
- highlighting good performance measures and accurate and appropriate performance information.
What's more, in response to GAO's report, five of the agencies described plans to improve their public reporting. The Department of Labor stated that its performance report for fiscal year 2016 will summarize a data quality assessment tool that the agency plans to pilot in the coming year. USDA, the Department of the Interior, and NASA also agreed to be more transparent and add new information on Performance.gov and in their performance plans and reports for fiscal years 2016 and 2017. Finally, DHS noted that it provided specific suggestions to OMB on how agencies could provide more comprehensive data quality information on Performance.gov.
When implemented, these improvements will lead to a more transparent government, making it easier for individuals to access performance information that is publicly reported in a clear and accessible way. While there are multiple approaches agencies can use to report on performance information quality, it will be important for agencies and OMB to consider how they can make this information readily accessible to external audiences—and to let the five specific GPRAMA requirements guide their efforts. Bottom line: By becoming more transparent, federal agencies maintain the confidence of Congress and citizens about the quality and credibility of the performance information they use to assess and achieve government goals.
GPRAMA's Requirements for Annual Performance Plans and Reports
Performance plans should:
- identify the planned level of performance for the current fiscal year and the next fiscal year
- explain how the agency will ensure the accuracy and reliability of its performance information
- identify the agency’s priority goals (APGs)
- be published every February, concurrent with the president’s budget.
Performance reports should:
- summarize the actual level of performance agencies achieved during the previous five fiscal years,
- explain how the agency ensures the accuracy and reliability of its performance information
- be published every February.
Source: GPRAMA. 31 U.S.C. §§ 1115(b) and 1116(b)(c). |GAO-15-788
