DR CTQS TEP Meeting

Virtual Conference via Zoom

Meeting 3: October 1st, 2019

Meeting Minutes

Project Overview:

The Centers for Medicare & Medicaid Services (CMS) has granted an award to the University of California San Francisco (UCSF) to develop a measure of computed tomography (CT) image quality and radiation safety. The project is a part of CMS’s Medicare Access & CHIP Reauthorization Act (MACRA)/Measure Development for the Quality Payment Program. The project title is “DR CTQS: Defining and Rewarding Computed Tomography Quality and Safety”. The Cooperative Agreement number is 1V1CMS331638-02-00. As part of its measure development process, UCSF convened groups of stakeholders and experts who contributed direction and thoughtful input to the measure developer during measure development and maintenance.

Project Objectives:

The goal of the project is to create a quality measure for CT to ensure image quality standards are preserved and harmful effects of radiation used to perform the tests are minimized. Radiation doses delivered by CT are far higher than conventional radiographs (x-rays), the doses are in the range known to be carcinogenic, and there is a significant performance gap across health care organizations and clinicians which has consequences for patients. The goal of the measure is to provide a framework where health care organizations and clinicians can assess their doses, compare them to benchmarks, and take corrective action to lower them while preserving the quality of images so that they are useful to support clinical practice. The measure will be electronically specified using procedural and diagnostic codes in billing data as well as image and electronic data stored with CT scans, typically stored within the Picture Archiving and Communication Systems (PACS) - the computerized systems for reviewing and storing imaging data or Radiology Information Systems (RIS).

TEP Objectives:

In its role as a measure developer, the University of California San Francisco is obtaining input from a broad group of stakeholders to develop a set of recommendations to develop a radiology quality and safety measure. The proposed measure will be developed with the close collaboration of the leadership from diverse medical societies as well as payers, health care organizations, experts in safety and accreditation, and patient advocates. A well-balanced representation of stakeholders on the TEP is intended to ensure the consideration of key perspectives and obtain balanced input.

Scope of Responsibilities:

The TEP’s role is to provide input and advice to the measure developer (University of California San Francisco) related to a series of planned steps throughout the 3-year project. The specific steps will include developing and testing a risk-adjusted measure which can be used to monitor CT image quality in the context of minimizing radiation doses while maintaining acceptable image quality. The TEP will assist UCSF in conceptualizing the measure and any appropriate risk adjustment of it. The TEP will assist UCSF with identifying barriers to implementing the proposed measure and test sites in which the developer can assess the feasibility and performance of its use. The TEP will assist UCSF with interpreting results obtained from the test sites and in suggesting modifications of the measure prior to it being incorporated into a software tool which will be made available to providers to enable them to report and monitor their performance. The TEP will provide input and advice to UCSF regarding the software tool to ensure that it is valuable for a wide range of stakeholders and CMS.

Guiding Principles:

Participation on the TEP is voluntary. Individuals participating on the TEP understand that their input will be recorded in the meeting minutes. Proceedings of the TEP will be summarized in a report that may be disclosed to the general public. If a participant has disclosed private, personal data by his or her own choice, then that material and those communications are not deemed to be covered by patient-provider confidentiality. Questions about confidentiality will be answered by the TEP organizers.

All potential TEP members must disclose any significant financial interest or other relationships that may influence their perceptions or judgment. It is unethical to conceal (or fail to disclose) conflicts of interest. However, the disclosure requirement is not intended to prevent individuals with particular perspectives or strong points of view from serving on the TEP. The intent of full disclosure is to inform the TEP organizers, other TEP members and CMS about the source of TEP members’ perspectives and how that might affect discussions or recommendations.

All potential TEP members should be able to commit to the anticipated time frame needed to perform the functions of the TEP.

Estimated Number and Frequency of Meetings:

TEP is expected to meet three times per year either in-person or via a webinar.

Table 1. TEP Member Name, Title, and Affiliation

Name	Title	Organization
Attendees
Niall Brennan, MPP	CEO	Health Care Cost Institute
Helen Burstin, MD, MPH, FACP	Executive Vice President	Council of Medical Specialty Societies
Mythreyi Bhargavan Chatfield, PhD	Executive Vice President	American College of Radiology
Jay Bronner, MD	President and Chief Medical Officer	Radiology Partners
Missy Danforth	Vice President of Health Care Ratings	The Leapfrog Group
Tricia Elliot, MBA, CPHQ	Director, Quality Measurement	Joint Commission
Jeph Herrin, PhD	Adjunct Assistant Professor	Yale University
Hedvig Hricak, MD, PhD	Radiology Chair	Memorial Sloan Kettering Cancer Center
Name	Title	Organization
Attendees
Leonard Lichtenfeld, MD, MACP	Interim Chief Medical Officer	American Cancer Society, Inc.
Matthew Nielsen, MD, MS	Professor	UNC Gillings School of Global Public Health
Debra Ritzwoller, PhD	Patient	Patient Representative
Lewis Sandy, MD	Executive Vice President, Clinical Advancement	UnitedHealth Group
Mary Suzanne Schrandt, JD	Patient	Patient Representative
Arjun Venkatesh, MD, MBA, MHS	Assistant Professor	Yale School of Medicine
Kenneth Wang, MD, PhD	Adjunct Assistant Professor	University of Maryland, Baltimore
Not in Attendance
Anthony “Tony” Siebert, PhD	Professor	University of California, Davis
Todd Villines, MD, FSCCT	Professor and Director of Cardiovascular Research and Cardiac CT Programs	University of Virginia

 

Ex Officio TEP
Amy Berrington de Gonzalez, DPhil	Branch Chief & Senior Investigator	National Cancer Institute; Division of Cancer Epidemiology & Genetics, Radiation Epidemiology Branch
Mary White, ScD	Chief, Epidemiology and Applied Research Branch	Centers for Disease Control and Prevention
CMS & CATA Representatives
Janis Grady	Project Officer	Centers for Medicare & Medicaid Services
Marie Hall	CATA Team	Health Services Advisory Group
UC Team
Rebecca Smith-Bindman, MD	Principal Investigator	University of California, San Francisco
Andrew Bindman, MD	Principal Investigator	University of California, San Francisco
Patrick Romano, MD, MPH	Co-Investigator	University of California, Davis
Naomi López-Solano, CCRP	Project Manager	University of California, San Francisco
Diana Ly, MPH	Project Manager	University of California, San Francisco

Technical Expert Panel Meeting

Prior to the meeting, TEP members received a copy of the agenda, presentation slides, link to DR-CTQS study website which contains minutes from the prior TEP meetings, honorarium documentation, and a conflict of interest form. The meeting was conducted with the use of PowerPoint slides.

9:00 AM           Call meeting to order by TEP Chair                                Dr. Helen Burstin

Dr. Helen Burstin called the meeting to order. She noted that the meeting will last for three hours with a break at the halfway point and will include a discussion period after each presentation.

9:05 AM           Roll Call and Updated Conflicts                                       Dr. Helen Burstin

TEP Members and Ex Officio members attendance listed above.

Conflict of interest defined as you, your spouse, your registered domestic partner, and/or your dependent children:

1. received income or payment as an employee, consultant or in some other role for services or activities related to diagnostic imaging

2. currently own, or have held in the past 12 months, an equity interest in any health care related company which includes diagnostic imaging as a part of its business

3. hold a patent, copyright, license or other intellectual property interest related to diagnostic imaging

4. hold a management or leadership position (i.e., Board of Directors, Scientific Advisory Board, officer, partner, trustee, etc.) in an entity with an interest in diagnostic imaging

5. received and cash or non-cash gifts from organizations or entities with an interest in diagnostic imaging

6. received any loans from organizations or entities with an interest in diagnostic imaging

7. received any paid or reimbursed travel from organizations or entities with an interest in diagnostic imaging

COIs were disclosed to UCSF prior to this TEP meeting via paperwork. No members had new financial conflicts that precluded their participation. TEP members were also asked to verbally disclose any COIs when introducing themselves for the purpose of group transparency. TEP members re-stated their affiliations and any existing conflicts. Dr. Helen Burstin stated her affiliation as the CEO of the Council of Medical Specialty Societies. She is now on the board of the Society to Improve Diagnosis in Medicine, although this is not a conflict of interest. Dr. Jay Bronner stated no new conflicts of interest. Dr. Jeph Herrin stated his affiliation with Yale University, and no new conflicts of interest. Dr. Matthew Nielsen reported his affiliation with the University of North Carolina. He noted he is the Quality Improvement Chair at the American Urological Association, however this association is not directly related to imaging. Dr. Debra Ritzwoller stated her affiliation with Kaiser Permanente Colorado and as a patient/guardian stakeholder. Dr. Kenneth Wang noted his affiliation with the Veterans Administration in Baltimore and University of Maryland. Of note, he is participating on his personal time not representing government. His conflicts include a small start-up and occasional reimbursements from Radiology Society of North America. Dr. Mary White reported her affiliation with the Centers for Disease Control & Prevention and had no new conflicts of interest. Dr. Arjun Venkatesh reported no updates to conflicts of interest, but reminded the group that he works under contract with CMS for the development of hospital quality measures and quality rating systems, and also leads quality measure development for the American College of Emergency Physicians. Niall Brennan stated that he had no new conflicts and that he is currently the President and CEO of the Health Care Cost Institute. Dr. Hedvig Hricak is currently the Chair of the Memorial Sloan Kettering Cancer Center Department of Radiology. She disclosed her current conflict as a board member of IBA. Dr. Mythreyi Chatfield stated her affiliation with the American College of Radiology, as the Executive Vice President of Quality and Safety, and had no new conflicts of interest to disclose. Tricia Elliot restated her role as the Director of Quality Measurement at The Joint Commission, and no new conflicts of interest. Dr. Leonard Lichtenfeld reminded the panel of his role as the Interim Chief Medical Scientific Officer of the American Cancer Society. He did not have any conflicts but mentioned his stock ownership in Google and noted that they have some interest in using augmented intelligence in radiology analytics. Dr. Lewis Sandy stated his affiliation with UnitedHealth Group as the Executive Vice President of Clinical Advancement, and had no new conflicts of interest to disclose. Dr. Amy Berrington restated her role as the Branch Chief and Senior Investigator in the Radiation Epidemiology Branch at the National Cancer Institute, and had no new conflicts of interest to disclose. Suzanne Schrandt restated her role as the Director of Patient Engagement at the Arthritis Foundation, and did not have new conflicts to disclose. Finally, Missy Danforth restated her role as the Vice President of Health Care Ratings at the Leapfrog Group, and had no new conflicts to declare.

9:15 AM         Measure Calculation & Reporting                                    Dr. Andrew Bindman

The presentation began with a review of the measure concept to identify diagnostic CT scans that are performed in an unsafe manner, with a balancing measure to prevent compromised image quality or excessive dose. In terms of measurement, the level of analysis will be done on each individual CT scan. CT scans will be bundled to the level of the individual clinician, or to the associated clinician-group. Each CT scan will be put into a category combining the anatomic area scanned with the indicated reason for study so as to account for the varying dosage depending on clinical need for high quality images. After scans are categorized, they will be judged on image adequacy and dose appropriateness to determine the failure rate. There will be two derived components that go into this assessment: radiation dose and image quality. Radiation dose will be compared against a threshold specified by anatomic area and clinical indication, as well as adjusted for patient size. Image quality will be compared against a threshold of calculated image noise, or compared against an artificial intelligence (AI) assessment of image quality adequacy. Setting of the thresholds will be driven by current clinical guidelines, evidence from the UCSF Dose Registry, and results from the DR-CTQS Image Quality Study. Currently there are 15 specific variables that contribute to the measure. These variables are either directly extracted or derived from combination of extracted variables from multiple sources including billing codes and diagnoses in claims data, PACS CT-scan RDSR, and PACS CT-scan images.

Next the presentation moved into reviewing potential methods for how the measure will be reported to CMS. The first option entails the reporting clinician/clinician-group installing publicly available software, developed by UCSF within its IT local environment. This software would have the capability of calculating the measure and enabling the clinician or clinician-group to self-report to CMS with or without the assistance of a 3^rd party. The second option entails the clinician or clinician-group submitting all CT scans and associated billing code data to a registry. The registry will then analyze the data in order to calculate the failure rate and would report to CMS. If the registry approach is utilized, it is anticipated that the image data would be used to calculate the failure rate, but once the necessary calculations are completed, the registry will retain limited data, or no data at all. The size of the image files makes it impractical to retain all of them over time.

9:30 AM         Discussion: Measure Calculation & Reporting                                  Dr. Burstin

Discussion opened with talk of prospective challenges of the software required for retaining in a registry. TEP members reported on feedback that they had elicited from colleagues at their institutions that any software associated with the measure not be maintained on servers local to the imaging sites. A TEP member from a large radiology practice noted that the practice sites associated with his practice expressed anxiety about having software foreign to their institution interacting with patient imaging data. As a result TEP members expressed a strong preference for a registry model. TEP members identified the process of linking multiple data sources (claims, PACS, etc.) using protected health information (PHI) then de-identifying the data prior to its transmission to a registry could present some technical challenges. Concerns were expressed about the ability to scale the bandwidth required to maintain and grow an image registry. TEP members put forth the idea of seeking public comment on the approach of reporting.

Another concern voiced by some TEP members is that although the measure’s intended use is to assess the quality of clinicians, the data needed to calculate the measure are often under the control and ownership of the hospital which maintains the data systems for hospital-based clinicians whose failure rate would be assessed by the measure. One TEP member pointed out the changing attitudes of hospitals; hospitals are beginning to view their data as a valuable asset, and they may not be willing to share the data for this purpose. It was also mentioned that many hospitals already receive several hundred data sharing requests on a monthly basis, and the proposed radiology measure could be a source of excessive burden if it also has an additional level of difficulty related to data security and governance. One TEP member commented that imaging centers may be more open to implementing the measure, as they have less of a stake in the asset value of their data. A panelist expressed knowledge of a hospital that was very interested in our measure for the value that it could bring to their patients.

Another concern that was expressed was about implementing the measure at the level of the individual clinician. There was concern about the reliability of reporting this measure at the individual clinician level. It was also stated that it might be a burden for an individual clinician to do all that is required. Participants suggested an option might be to apply this measure at the clinician-group TIN level. It was also suggested that applying this measure at the level of the facility might improve the reliability of the measure and ease the burden of reporting.

The UC Team acknowledged these concerns and will assess these as a part of the beta testing. If these concerns are found to be valid then the UC Team may recommend to CMS to apply this measure at the clinician-group level defined by a minimum size at the TIN level. The UC team will also explore whether the CMS testing program focused on facility level quality could also support and adopt the measure.

9:50 AM         Update on Alpha Testing                                      Dr. Rebecca Smith-Bindman

Dr. Smith-Bindman reviewed the elements of what is being tested as part of alpha testing using UCSF Health billing data and UCSF International CT Dose Registry data. The Registry data are used to: 1) test application of inclusion and exclusion criteria, 2) categorize by anatomic area & clinical indication, 3) calculate dose within each category, 4) compare observed dose to expected dose, 5) evaluate the distribution of observed vs. expected dose to inform the construction of a failure rate. Alpha testing phase 1 only uses data from the Registry, while alpha testing phase 2 combines Registry data with the UCSF electronic health record data to support the assessment of whether the method the UC Team creates to put CT scans into categories for assessment is valid. These CT-categories (named CT-CAT by the UC Team) were obtained through literature review, empirical data from the UCSF Dose Registry, and the input and expertise of the TEP members. For example, one TEP member alerted the UC Team to a previously unidentified category of full body low dose CT for multiple myeloma. Dr. Smith-Bindman noted that TEP members had provided feedback indicating they believed that when there was a discrepancy between observed doses in the field and recommended doses from guidelines that TEP members felt the guidelines should be the basis for setting of dose thresholds. In the end, the UC Team is proposing 21 CT-CAT categories, some of which have different radiation thresholds for different clinical indications within anatomic areas.

Dr. Smith-Bindman reviewed data on the distribution of CT-CAT within the UCSF Registry. Within the UCSF Registry, 71% of all CT scans are in CT-CAT that would use routine doses for the designated anatomical area. Dr. Smith-Bindman believes there is potential to combine some CT-CAT. For example, the anatomical distinction between different parts of the spine may not be necessary as the observed doses for those scans are remarkably similar. Future decisions about whether to combine CT-CAT will be informed by the results of the image quality study and clinicians’ ratings of dose adequacy. The quality study will directly assess the image quality clinicians believe is required for different diagnostic tasks, and will be used to further refine the CT-CAT.

Dr. Smith-Bindman then turned to the topic of Alpha testing phase 2. In this phase of testing the UC Team is attempting to validate that the indications for CT exams based on claims data and diagnoses at the time the test was ordered are accurate. Accuracy is determined by comparing the proposed approach with more robust clinical data available within the UCSF Registry and from UC Health. For cases where there is disagreement, we will be conducting chart review to support a gold-standard determination of the clinical indication against which to judge the accuracy of the approach using claims and diagnoses at the time the CT scan was ordered. The results of this testing phase will be presented at the next TEP meeting.

10:05 AM       Discussion: Alpha Testing                                                         Dr. Helen Burstin

Discussion opened with a question regarding the source of the diagnostic codes used for determining clinical indication. Dr. Smith-Bindman clarified that the diagnostic codes would be gleaned from the clinician order rather than the diagnosis after the CT scan is performed. TEP members asked that the UC Team provide clarification on data sources moving forward. A TEP member asked the UC Team to provide the specific coding list and assessment of the completeness of the ICD-10 coding list. Dr. Smith-Bindman recommended that the question be deferred for discussion until the next TEP meeting, when the UC Team will have data to address it from testing and re-testing of the method on data from UCSF as well as external sites. The UC Team will share the accuracy of the approach for determining the CT-CAT and the accuracy tradeoffs involved with different algorithms. Dr. Romano also highlighted that the UC Team has an ICD-10 expert coder on the team to help with this process.

10:25 AM       Quick Recess                                                                                                                

10:35 AM       Beta Testing                                                           Dr. Rebecca Smith- Bindman

Dr. Smith-Bindman began by reminding TEP members that Beta testing is designed to determine whether the approaches tested within the UCSF Dose Registry are generalizable to a wider variety of settings as would be needed for CMS quality reporting. Beta 1 testing will be performed in 6 sites and will support an assessment of the proportion of CT scans that are included or excluded and to enable a comparison of the distribution of CT scan types and dose distribution in the CT-CAT with what is observed in the UCSF Registry. Beta 2 testing will expand to additional sites, incorporate risk adjustments based on patient size, and include calculations of global image noise to support the assessment of image quality or potentially AI assessments of quality. Beta 3 testing will expand to additional external sites, potentially incorporate AI assessments of image quality, and capture information on the burden of reporting. The assessment of burden will be done by surveying clinicians, capturing their experience of collecting and preparing the data for reporting. Beta 4 testing will include modifications from earlier testing steps and any additional modifications related to requested public comments.

Beta testing may include applying software within the local practice or may evolve over time to just a registry model, depending on input from TEP members and testing sites. The UC Team will begin testing with radiologist groups, and in later rounds of Beta testing, expand to include sites where the CT scans are performed by other clinical specialties. For example, the UC Team has identified a urology group interested in serving as a Beta testing site, and the UC Team welcomes advice from TEP members about other potential testing sites. The UC Team is also discussing the possibility of conducting testing in collaboration with one of the major radiation dose management software vendors in the United States.

10:55 AM       Discussion: Beta Testing                                                                       Dr. Burstin

TEP members asked about the role of non-radiologists during these testing phases. Dr. Smith-Bindman explained that a large proportion of CT scans are performed and sometimes interpreted, by non-radiologists. These include urologists, cardiologists, as well as a number of other specialties. Many of these specialties have their own CT scanners, and they bill to CMS, either for the technical fee alone, or for the total cost of the study. This moved into a discussion of who is held ultimately responsible for appropriate dosage: the clinician who interprets the CT scan or the personnel such as the technician at the facility performing the scan. There was a discussion about possible tension between those who perform the scans and those who read them, and who are to be responsible for the work of ensuring they are done in a safe manner. TEP members suggested investigating how many non-radiologists are billing to CMS for interpretation of CT scans. There was discussion but no clear consensus among TEP members as to whether the proposed radiation measure should be applied only to the clinician who interprets the scan or if the facility that performs the scan should also be held accountable. One TEP member suggested, and several TEP members were supportive, of an approach in which both the clinician and the facility share responsibility and any incentives.

Another TEP member raised a question of whether a different standard for judging performance should be applied to radiologists versus non-radiologists who read CT scans. The point was made that non-radiologists might need higher doses to make up for less expertise in reading CT scans. A TEP member representing patient interests touched upon the need to focus on what is best for the patient, i.e., the best dose possible regardless of the specialty of the clinician reading the scan.

The UC Team agreed with this perspective and noted that there are no data to support a recommendation for using different doses for the same test based on whether the clinician is a radiologist or a non-radiologist. The UC Team noted that some consideration was given to the idea of including non-radiologists in the image quality study but the UC Team decided this was impractical because non-radiologists would generally not be able to read scans outside of the narrow clinical area in which they furnish services.

11:15 AM       Measure Stewardship                                                          Dr. Patrick Romano

Dr. Romano began by describing the definition of a measure steward, as defined by both the NQF Glossary and the CMS Blueprint. Dr. Romano also provided an example of the agreement between NQF and the measure steward. The ongoing commitment of measure stewardship was emphasized. The measure steward for the radiation measure would need to anticipate maintaining support for the measure for at least 3 years after NQF endorsement. The UC Team is currently planning to submit the measure for endorsement in the 2021/2022 cycle, meaning that a steward would have to be prepared to sustain the measure three years beyond that time. Dr. Romano pointed out that the rapidly evolving technology in this field requires refinement of the dose thresholds, risk-adjustment parameter estimates, and the image quality assessment/AI tools. Dr. Romano stated that measures are most commonly stewarded by federal agencies such as CMS, CDA, VA, etc. State and regional health departments are also known to serve as measure stewards, as well as national accrediting and certification organizations. Business models for stewardship was then described. These include congressional appropriations to federal agencies, membership dues, accreditation or certification fees, philanthropic support, or user fees and service sales. Some business models that typically do not work for measures developed with public resources include: relying on internal resources of the measure developer, wishful thinking for grant support from donor foundations, or user fees. Dr. Romano identified potential opportunities and threats related to stewardship of the proposed radiation measure. First, CMS has stated that it will not be able to provide financial resources for measure stewardship after the project period ends. Opportunities to develop a business model may be hampered by any expectations that CMS has for making the software tools that support the measure publicly available at no cost to clinicians. He emphasized that resources would be needed to support the necessary technical expertise to maintain the measure over time. He pointed out that there might be a role for a specialty society such as the American College of Radiology to act as a measure steward, but that this might be problematic given there are likely to be clinicians not in the specialty of radiology who would be assessed on this measure and would not report through the ACR. Dr. Romano also raised the possibility of an accrediting organization to play the role as measure steward, but that it might be challenging to find an organization that does accrediting in all of the settings in which CT scans are performed.

Prior to the TEP meeting, the UC Team asked TEP members to let the UC Team know of their organization’s role in measure stewardship and reporting. Dr. Romano shared the results of the input. Out of the 19 TEP members, 9 responded to the UC Team’s inquiry. Four representatives reported that their organization would under certain circumstances consider the role of stewardship of the quality measure. These organizations were the Joint Commission, Radiology Partners, American College of Radiology, and University of Virginia.

11:30 AM       Discussion: Measure Stewardship                                                       Dr. Burstin

TEP members acknowledged the challenges of measure stewardship. One TEP member suggested a consortium model in which the relevant specialty societies would each financially support measure stewardship. It was also suggested that UC Team explore the potential of incorporating the measure into accreditation programs, such as the imaging accreditation program offered by the American College of Radiology (ACR). A TEP member representative of the ACR indicated that the ACR’s current accreditation program is focused on the technical component of imaging, not the clinician component which is being targeted by this measure. The ACR did not express interest in a measure stewardship role related to assessing clinician performance. Another idea raised was for CMS to incorporate the measure into required hospital level reporting. It was discussed that this would facilitate reporting. In many settings, such as when a radiologist works in a hospital, the clinician who would be responsible for reporting the measure does not control the information systems that contain the required data. In such cases, the clinician could face significant barriers to reporting. On the other hand, if the facility that controlled the information system were also held accountable for the quality of the CT scans, then the barriers for reporting at the clinician level would be reduced. Such an approach could also support the business case for maintaining the measure over time. When hospitals have been held accountable for other measures, they have typically contributed to registry fees which, were it to happen with the proposed radiology measure, it could help to support the maintenance of the proposed measure over time.  A TEP member representative from the Joint Commission said that if the measure was also applied at the hospital level, then there would be interest for the Joint Commission to consider a role in measure stewardship. TEP members emphasized once again the challenges of producing the measure at the individual clinician level, and encouraged the UC Team to focus on the clinician-group or facility level. There was discussion about using the measure to assess both the facility level failure rate and clinician-group level failure rate. Another TEP member recommended contacting other academics who have taken on the role of measure steward and learning from their experience. A question was raised about whether single or multiple registries might be used to support the measure. TEP members responded that it was most practical to rely upon a single registry to ensure that the measure is captured and reported in a consistent manner.

The UC Team expressed appreciation for the TEP members input. Based on the discussion the UC Team has not identified any organization that is currently prepared to take on the role of measure steward if it is only applied at the practitioner level. Furthermore TEP members have identified a risk that the data needed to calculate the measure may be controlled by hospitals and other institutions where many of the clinicians work. The UC Team plans to discuss this risk with CMS to see if there are ways to apply the measure not only at the clinician/clinician group level but also at a facility level to make it more likely that the necessary data will be made available. Another important consideration for building a business case for the measure has to do with whether or not the measure will be required or optional. The UC Team will discuss this with CMS and share what it learns at an upcoming meeting of the TEP in order to further refine a business case for measure stewardship.

11:50 AM       Wrap Up and Next Steps                                                                 Dr. Bindman

The UC Team thanked the TEP members for their active engagement. TEP members were informed that the UC Team would carefully consider all of the input and discuss suggestions with CMS sponsors. The TEP would be updated on progress at upcoming meetings. The next TEP meeting is anticipated to be in-person in the spring of 2020. TEP members were reminded of the process to submit a request for an honorarium.

12:00 PM        Meeting Adjourned                                                                   Dr. Helen Burstin