We hope you find the following FAQ informative and helpful. If you have questions we haven’t answered, please contact firstname.lastname@example.org. Please also download our “executive summary”: a short guide to ICHOM’s work and ambitions.
ICHOM was founded in 2012 by Professor Michael Porter of Harvard Business School,Martin Ingvar of the Karolinska Institute and the Boston Consulting Group. ICHOM’s mission is to unlock the potential of value-based health care by defining global Standard Sets of outcome measures that really matter to patients for the most relevant medical conditions and by driving adoption and reporting of these measures worldwide.
Outcomes are the results of treatment that patients care about most.
Outcomes are not “outputs”; they are not lab results; they are not technical details. They’re real-world results, like physical functioning or level of pain. How soon after treatment can a patient with low-back pain expect to return to work? How likely is a man to experience incontinence or sexual dysfunction after treatment for prostate cancer? These are questions about outcomes. Unfortunately, today, in health care systems around the world, evaluation efforts take into account a number of clinical indicators, structural metrics, and even reputation – but they tend to ignore outcomes.
- patients are able to choose providers based on informed expectations of outcomes and the associated costs;
- providers that deliver superior outcomes at competitive costs thrive, while others improve or lose market share;
- payers negotiate contracts based on results and encourage innovation to achieve those results; and
- suppliers succeed by marketing their products on value, showing improved patient outcomes relative to costs.
In addition to presentations by our cofounders, we have had patient advocates, as well as representatives of leading payer and provider organizations. In 2013, our conference featured Tom Lee (Press Ganey), John Rumsfeld (Veterans Health Administration), and Tim Kelsey (NHS England). Our 2014 conference featured Patrick Conway (CMS), Rich Lesser (BCG), Martin Makary (Johns Hopkins), Edith Schippers (Minister of Health-Netherlands), and Bernard J. Tyson (Kaiser).
The next ICHOM conference will take place at University College London on May 16-17, 2016. Click here to learn more and register!
ICHOM is a nonprofit organization. Our funding comes from our cofounder organizations, our Sponsoring Partners, our Strategic Partners, and our individual donors. Our Sponsoring Partners include leading providers, such as Boston Children’s Hospital; payers, such as Harvard Pilgrim; and patient advocacy organizations, such as the American Heart Association and Bowel Cancer Australia. In 2015, ICHOM has forged Strategic Alliances with five health care pioneers:
- Erasmus University Medical Center in the Netherlands,
- the Agency for Clinical Innovation in Australia,
- Aneurin Bevan in the United Kingdom,
- the Dutch hospital group Santeon, and
- the multinational hospital group Ramsay Health Care.
ICHOM Strategic Partners are committed to support our standardization efforts and to implement one or more of the Standard Sets. In addition, they will be among the first participants in the ICHOM global benchmarking program, which we are now building, and which we will showcase at our 2016 conference.
For a complete list of our Strategic Partners and Sponsoring Partners, click here.
ICHOM does not produce educational content for patients. However, we recommend UptoDate (www.uptodate.com), which offers free, vetted materials. For literature about shared decision-making, while we don’t have a particular recommendation, you may wish to consult the Center for Shared Decision Making at Dartmouth, which offers some useful sample materials (http://med.dartmouth-hitchcock.org/csdm_toolkits.html), and/or the Informed Medical Decisions Foundation, which has additional content (http://www.informedmedicaldecisions.org/what-is-shared-decision-making/shared-decision-making-resources/).
The National Quality Forum (NQF) is a public-private partnership in the United States founded in 1999 to improve the quality of American health care. To date, NQF’s primary focus has been on endorsing measures for accountability, meaning those deemed valid for inclusion in pay-for-reporting programs or public transparency programs. For measures to be endorsed by the NQF, a rigorous process demonstrating performance variation as well as adequate risk-adjustment is required, among other things. As many ICHOM measures are patient-reported outcomes, on which performance variation is relatively unstudied and risk-adjustment undeveloped, these measures will require time before those steps can be fulfilled and the measures endorsed. Although we agree with the NQF in the long-term goal of public reporting and payment linked to outcomes, we see great value in first measuring for improvement and are focused there at present.
The concept of disease burden is used by the World Health Organization and other public health agencies to measure the overall health impact of a disease or risk factor (see here). It is typically measured in terms of disability-adjusted life years (DALYs), a composite of the years of life lost from a disease and the years of imperfect health living with a disease. Diseases with high mortality rates, such as coronary artery disease, and those with prolonged rates of disability but low mortality rates, such as low back pain, have high impact on DALYs and thus high measured disease burden.
Recently, the Institute for Health Metrics and Evaluation (IHME), based in Seattle, Washington, has taken a lead role in improving and expanding the measurement of global disease burden. As part of this effort, they have released the Global Disease Burden data visualization tool, which allows users to understand the disease burden of particular conditions. We use this tool and the underlying data, which have generously been shared with ICHOM by the IHME, to estimate the disease burden of the conditions we have covered. It is important to note that our disease burden estimates are for developed countries, as that is the primary focus of our work.
ICHOM prioritizes the development of new Standard Sets based on three primary factors: burden, clinical engagement, and funding. In general, conditions with higher disease burden receive higher priority given the opportunity for greater impact once implemented. Clinical engagement refers to the availability of respected clinical leaders who are eager for such a Standard Set, are willing to help develop it, and will champion its adoption globally. Finally, funding to cover the development cost of a Standard Set is necessary to proceed.
We require 150,000 USD to develop an ICHOM Standard Set. This amount covers the staff to run the project (a full-time ICHOM Project Leader, Standardization Associate, and Research Fellow), management oversight by ICHOM leadership, and marketing and travel expenses. Funding can be sourced by a single institution or by multiple institutions combining together.
To avoid potential conflicts of interest in recommending outcome indicators that bias toward specific products, we do not include life science representatives in our Working Groups or as direct funders. However, we recognize that life science firms have an important role to play in enabling value-based health care, with ICHOM Standard Sets as an enabling force. For this reason, we will be rolling out an open review period for all stakeholders, including medical technology and life sciences representatives, to comment on our Standard Sets, starting with those conditions finishing in the fourth quarter of 2015. We also welcome medical technology and life science firms to engage with specialty societies or respected patient advocacy organizations to contribute funding that might then be used to support ICHOM’s work at arm’s length.
After a Standard Set has been developed, a subset of the Working Group members continue their involvement with ICHOM and the Standard Set via a Steering Committee, or “SteerCo.” Steering Committee members drive promotion and adoption of their Standard Set and govern its direction by reviewing and approving revisions. SteerCo members also oversee and approve the usage and analysis of data collected in accordance with the Standard Set. This is informed by the implementation of the Standard Set in myriad settings as well as changes in the processes and tools used to measure outcomes. Finally, every SteerCo is charged with advising ICHOM on our global-comparisons effort.
The unit of analysis is “what” or “who” is being analyzed. ICHOM Standard Sets have several units of analysis. These include the individual patient, groups or populations of patients (as defined within the scope of each Standard Set), provider organizations, provider networks, health care plans, and individual health care professionals.
Patients are involved in the development of our Standard Sets in three primary ways.
- Patient Focus Groups: To help develop the list of outcome domains that will be included in the Standard Set, an international group of patient and care representatives with experience in the relevant medical condition independently compose a list of outcome domains that have mattered most to them. This work lays the groundwork for the standardization process.
- Patient Validation Surveys: Once the list of outcome domains has been finalized through a systematic literature and registry review, and reviewed in patient focus- group discussions and Working Group teleconferences, a survey to validate these outcomes is distributed to large international networks of patients and caregivers with expertise in the relevant medical condition. These networks provide further input on the outcome domains until the Standard Set is published.
- Open Review Period: Once the outcome measures have been finalized, patient charities and advocacy organizations are invited to provide input on the Standard Set.
We also engage patient charities and advocacy organizations in our wider mission to advance a patient-centered approach to health care through patient-reported outcomes measurement.
Each Standard Set is made up of the following components.
- Outcomes:These are the patient-centered outcomes that represent true success in managing the specified medical condition.
- Case-mix variables are factors that affect patient outcomes, but which we cannot control as part of the management of the condition. We measure these to build risk-adjustment models that ensure fair comparison of outcomes across centers.
- Measurement tools are the validated instruments that are used to measure the outcomes and case-mix variables.
- Data sources: These can be administrative, clinician-reported or patient-reported.
- Time points are the specific points during and after treatment that data are collected.
At the conclusion of the Working Group process a manuscript explaining the process to arrive at the Standard Set and motivation for selected measures is submitted to peer-reviewed academic journals for publication. Please visit any of our completed conditions on our Standard Sets page for links to currently available publications.
ICHOM has had over 400 Working Group members. The Working Group is composed of patients, patient advocates, registry leaders, and physician leaders representing all specialties involved in caring for a particular condition. These members come from multiple cultures and continents, truly embodying a team with a diverse perspective.
PROMs are thoroughly researched and then selected based on the following five criteria.
- Coverage of outcome domains of importance
- Psychometric Quality
- Feasibility/burden of assessment
- Financial/licensing requirements
- Established in the field/locations in use/number of translations
With respect to criterion #2, Psychometric Quality, we follow the ISOQOL minimum standards for PROM measures (See Reeve et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res (2013) 22:1889–1905).
No. We research the PROMs that are available in the field, per condition. The recommended PROM is selected based on the criteria listed in the previous question. When there are few or no PROMs available, we may recommend separate, patient-reported, single-item questions that have been used in registries; PROMs that have been used in a research setting; or, defer to clinical/administrative information as a substitute. For example, in the Lung Cancer Standard Set, clinical/administrative data are used for “location of death” and “number of days in hospital in last 30 days of life” to collect Quality of Death.
After a Standard Set is finalized, it is made available on ICHOM’s website in the form of a Flyer and a Reference Guide. The Flyer contains an overview of the recommended outcome domains, as well as information about the sponsors of the Standard Set and the Working Group members. The Reference Guide is a detailed document that describes all of the Standard Set’s domains, measures, and case-mix factors. A large part of the Reference Guide is the Data Dictionary, designed to help interested providers to measure the ICHOM Standard Sets as consistently as possible according to the Working Group recommendation. Additionally, the Standard Set is published in academic journals, to communicate the work to the medical community. The next step after finalizing the Standard Set is implementation.
ICHOM Standard Sets are developed by international Working Groups of leading clinicians, outcomes researchers, registries leaders, and patient advocates. Members of the Working Group are selected by ICHOM based on their expertise and meet via teleconference over a period of about nine months to develop a Standard Set.
The Working Group is lead by the Project Team, which performs background research and develops proposals. These proposals are presented to the larger Working Group for discussion, refinement, and ratification.
Finally, representatives from key stakeholders such as implementing provider organizations, patient advocacy groups, payers, or governments form a Reference Group. This group provides feedback to the Working Group at key points in the Standard Set development but has no formal decisional role in the process.
The relationship between these groups is shown below.
The schematic below depicts the structured process used to develop ICHOM Standard Sets. The Working Group meets for a series of 8 teleconferences shown in the top row. Each teleconference focuses on one aspect Standard Set development. For example, the first teleconference focuses on the scope of the Standard Set. The following teleconference features a discussion of the outcome domains to include in the Standard Set.
The Project Team creates proposals and presents supporting information to guide each teleconference discussion. Project Team proposals are developed by synthesizing information from the literature (grey), input from patient representatives (blue), and advice from external experts (orange). For example, in preparation for Call 1, the Project Team performs a systematic literature review to determine the outcome domains currently measured for a particular medical condition and facilitates a patient focus group to understand the outcomes that are most important to patients with that condition. It then presents this information to the Working Group in Call 1 for discussion.
Following each teleconference, the Project Team’s proposal is presented to the Working Group for voting via on-line survey. This may either be a single round of voting in which Working Group Members are asked if they agree or disagree with each item in a proposal (grey arrows) or a 2-round modified Delphi voting process (orange arrows). In the modified Delphi process, Working Group Members are asked to rank potential outcome domains or case-mix variable domains by their importance to include in the Standard Set. For a domain to be included in the Standard Set it must be highly ranked by at least 70% of respondents. To be conclusively excluded from the set, it must be ranked poorly by 70% of respondents. Domains not ranked conclusively after round 1 are represented for voting in round 2. All proposal items must be agreed upon by at least 70% of Working Group Members to be included in the final Standard Set.
ICHOM Standard Sets include a combination of administrative, clinical, and patient-reported data. A number of different models are currently being employed to capture these data. Common methods include building data collection into an existing electronic health record (EHR) system, manual chart abstraction, or a combination of the two. To collect patient-reported outcomes (PROs), many providers use simple pen-and-paper solutions, while others have contracted with one of ICHOM’s Certified IT Suppliers to measure PROs electronically.
Measuring outcomes is a complex and long-term endeavor. Implementation can take a year or more and require additional manpower, technology, and resources. Therefore, it is critical that there is commitment and buy-in at all levels of the organization before beginning. For more information on the resources required to implement and measure ICHOM Standard Sets, please contact us at email@example.com.
Yes, all ICHOM Standard Sets are available online at no cost to registered ICHOM.org users. The Reference Guides that accompany each Standard Set include all the necessary information to implement on your own, including a detailed data dictionary for those who may be interested in sharing, comparing, or benchmarking their outcomes with other institutions measuring the same Standard Set(s).
Every provider should try to measure every outcome for every patient that he or she treats. ICHOM Standard Sets are designed specifically as “minimum sets,” meaning that they include only the most essential outcomes of a given medical condition. Adopters of the Standard Sets may well choose to continue to track various process metrics or even additional outcomes, but we encourage them to collect the Standard Sets – in their entirety – as a starting point. Recognizing the challenge of getting started with outcomes measurement, the adoption of the Standard Set can of course be phased over time, but your ambition should be to measure the entire Set in the target state.
My hospital is very small and resource-constrained, but we want to start measuring. How do we get started?
Although we believe that routine outcomes measurement should be the standard of care, many organizations find that the easiest way to get started by securing a research grant to cover at least a portion of the startup costs. Starting small, with only a few physicians or a sampling of patients, is another method of keeping costs down and ensuring smooth integration of the program into clinical workflow.
Can the Standard Sets be measured in middle-income countries and resource-poor settings? Are there examples of this?
Yes, ICHOM Standard Sets are currently being measured in India and Malaysia. While hospitals and clinics in middle-income countries and resource-poor settings may face additional challenges, the measure of success in health care is remarkably consistent around the world.
There is no set cost for implementing an ICHOM Standard Set. Common factors affecting cost of implementation include IT infrastructure, patient volume, and the method of data collection.
We think of the next steps in three levels. The first is to ensure that the data you generate is reported back to the clinicians in your teams. This alone will lead to valuable insight and help drive improvement within your practice or organization. The second level is to start comparing, when possible, inside your country and region. You will likely find that in some outcome domains you and your teams are performing among the best, while in others there may be room to improve. The third level is to take part in ICHOM’s Global Comparison project, which we are currently developing and which we believe is the best way to accelerate learning – and improvement – around the world.
We do not ask providers to share their data with us. ICHOM is not a registry. We believe, however, that there is tremendous value generated by providers sharing risk-adjusted data with one another. That’s why ICHOM is currently working to set up an infrastructure to advance comparison, not only within organizations, but also across countries. Still, even if you and your colleagues do not choose to share data, we encourage you to use the Standard Sets to measure your own performance.
Implementation can move more quickly or more slowly depending on how aggressively you want to invest in beginning to measure your outcomes and what your starting point is. ICHOM believes that most institutions should be able to implement within 6 months to a year, but some implementation efforts may go even faster than that.
ICHOM broadly supports Standard Set implementation by sharing knowledge, success stories, and implementation resources on our blog and in our online library. These materials are available at no cost to all registered ICHOM.org users. For more focused support, ICHOM organizes Implementation Communities (collaboratives) and works individually with institutions to build internal capacity to measure and improve outcomes. For pricing and availability, please contact us at firstname.lastname@example.org.
Since ICHOM Standard Sets are publicly available and do not require a license, it is difficult to track precisely. However, we know of approximately 50 institutions and 14 registries currently implementing or measuring at least one ICHOM Standard Set. Over 120 more have expressed interest and hope to begin implementing soon.
An ICHOM Implementation Community is a group of value-oriented institutions that are moving through the implementation process together for a given medical condition. These groups of like-minded institutions, brought together by ICHOM, support each other by discussing best practices, key challenges, and lessons learned. An example is the Cleft Lip and Palate Community, in which ten organisations representing eight countries are collaborating to implement the complete Standard Set.
ICHOM is currently working on a mechanism for connecting institutions that are measuring Standard Sets at their institutions, as communication and knowledge sharing are key factors to individual and international success. While this mechanism is under development, please contact us at email@example.com if you would like to be connected with other measuring partners.
ICHOM likes to feature institutions that have implemented (or are implementing) on our blog or in longer case studies and video documentaries. If you are interested in sharing your story, please contact us at firstname.lastname@example.org.
We do not yet have examples of payers specifically using ICHOM Standard Sets for payment models, although we are currently working with a number of organizations toward this goal.