Frequently-asked Questions | ICHOM – International Consortium for Health Outcomes Measurement

Frequently-asked Questions

Frequently-asked Questions

We hope you find the following FAQ informative and helpful. If you have questions we haven’t answered, please contact ichomteam@ichom.org. Please also download our “executive summary”: a short guide to ICHOM’s work and ambitions.

General

Why and when was ICHOM founded?

ICHOM was founded in 2012 by Professor Michael Porter of Harvard Business School,Martin Ingvar of the Karolinska Institute and the Boston Consulting Group. ICHOM’s mission is to unlock the potential of value-based health care by defining global Standard Sets of outcome measures that really matter to patients for the most relevant medical conditions and by driving adoption and reporting of these measures worldwide.

What are outcomes?

Outcomes are the results of treatment that patients care about most.

Outcomes are not “outputs”; they are not lab results; they are not technical details. They’re real-world results, like physical functioning or level of pain. How soon after treatment can a patient with low-back pain expect to return to work? How likely is a man to experience incontinence or sexual dysfunction after treatment for prostate cancer? These are questions about outcomes. Unfortunately, today, in health care systems around the world, evaluation efforts take into account a number of clinical indicators, structural metrics, and even reputation – but they tend to ignore outcomes.

What is value-based health care?

Value is defined as the outcomes that patients experience relative to the cost of delivering those outcomes. Value-based health care, or VBHC, is health care that delivers the best possible outcomes to patients for the lowest possible cost. We believe that choice and competition in health care should be based on value. By restructuring care-delivery around outcomes, and promoting superior outcomes with financial incentives, health care systems will improve quality and curb inefficiencies. This will benefit every stakeholder across the health care spectrum. In a value-based world,
  • patients are able to choose providers based on informed expectations of outcomes and the associated costs;
  • providers that deliver superior outcomes at competitive costs thrive, while others improve or lose market share;
  • payers negotiate contracts based on results and encourage innovation to achieve those results; and 
  • suppliers succeed by marketing their products on value, showing improved patient outcomes relative to costs.

Who has spoken at ICHOM's recent conferences?

In addition to presentations by our cofounders, we have had patient advocates, as well as representatives of leading payer and provider organizations. In 2013, our conference featured Tom Lee (Press Ganey), John Rumsfeld (Veterans Health Administration), and Tim Kelsey (NHS England). Our 2014 conference featured Patrick Conway (CMS), Rich Lesser (BCG), Martin Makary (Johns Hopkins), Edith Schippers (Minister of Health-Netherlands), and Bernard J. Tyson (Kaiser).

The next ICHOM conference will take place in Washington, DC on October 25-26, 2017. Click here to learn more and register!

How is ICHOM financed?

ICHOM is a nonprofit organization. Our funding comes from our cofounder organizations, our Sponsoring Partners, our Strategic Partners, and our individual donors. Our Sponsoring Partners include leading providers, such as Boston Children’s Hospital; payers, such as Harvard Pilgrim; and patient advocacy organizations, such as the American Heart Association and Bowel Cancer Australia. In 2015, ICHOM has forged Strategic Alliances with five health care pioneers:

ICHOM Strategic Partners are committed to support our standardization efforts and to implement one or more of the Standard Sets. In addition, they will be among the first participants in the ICHOM global benchmarking program, which we are now building, and which we will showcase at our 2016 conference.

For a complete list of our Strategic Partners and Sponsoring Partners, click here.

Does ICHOM produce educational content for patients?

ICHOM does not produce educational content for patients. However, we recommend UptoDate (www.uptodate.com), which offers free, vetted materials. For literature about shared decision-making, while we don’t have a particular recommendation, you may wish to consult the Center for Shared Decision Making at Dartmouth, which offers some useful sample materials (http://med.dartmouth-hitchcock.org/csdm_toolkits.html), and/or the Informed Medical Decisions Foundation, which has additional content (http://www.informedmedicaldecisions.org/what-is-shared-decision-making/shared-decision-making-resources/).

Are ICHOM measures endorsed by the National Quality Forum in the US?

The National Quality Forum (NQF) is a public-private partnership in the United States founded in 1999 to improve the quality of American health care. To date, NQF’s primary focus has been on endorsing measures for accountability, meaning those deemed valid for inclusion in pay-for-reporting programs or public transparency programs. For measures to be endorsed by the NQF, a rigorous process demonstrating performance variation as well as adequate risk-adjustment is required, among other things. As many ICHOM measures are patient-reported outcomes, on which performance variation is relatively unstudied and risk-adjustment undeveloped, these measures will require time before those steps can be fulfilled and the measures endorsed. Although we agree with the NQF in the long-term goal of public reporting and payment linked to outcomes, we see great value in first measuring for improvement and are focused there at present.

What is the legal status of the US organization ICHOM?

ICHOM is a US non-profit organization, organized under Internal Revenue Service Code Section 501(c)(3). There are no shares, dividends or similar that could work as vehicles for financial profit being paid out from ICHOM. No individual, organization, company or similar is gaining financial profit from ICHOM.

What is the legal status of the UK branch of ICHOM?

ICHOM has a UK establishment which is located in London and is registered with Companies House in the UK. This is not a separate entity to the US organization. There are no shares, dividends or similar that could work as vehicles for financial profit being paid out from ICHOM in the UK. No individual, organization, company or similar is gaining financial profit from ICHOM in the UK.

In addition to the above establishment, ICHOM has set up a separate legal entity in the UK. This is called ICHOM Ltd and is a non-profit organization limited by guarantee. This organization is used to receive grants from EU institutions. There are no shares, dividends or similar that could work as vehicles for financial profit being paid out from ICHOM Ltd in the UK. No individual, organization, company or similar is gaining financial profit from ICHOM Ltd in the UK.

Is any individual, organization, company or similar able to access or use information from ICHOM, to gain financial profit?

ICHOM’s Standard Sets are published on its web site for all to see and benefit. Whether anyone in the public gains financial profit from that is up to how they use the Standard Sets.

What is the legal relationship specifically between ICHOM and the three founding organizations and the individual representatives from these organizations?

The three founders of ICHOM are Michael Porter, personally, of Harvard Business School; Martin Ingvar, personally, of the Karolinska Instituet; and Boston Consulting Group (BCG). Michael Porter and Martin Ingvar serve on the Board, along-with Stefan Larsson representing BCG. The named organizations and associated individuals do not gain any type of financial profit from any branch of ICHOM.

Standardization

What is the percentage of disease burden that ICHOM Standard Sets cover based on?

The concept of disease burden is used by the World Health Organization and other public health agencies to measure the overall health impact of a disease or risk factor (see here). It is typically measured in terms of disability-adjusted life years (DALYs), a composite of the years of life lost from a disease and the years of imperfect health living with a disease. Diseases with high mortality rates, such as coronary artery disease, and those with prolonged rates of disability but low mortality rates, such as low back pain, have high impact on DALYs and thus high measured disease burden.

Recently, the Institute for Health Metrics and Evaluation (IHME), based in Seattle, Washington, has taken a lead role in improving and expanding the measurement of global disease burden. As part of this effort, they have released the Global Disease Burden data visualization tool. This tool allows users to easily understand the disease burden of particular conditions. We use this tool and the underlying data, which has been generously given to us by the IHME, to estimate the disease burden of the conditions we have covered. It is important to note that our disease burden estimates are for developed countries, as that is the primary focus of our work.

How does ICHOM decide what Standard Sets to develop?

ICHOM prioritizes the development of new Standard Sets on three primary factors: burden, clinical engagement, and funding. In general, conditions with higher disease burden receive higher priority given the opportunity for greater impact once implemented. Clinical engagement refers to the availability of respected clinical leaders who are eager for such a Standard Set, are willing to help develop it, and will champion its adoption globally. Finally, funding to cover the development cost of a Standard Set is necessary to proceed. To view a list of upcoming projects, please visit http://www.ichom.org/medical-conditions.

How much funding is required to develop a Standard Set?

We require $150,000 USD to develop an ICHOM Standard Set. This amount covers the staff to run the project (a full-time ICHOM Project Leader, Standardization Associate, and Research Fellow), management oversight by ICHOM leadership, and marketing and travel expenses. Funding can be sourced by a single institution or by multiple institutions combining together.

Notably, the cost to use our work is free. To encourage widespread awareness and adoption of our Standard Sets, we make publically available all Reference Guides, data dictionaries, gap analysis tools and publications.

Can life science firms fund ICHOM Standard Sets?

To avoid potential conflicts of interest in recommending specific outcome indicators that bias towards specific products, we do not include life science representatives in our Working Groups or as direct funders. However, we do recognize that life science firms have a strong role to play in enabling value-based health care. For this reason, we hold an open review period in which all stakeholders, including medical technology and life sciences representatives, can comment on our Standard Sets while they are still in development.

How does ICHOM plan to update Standard Sets over time?

After a Standard Set has been developed, a sub-group of the Working Group continue their involvement with ICHOM and the Standard Set by joining the Steering Committee, or ‘SteerCo’.  Their role involves:

  • Driving the promotion and adoption of the Standard Set, and developing strategies for expanding its implementation.
  • Providing governance and direction for the Standard Set over time, by reviewing and approving any revisions, and overseeing data usage and analysis. Revisions to the set may be informed by implementation work at various organizations, including those that are part of ICHOM Implementation Communities and ICHOM on-site capacity-building support programs. Developments in outcome measurement, such as new tools or updated versions of tools may also prompt revisions to the set.
  • Advising on the development of analytic plans for global comparisons and building a global community.

What is the unit of analysis of ICHOM Standard Sets?

The unit of analysis is the major entity being analyzed – the ‘what’ or the ‘who’. For ICHOM’s Standard Sets there is more than one unit of analysis, including the individual patient, groups or large populations of patients (as defined within the scope of each Standard Set), provider organizations, provider networks, healthcare plans and individual healthcare professionals.

How are patients involved with ICHOM?

We involve patients in the development of our Standard Sets using three methods:

  1. Patient Advisory Groups: To help develop the list of outcome domains that will be included in the Standard Set, an international group of patient, parent or carer representatives associated with the relevant medical condition independently compose a list of outcome domains that have mattered most to them. This then feeds into the Working Group process.
  2. Patient Validation Surveys: Once the list of outcome domains has been finalized through a literature and registry review, patient advisory group discussion and speciality/methodologist Working Group teleconferences, it is distributed to large international networks of patients, parents or carers associated with the relevant medical condition. These networks provide further input on the outcome domains within the Standard Set until it is published.
  3. Open Review Period: Once the outcome measures have been finalized, patient charities and advocacy organizations are invited to provide input on the Standard Set.

We also engage patient charities and advocacy organizations in our wider mission – of supporting a patient-centered approach to health care through patient-reported outcomes measurement.

What are the components of each Standard Set?

Each Standard Set is made up of the following components:

  1. Outcomes: The patient-centered outcomes that represent true success in managing the specified medical condition.
  2. Case-mix variables: Factors that will affect the outcomes above, but which we cannot control as part of management of the condition. We measure these to build risk-adjustment models that ensure fair comparison of outcomes across centres.
  3. Measurement tools: Validated instruments that are used to measure the outcomes and case-mix variables.
  4. Data sources: These can be administrative, clinician-reported or patient-reported.
  5. Time points: Specified time points for data collection.

Are there any publications based on ICHOM’s work?

At the conclusion of the Working Group process a manuscript explaining the process to arrive at the Standard Set and motivation for selected measures is submitted to peer-reviewed academic journals for publication. Currently, eight Standard Set articles have been published:

  • Localized Prostate Cancer
  • Coronary Artery Disease
  • Lower Back Pain
  • Cataracts
  • Advanced Prostate Cancer
  • Stroke
  • Cleft Lip & Palate
  • Advanced Macular Degeneration
  • Lung cancer
  • Hip and Knee osteoarthritis
  • Colon Cancer
  • Breast Cancer

These publications, along with a number of high-level publications on value-based health care and case studies highlighting the achievements at implementing hospitals are available at http://www.ichom.org/file-library/

How many WG members does ICHOM have? Who are they? Where do they come from?

The ICHOM Working Groups are composed of an ICHOM Project Team and approximately 15-20 Working Group members. The ICHOM Project Team consists of the Working Group Lead or Chair; a clinician with an international reputation and experience in the field, a Project Leader, Research Fellow, and a Standardization Associate. The Working Group members are composed of international volunteer representatives with a keen interest in outcomes measurement. They are clinical leaders and patient/carer advocates, representing many geographies and all specialties involved in caring for a particular condition. All Working Group Members participate on a voluntary basis. In total, members have come from 39 countries in 6 continents.

Based on what specific criteria are Patient Reported Outcome Measures (PROMs) selected?

PROMs are thoroughly researched and then selected based on the following criteria:

  1. Coverage of outcome domains of importance
  2. Psychometric Quality
  3. Feasibility/Burden of assessment
  4. Financial/Licensing requirements
  5. Established in the field/Locations in use/# translations

With respect to criterium #2, Psychometric Quality, we follow the ISOQOL minimum standards for PROM measures (Reeve et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res (2013) 22:1889–1905).

isoqol

Do you create your own measurement tools?

No. We research the PROMs that are available in the field, per condition. The recommended PROM is selected based on the criteria listed in the previous question. When there are few or no PROMs available, we may recommend separate, patient-reported, single-item questions that have been used in registries; PROMs that have been used in a research setting; or, defer to clinical/administrative information as a substitute. For example, in the Lung Cancer Standard Set, clinical/administrative data are used for “location of death” and “number of days in hospital in last 30 days of life” to collect Quality of Death.

Once the Standard Set is done, what happens then?

After a Standard Set is finalized, it is made available through the ICHOM website in the form of a Flyer and a Reference Guide. The Flyer contains a high level overview of the recommended outcome domains, as well as information about the sponsors and the Working Group members. The Reference Guide is a detailed document describing all domains and measures, as well as the case mix factors. A large part of the Reference Guide is the Data Dictionary, designed to help interested providers to measure the ICHOM Standard Sets as consistently as possible according to the Working Group recommendation. Additionally, the Standard Set is published in academic journals, to communicate the work to the medical community. Each Standard Set is reviewed and updated on a regular basis. The next step after finalizing the Standard Set is implementation and benchmarking.

Do you present your work at conferences?

Yes, the work is presented at conferences globally. Please check our social media pages and newsletter to see where we present our work.

Conferences

Who develops the ICHOM Standard Sets?

ICHOM Standard Sets are developed by international Working Groups of leading clinicians, outcomes researchers, registries leaders, and patient advocates. Members of the Working Group are selected by ICHOM based on their expertise and meet via teleconference over a period of about 9 months to develop a Standard Set.

The Working Group is led by the Project Team, which performs background research and develops proposals. These proposals are presented to the larger Working Group for discussion, refinement, and ratification.

Finally, representatives from key stakeholders such as implementing provider organizations, patient advocacy groups, payers, or governments form a Reference Group and are invited to participate in the Open Review Survey. Their feedback is then presented to the Working Group at key points in the Standard Set development.

 

relationship

 

How do ICHOM Working Groups determine what to include in a Standard Set?

The schematic below depicts the structured process used to develop ICHOM Standard Sets. The Working Group meets for a series of 8 teleconferences shown in the top row. Each teleconference focuses on one aspect Standard Set development. For example, the first teleconference focuses on the scope of the Standard Set. The following teleconference features a discussion of the outcome domains to include in the Standard Set.

The Project Team creates proposals and presents supporting information to guide each teleconference discussion. Project Team proposals are developed by synthesizing information from the literature (grey), input from patient representatives (blue), and advice from external experts (orange). For example, in preparation for Call 1, the Project Team performs a literature review to determine the outcome domains currently measured for a particular medical condition and facilitates a patient advisory group to understand the outcomes that are most important to patients with that condition. It then presents this information to the Working Group in Call 1 for discussion.

Following each teleconference, the Project Team’s proposal is presented to the Working Group for voting via on-line survey. This may either be a single round of voting in which Working Group Members are asked if they agree or disagree with each item in a proposal (grey arrows) or a 3-round modified Delphi voting process (orange arrows). In the modified Delphi process, Working Group Members are asked to rank potential outcome domains or case-mix variable domains by their importance to include in the Standard Set. For a domain to be included in the Standard Set it must be highly ranked by at least 80% of respondents. To be conclusively excluded from the set, it must be ranked poorly by 80% of respondents. Domains not ranked conclusively after round 1 are represented for voting in round 2 and if not consensus reached for final yes/no voting in round 3. All proposal items must be agreed upon by at least 80% of Working Group Members to be included in the final Standard Set.

schematic

 

Implementation

How are ICHOM Standard Sets measured? What are the requirements to collect ICHOM data?

ICHOM Standard Sets include a combination of administrative, clinical, and patient-reported data. A number of different models are currently being employed to capture these data. Common methods include building data collection into an existing electronic health record (EHR) system, manual chart abstraction, or a combination of the two. To collect patient-reported outcomes (PROs), many providers use simple pen-and-paper solutions, while others have contracted with one of ICHOM’s Certified IT Suppliers to measure PROs electronically.

What do I need before I can start measuring?

Measuring outcomes is a complex and long-term endeavor. Implementation can take a year or more and require additional manpower, technology, and resources. Therefore, it is critical that there is commitment and buy-in at all levels of the organization before beginning. For more information on the resources required to implement and measure ICHOM Standard Sets, please contact us at implement@ichom.org.

Can I implement if I wasn’t in a Working Group?

Yes, all ICHOM Standard Sets are available online at no cost to registered ICHOM.org users. The Reference Guides that accompany each Standard Set include all the necessary information to implement on your own, including a detailed data dictionary for those who may be interested in sharing, comparing, or benchmarking their outcomes with other institutions measuring the same Standard Set(s).

Do I have to measure every outcome recommended in an ICHOM Standard Set?

Every provider should try to measure every outcome for every patient that he or she treats. ICHOM Standard Sets are designed specifically as “minimum sets,” meaning that they include only the most essential outcomes of a given medical condition. Adopters of the Standard Sets may well choose to continue to track various process metrics or even additional outcomes, but we encourage them to collect the Standard Sets – in their entirety – as a starting point. Recognizing the challenge of getting started with outcomes measurement, the adoption of the Standard Set can of course be phased over time, but your ambition should be to measure the entire Set in the target state.

My hospital is very small and resource-constrained, but we want to start measuring. How do we get started?

Although we believe that routine outcomes measurement should be the standard of care, many organizations find that the easiest way to get started by securing a research grant to cover at least a portion of the startup costs. Starting small, with only a few physicians or a sampling of patients, is another method of keeping costs down and ensuring smooth integration of the program into clinical workflow.

Can the Standard Sets be measured in middle-income countries and resource-poor settings? Are there examples of this?

Yes, ICHOM Standard Sets are currently being measured in India and Malaysia. While hospitals and clinics in middle-income countries and resource-poor settings may face additional challenges, the measure of success in health care is remarkably consistent around the world.

How much does implementing an ICHOM Standard Set cost?

There is no set cost for implementing an ICHOM Standard Set. Common factors affecting cost of implementation include IT infrastructure, patient volume, and the method of data collection.

Once I start collecting data, what should I do?

We think of the next steps in three levels. The first is to ensure that the data you generate is reported back to the clinicians in your teams. This alone will lead to valuable insight and help drive improvement within your practice or organization. The second level is to start comparing, when possible, inside your country and region. You will likely find that in some outcome domains you and your teams are performing among the best, while in others there may be room to improve. The third level is to take part in ICHOM’s Global Comparison project, which we are currently developing and which we believe is the best way to accelerate learning – and improvement – around the world.

Does ICHOM want my data?

We do not ask providers to share their data with us. ICHOM is not a registry. We believe, however, that there is tremendous value generated by providers sharing risk-adjusted data with one another. That’s why ICHOM is currently working to set up an infrastructure to advance comparison, not only within organizations, but also across countries. Still, even if you and your colleagues do not choose to share data, we encourage you to use the Standard Sets to measure your own performance.

What if I want to implement at my own pace?

Implementation can move more quickly or more slowly depending on how aggressively you want to invest in beginning to measure your outcomes and what your starting point is. ICHOM believes that most institutions should be able to implement within 6 months to a year, but some implementation efforts may go even faster than that.

What if I have questions related to implementation?

Feel free to reach out to the ICHOM team with questions on implementation (implement@ichom.org). A member of the ICHOM team will be in touch with you to help with your questions if we can.

What implementation support options does ICHOM offer? What do they cost?

ICHOM broadly supports Standard Set implementation by sharing knowledge, success stories, and implementation resources on our blog and in our online library. These materials are available at no cost to all registered ICHOM.org users. For more focused support, ICHOM organizes Implementation Communities (collaboratives) and works individually with institutions to build internal capacity to measure and improve outcomes. For pricing and availability, please contact us at implement@ichom.org.

Do you have any examples of ICHOM Standard Set impact?

Yes. Take a look at the ICHOM blog and online library to see blog posts and case studies highlighting outcome measurement success stories

How many organizations have implemented ICHOM Standard Sets?

Since ICHOM Standard Sets are publicly available and do not require a license, it is difficult to track precisely. However, we know of approximately 650 institutions and 13 registries currently implementing or measuring at least one ICHOM Standard Set across 32 countries.

What is an ICHOM Implementation Community?

An ICHOM Implementation Community is a group of value-oriented institutions that are moving through the implementation process together for a given medical condition. These groups of like-minded institutions, brought together by ICHOM, support each other by discussing best practices, key challenges, and lessons learned. An example is the Cleft Lip and Palate Community, in which ten organisations representing eight countries are collaborating to implement the complete Standard Set.

How do I get in touch with other organizations that are implementing?

ICHOM is currently working on a mechanism for connecting institutions that are measuring Standard Sets at their institutions, as communication and knowledge sharing are key factors to individual and international success. While this mechanism is under development, please contact us at implement@ichom.org if you would like to be connected with other measuring partners.

Do I get recognition for implementation?

ICHOM likes to feature institutions that have implemented (or are implementing) on our blog or in longer case studies and video documentaries. If you are interested in sharing your story, please contact us at implement@ichom.org.

Do you have examples of payers that are using Standard Sets for payment models?

We do know of some organisations now using ICHOM Standard Sets as a basis for outcomes related remuneration. Please contact us at implement@ichom.org, should you wish to learn more.