- About Us
- ICHOM Sets
- Learning Collaboratives
- How we help you
ICHOM was founded in 2012 by Professor Michael Porter of Harvard Business School, Martin Ingvar of the Karolinska Institute and Stefan Larsson of the Boston Consulting Group. ICHOM’s mission is to unlock the potential of value-based health care by defining global Patient-Centered Outcome Measures of outcome measures that really matter to patients for the most relevant medical conditions and by driving adoption and reporting of these measures worldwide.
Outcomes are the results of treatment that patients care about most.
Outcomes are not “outputs”; they are not lab results; they are not technical details. They’re real-world results, like physical functioning or level of pain. How soon after treatment can a patient with low-back pain expect to return to work? How likely is a man to experience incontinence or sexual dysfunction after treatment for prostate cancer? These are questions about outcomes. Unfortunately today, in health care systems around the world, evaluation efforts take into account a number of clinical indicators, structural metrics, and even reputation – but they tend to ignore outcomes.
Value is defined as the outcomes that patients experience relative to the cost of delivering those outcomes. Value-based healthcare, or VBHC, is health care that delivers the best possible outcomes to patients for the lowest possible cost. We believe that choice and competition in healthcare should be based on value. By restructuring care-delivery around outcomes, and promoting superior outcomes with financial incentives, health care systems will improve quality and curb inefficiencies. This will benefit every stakeholder across the health care spectrum. In a value-based world,
In addition to presentations by our cofounders, we have had patient advocates, as well as representatives of leading payer and provider organizations. In 2019, our conference featured Nicola Bedlington (Secretary General, European Patients' Forum), Pieter de Bey (Director, Santeon), Jan Hazelzet (Professor in Healthcare Quality and Outcomes, Erasmus MC), Nathalie Moll (Secretary General, EFPIA) and Jan Kimpen (Chief Medical Officer, Philips). For more information on our next conference, please visit conference.ichom.org
ICHOM is a non-profit organization. Our funding comes from our cofounder organizations, our Sponsoring Partners, our Partners, and our individual donors.
ICHOM does not produce educational content for patients. However, we recommend UptoDate (www.uptodate.com), which offers free, vetted materials. For literature about shared decision-making, while we don’t have a particular recommendation, you may wish to consult the Center for Shared Decision Making at Dartmouth, which offers some useful sample materials, and/or the Informed Medical Decisions Foundation, which has additional content.
The National Quality Forum (NQF) is a public-private partnership in the United States founded in 1999 to improve the quality of American health care. To date, NQF’s primary focus has been on endorsing measures for accountability, meaning those deemed valid for inclusion in pay-for-reporting programs or public transparency programs. For measures to be endorsed by the NQF, a rigorous process demonstrating performance variation as well as adequate risk-adjustment is required, among other things. As many ICHOM measures are patient-reported outcomes, on which performance variation is relatively unstudied and risk-adjustment undeveloped, these measures will require time before those steps can be fulfilled and the measures endorsed. Although we agree with the NQF in the long-term goal of public reporting and payment linked to outcomes, we see great value in first measuring for improvement and are focused there at present.
ICHOM is a US non-profit organization, organized under Internal Revenue Service Code Section 501(c)(3). There are no shares, dividends or similar that could work as vehicles for financial profit being paid out from ICHOM. No individual, organization, company or similar is gaining financial profit from ICHOM.
ICHOM has a UK establishment which is located in London and is registered with Companies House in the UK. This is not a separate entity to the US organization. There are no shares, dividends or similar that could work as vehicles for financial profit being paid out from ICHOM in the UK. No individual, organization, company or similar is gaining financial profit from ICHOM in the UK.
In addition to the above establishment, ICHOM has set up a separate legal entity in the UK. This is called ICHOM Ltd and is a non-profit organization limited by guarantee. This organization is used to receive grants from EU institutions. There are no shares, dividends or similar that could work as vehicles for financial profit being paid out from ICHOM Ltd in the UK. No individual, organization, company or similar is gaining financial profit from ICHOM Ltd in the UK.
ICHOM’s Patient-Centered Outcome Measures are published on its website for all to see and benefit. Whether anyone in the public gains financial profit from that is up to how they use the Patient-Centered Outcome Measures.
The three founders of ICHOM are Michael Porter of Harvard Business School; Martin Ingvar of the Karolinska Institutet; and Boston Consulting Group (BCG). Michael Porter and Martin Ingvar serve on the Board, along with Stefan Larsson representing BCG. The named organizations and associated individuals do not gain any type of financial profit from any branch of ICHOM.
The concept of disease burden is used by the World Health Organization and other public health agencies to measure the overall health impact of a disease or risk factor (see here). It is typically measured in terms of disability-adjusted life years (DALYs), a composite of the years of life lost from a disease and the years of imperfect health living with a disease. Diseases with high mortality rates, such as coronary artery disease, and those with prolonged rates of disability but low mortality rates, such as low back pain, have high impact on DALYs and thus high measured disease burden.
Recently, the Institute for Health Metrics and Evaluation (IHME), based in Seattle, Washington, has taken a lead role in improving and expanding the measurement of global disease burden. As part of this effort, they have released the Global Disease Burden data visualization tool. This tool allows users to easily understand the disease burden of particular conditions. We use this tool and the underlying data, which has been generously given to us by the IHME, to estimate the disease burden of the conditions we have covered. It is important to note that our disease burden estimates are for developed countries, as that is the primary focus of our work.
ICHOM prioritizes the development of new Sets on three primary factors: burden, clinical engagement, and funding. In general, conditions with higher disease burden receive higher priority given the opportunity for greater impact once implemented. Clinical engagement refers to the availability of respected clinical leaders who are eager for such a Set, are willing to help develop it, and will champion its adoption globally. Finally, funding to cover the development cost of a Set is necessary to proceed.
We require $500,000 USD to develop an ICHOM Set. This amount covers the staff to run the project (a full-time ICHOM Project Leader, Standardization Associate, and Research Fellow), management oversight by ICHOM leadership, and marketing and travel expenses. Funding can be sourced by a single institution or by multiple institutions combining together.
Notably, the cost to use our work is free. To encourage widespread awareness and adoption of our Sets, we make publicly available all Reference Guides, data dictionaries, gap analysis tools and publications.
To avoid potential conflicts of interest in recommending specific outcome indicators that bias towards specific products, we do not include life science representatives in our Working Groups or as direct funders. However, we do recognize that life science firms have a strong role to play in enabling value-based healthcare. For this reason, we hold an open review period in which all stakeholders, including medical technology and life sciences representatives, can comment on our Sets while they are still in development.
After a Set has been developed, a sub-group of the Working Group continue their involvement with ICHOM and the Set by joining the Steering Committee, or ‘SteerCo’. Their role involves:
The unit of analysis is the major entity being analyzed – the ‘what’ or the ‘who’. For ICHOM’s Sets there is more than one unit of analysis, including the individual patient, groups or large populations of patients (as defined within the scope of each Set), provider organizations, provider networks, healthcare plans and individual healthcare professionals.
We involve patients in the development of our Sets using three methods:
We also engage patient charities and advocacy organizations in our wider mission – of supporting a patient-centered approach to healthcare through patient-reported outcomes measurement.
Each Set of Patient-Centered Outcome Measures is made up of the following components:
At the conclusion of the Working Group process a manuscript explaining the process to arrive at the Set of Patient-Centered Outcome Measures and motivation for selected measures is submitted to peer-reviewed academic journals for publication. The manuscripts are available to view on each Set of Patient-Centered Outcome Measures page. They also appear in our Resource Library.
The ICHOM Working Groups are composed of an ICHOM Project Team and approximately 15-20 Working Group members. The ICHOM Project Team consists of the Working Group Lead or Chair; a clinician with an international reputation and experience in the field, a Project Leader, Research Fellow, and a Standardization Associate. The Working Group members are composed of international volunteer representatives with a keen interest in outcomes measurement. They are clinical leaders and patient/carer advocates, representing many geographies and all specialties involved in caring for a particular condition. All Working Group Members participate on a voluntary basis. In total, members have come from 39 countries in 6 continents.
PROMs are thoroughly researched and then selected based on the following criteria:
With respect to criterium #2, Psychometric Quality, we follow the ISOQOL minimum standards for PROM measures (Reeve et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res (2013) 22:1889–1905).
No. We research the PROMs that are available in the field, per condition. The recommended PROM is selected based on the criteria listed in the previous question. When there are few or no PROMs available, we may recommend separate, patient-reported, single-item questions that have been used in registries; PROMs that have been used in a research setting; or, defer to clinical/administrative information as a substitute. For example, in the Lung Cancer Set of Patient-Centered Outcome Measure, clinical/administrative data are used for “location of death” and “number of days in hospital in last 30 days of life” to collect Quality of Death.
After a Set of Patient-Centered Outcome Measures is finalized, it is made available through the ICHOM website in the form of a Flyer and a Reference Guide. The Flyer contains a high-level overview of the recommended outcome domains, as well as information about the sponsors and the Working Group members. The Reference Guide is a detailed document describing all domains and measures, as well as the case mix factors. A large part of the Reference Guide is the Data Dictionary, designed to help interested providers to measure the ICHOM Patient-Centered Outcome Measures as consistently as possible according to the Working Group recommendation. Additionally, the Set is published in academic journals, to communicate the work to the medical community. Each Set is reviewed and updated on a regular basis. The next step after finalizing the Set is implementation and benchmarking.
Yes, the work is presented at conferences globally. Please check our social media pages and newsletter to see where we present our work.
ICHOM Sets are developed by international Working Groups of leading clinicians, outcomes researchers, registries leaders, and patient advocates. Members of the Working Group are selected by ICHOM based on their expertise and meet via teleconference over a period of about 9 months to develop a Set.
The Working Group is led by the Project Team, which performs background research and develops proposals. These proposals are presented to the larger Working Group for discussion, refinement, and ratification.
Finally, representatives from key stakeholders such as implementing provider organizations, patient advocacy groups, payers, or governments form a Reference Group and are invited to participate in the Open Review Survey. Their feedback is then presented to the Working Group at key points in the Set development.
The schematic below depicts the structured process used to develop ICHOM Sets. The Working Group meets for a series of 8 teleconferences shown in the top row. Each teleconference focuses on one aspect of the Set development. For example, the first teleconference focuses on the scope of the Set. The following teleconference features a discussion of the outcome domains to include in the Sets.
The Project Team creates proposals and presents supporting information to guide each teleconference discussion. Project Team proposals are developed by synthesizing information from the literature (grey), input from patient representatives (blue), and advice from external experts (orange). For example, in preparation for Call 1, the Project Team performs a literature review to determine the outcome domains currently measured for a particular medical condition and facilitates a patient advisory group to understand the outcomes that are most important to patients with that condition. It then presents this information to the Working Group in Call 1 for discussion.
Following each teleconference, the Project Team’s proposal is presented to the Working Group for voting via on-line survey. This may either be a single round of voting in which Working Group Members are asked if they agree or disagree with each item in a proposal (grey arrows) or a 3-round modified Delphi voting process (orange arrows). In the modified Delphi process, Working Group Members are asked to rank potential outcome domains or case-mix variable domains by their importance to include in the Set.. For a domain to be included in the Set of Patient-Centered Outcome Measures it must be highly ranked by at least 80% of respondents. To be conclusively excluded from the set, it must be ranked poorly by 80% of respondents. Domains not ranked conclusively after round 1 are represented for voting in round 2 and if not consensus reached for final yes/no voting in round 3. All proposal items must be agreed upon by at least 80% of Working Group Members to be included in the final Set.
ICHOM Sets include a combination of administrative, clinical, and patient-reported data. A number of different models are currently being employed to capture these data. Common methods include building data collection into an existing electronic health record (EHR) system, manual chart abstraction, or a combination of the two. To collect patient-reported outcomes (PROs), many providers use simple pen-and-paper solutions, while others have contracted with one of ICHOM’s Certified IT Suppliers to measure PROs electronically.
Measuring outcomes is a complex and long-term endeavor. Implementation can take a year or more and require additional manpower, technology, and resources. Therefore, it is critical that there is commitment and buy-in at all levels of the organization before beginning. For more information on the resources required to implement and measure ICHOM Sets, please contact us at email@example.com
Yes, all ICHOM Sets are available online at no cost. The Reference Guides that accompany each Set include all the necessary information to implement on your own, including a detailed data dictionary for those who may be interested in sharing, comparing, or benchmarking their outcomes with other institutions measuring the same Set(s).
Every provider should try to measure every outcome for every patient that he or she treats. ICHOM Sets are designed specifically as “minimum sets,” meaning that they include only the most essential outcomes of a given medical condition. Adopters of the Sets may well choose to continue to track various process metrics or even additional outcomes, but we encourage them to collect the Sets – in their entirety – as a starting point. Recognizing the challenge of getting started with outcomes measurement, the adoption of the Set can of course be phased over time, but your ambition should be to measure the entire Set in the target state.
Although we believe that routine outcomes measurement should be the standard of care, many organizations find that the easiest way to get started by securing a research grant to cover at least a portion of the start-up costs. Starting small, with only a few physicians or a sampling of patients, is another method of keeping costs down and ensuring smooth integration of the program into clinical workflow.
Yes, ICHOM Sets are currently being measured in India and Malaysia. While hospitals and clinics in middle-income countries and resource-poor settings may face additional challenges, the measure of success in health care is remarkably consistent around the world.
There is no set cost for implementing an ICHOM Set. Common factors affecting cost of implementation include IT infrastructure, patient volume, and the method of data collection.
We think of the next steps in three levels. The first is to ensure that the data you generate is reported back to the clinicians in your teams. This alone will lead to valuable insight and help drive improvement within your practice or organization. The second level is to start comparing, when possible, inside your country and region. You will likely find that in some outcome domains you and your teams are performing among the best, while in others there may be room to improve. The third level is to take part in ICHOM’s Global Comparison project, which we are currently developing and which we believe is the best way to accelerate learning – and improvement – around the world.
We do not ask providers to share their data with us. ICHOM is not a registry. We believe, however, that there is tremendous value generated by providers sharing risk-adjusted data with one another. To share your findings or to speak with other organisations who are implementing Sets, please visit ICHOM Connect at connect.ichom.org
Implementation can move more quickly or more slowly depending on how aggressively you want to invest in beginning to measure your outcomes and what your starting point is. ICHOM believes that most institutions should be able to implement within 6 months to a year, but some implementation efforts may go even faster than that.
If you have questions about implementation and wish to speak to other organisations, please visit ICHOM Connect at connect.ichom.org
ICHOM broadly supports Set implementation by sharing knowledge, success stories, and implementation resources in our resource library. These materials are available at no cost to you. For more focused support, ICHOM has created an online portal, ICHOM Connect - please visit connect.ICHOM.org for more details.
Yes. Take a look at the ICHOM resource library to see case studies highlighting outcome measurement success stories. Our annual conference will also reference ICHOM Sets impact. To attend, please visit conference.ICHOM.org
Since ICHOM Patient-Centered Outcome Measures are publicly available and do not require a license, it is difficult to track precisely. However, we know of approximately 650 institutions and 13 registries currently implementing or measuring at least one ICHOM Set of Patient-Centered Outcome Measures across 32 countries.
ICHOM Connect is an online community of value-oriented institutions that are moving through the implementation process for a given medical condition. These groups of like-minded institutions, brought together by ICHOM, support each other by discussing best practices, key challenges, and lessons learned.
Please join ICHOM Connect to discuss your implementation journey with other organizations. Visit connect.ichom.org for more information.
ICHOM likes to feature institutions that have implemented (or are implementing) in case studies or at our annual conference. If you are interested in sharing your story, please contact firstname.lastname@example.org
We do know of some organisations now using ICHOM Set of Patient-Centered Outcome Measures as a basis for outcomes related remuneration. Please contact email@example.com for more information.
According to Michael Porter, value should always be defined around the customer, and in a functioning healthcare system, the creation of value for patients should determine the rewards for all other actors in the system. Since value depends on results, not inputs, value in healthcare is measured by the outcomes achieved, rather than the volume of services delivered. Shifting focus from volume to value is therefore central.
Since value is defined as outcomes relative to cost, it encompasses efficiency.
In this context, the universal development and reporting of outcomes at the medical condition level is the single highest priority to improve the performance of the healthcare system.
Outcome measurement empowers and benefit patients, clinicians and payors.
a) Data transparency will enable patients to not only follow their outcomes and see how they are responding, but also to choose the most suitable provider for their care according to what matters most to them.
b) Clinicians will also benefit as measuring outcomes will enable them to compare practices and learn how they can improve.
c) And finally, it will allow payors to measure return on investment and DIRECT PATIENTS to high value providers.
The criteria for selecting a Standard Set to implement can be categorised into three domains:
First, and most importantly, there is the cultural factor: motivated and committed staff are key to success. As they will be responsible for executing the implementation, they should have an understanding and belief that it will be beneficial for staff and patients.
The second factor is to align the implementation efforts with current measurement activities. In other words, try to choose a condition in which a large percentage of the Standard Set is already being collected, as it will make the implementation process more straightforward.
In addition, consider the Patient Reported Outcomes measurement tools included, how many there are, if a license fee is required, and if official translations in your language are available.
The third factor is the long-term impact potential. In this case, you need to consider:
a) The availability of other sites already implementing the Standard Set, which will allow early global comparison.
b) The annual patient volume: Larger volumes enable strong proof-of-concept to demonstrate feasibility. On the other hand, smaller volumes may help accelerate implementation.
c) Regarding measurement cycle timelines, shorter time lines allow measuring outcomes faster and implementing improvements will result in seeing benefits quicker.
Regarding Standard Set and PROMs translation, please refer to each condition reference guide to check if there’s a translation available in your local language. If not, we suggest two alternatives to deal with translations:
First, use an established translation provider to perform the translation. ICHOM recommends using the ISPOR Principles of Good Practice in the Cross-Cultural Adaptation Process for Patient-Reported Outcomes Measures.
As a temporary solution, the Implementing Team performs the translation into local language. This will not have been thoroughly tested and validated but will allow the organisation to begin measurement.
A key step in preparing for implementation is to determine if a licence for using a PROM is required. This information is also available in the PROMs section in each condition reference guide.
Another issue to consider from the beginning is to research local and national ethical and legal requirements for project implementation. Ethical approval may be required for data handling and management, particularly if the intention is to benchmark. It can be applied for early in the project (during planning and diagnostics), regardless of whether there is a desire to publish (but if to publish is an objective, it is a must).
First, there is the Preparation phase. This includes:
a) Defining the scope of the project, which is part of the project planning to determine and document a list of objectives, deliverables, tasks, deadlines and costs. In other words, it is what needs to be achieved to deliver the project.
It’s fundamental to also appoint a capable and motivated project manager who will be responsible for guiding and executing the implementation.
Then, establish the project team, responsible for executing the implementation, with the support of the project manager.
Next, it’s important to plan a meeting with key stakeholders to inform them about the project and engage them with the vision.
Finally, develop and implementation plan to guide you throughout the project. This is the phase within the project life cycle that integrates the project into the organization, including: software, data, training, documentation and required organisational changes.
b) In the diagnostic phase, one of the main challenges is with the IT and informatic structure, to compare the data that is being collected against what is not is crucial. Another key element is to decide the paths and how data will be collected and act accordingly.
c) The roll out is the actual implementation, when pilot data collection begins. This allows us to assess challenges, review the data collection process and act accordingly to improve the data collection pathway.
d) Once the full data set has been implemented, data is audited to ensure it is complete and valid.
e) Finally, and one of the MOST IMPORTANT ISSUES, is to decide how data is going to be used and start reporting to clinicians and patients.
These phases need to be supported with continuous change management as the final objective should not be collecting data, but how successfully you use that information to drive change and improve practice.
Having a committed team is essential to drive a project to succeed. The core team should compromise a clinical lead and a project manager who can drive the day-to-day tasks:
The project manager will require 2 days a week to work on implementation.
We recommend this person has the following skills: experience in project management, especially in change management in a clinical environment. He or she should also be familiar with quality improvement methodologies and local internal infrastructure. The project manager would be responsible for guiding, coordinating and running the site through the implementation process on a day to day basis internally, and ensuring key deliverables in the project plan occur on time and to schedule. He or she will also assist with key implementation tasks such as: designing the project plan, preparing for the launch event, organising key internal meetings with IT, informatics, clinicians etc., escalating concerns or risks to the steering committee for action.
The other key member is the clinical lead, who typically requires half a day per week to work on the project. This person is ideally clinically trained in the condition area, be respected by peers so they can lead by example, have experience participating in quality improvement initiatives and be open to driving change within the hospital and care system.
Also, we suggest a Steering Committee is assembled, comprising the site leadership such as hospital clinical executives, lead of IT, head of clinical department, etc. This committee would provide oversight and accountability for the project, as well as reviewing progress, making decisions quickly and allocating financial and human resources when needed.
Finally, there is the hospital team, responsible for executing the implementation with the support of the manager.
For these we suggest:
a) Preparing educational packages and brochures for patients
b) Developing educational videos to show in waiting areas
c) Including information on website and social media
d) Organising a project launch meeting for clinical staff/managers/other stakeholders
e) Transferring knowledge to staff to educate patients about PROMs benefits and how patients are going to be contacted for data collection
f) Keeping patients and staff informed about outcome measurement and changes that will happen in their care in relationship with results.
First, you need to assess your existing IT infrastructure and remember, the strategy you use to collect the data is your decision according to what suits you best.
The things you need to consider are:
a) What solution is best suited for integrated data collection across your site?
b) Does your selected solution enable measurement of the full set including PROMS?
c) How often will you be collecting data and what volume of patients you will have? If you’re using paper questionnaires with manual entry of data to an excel warehouse, it can be implemented immediately with no upfront cost; however, it’s not a scalable solution and you might have errors when transcribing to excel database.
d) Do you have sufficient resources to develop an IT platform for collecting data?
A second item to consider is how you are going to use the collected data, to whom, and how often it will be presented. Deciding this in advance will help you decide which will be the best IT solution to implement in the long term.
For you to have a better understanding of your IT requirements, you need to know what your existing administrative, clinical and patient-reported data collection methods are. Also, what are the IT capabilities. This includes how easy it is to modify what is being collected, the ability to provide real time analysis, how easy it is to use the IT system alongside traditional paper notes, and its ability to create dashboards to collect and view data.
Many sites already collect most of the required data, only in a different coding. ICHOM’s GAP analysis will help you understand what data your hospital is currently collecting against our reference guide.
A final key step in project implementation is auditing data collection to assess its quality and utility for its purpose.
The first step is to define the data items that you’re going to measure and assess from your pilot sample.
Then, when measuring, you need to consider:
a) Data completeness including proportion of data items entered, completed at the right time point and complying with the ICHOM standard set.
b) Data validity, including that data has a value that satisfies the response option in the reference guide and collected at the right time point.
c) And third, that ideally data should be 100% accurate during transcription from paper to electronic medical records.
Finally, determine whether data quality is good or bad.
First, leadership is critical to success. Enthusiastic and dedicated leaders are critical to help generate support. It’s vital for leaders to make clinicians feel supported and to set guardrails for how data will be used.
Second, outcomes measurement is a team effort. It’s necessary to ensure everyone understands its goals and is engaged in the process. You’ll need all the stakeholders on board to make it work.
Third, there’s no “One size fits all” solution. The implementation journey will be different for every organisation depending on its characteristics and resources available.
Fourth, measurement doesn’t have to be burdensome: there are opportunities to make this easy for clinicians: evaluate the possibility to seamlessly integrate measurement to workflow, specially using health information technology.
And fifth, data needs to be accessible and actionable, meaning that it should be reported in a way that enables improvement in daily clinical practices to improve patient outcomes.