Ways of developing quality indicators in cancer care — opportunities, challenges and limitations for the Polish healthcare system

Abstract

The aim of this article is to present possible ways of developing quality indicators in oncological care in the Polish healthcare system based on practice from other countries. 

The development of indicators in the healthcare systems presented in this paper is a process following several stages. Initially, at the planning stage, the clinical area to be assessed is selected and teams responsible for developing the indicators are organized. This is followed by the development stage in which the measurement team prioritizes and selects clinical indicators on the basis of documentation and knowledge gained from the scientific literature. After selecting the target clinical indicators, the specifications for each measure are operationalized, along with inclusion and exclusion criteria for the target population, and data sources are identified. In each of the foreign healthcare systems analyzed, determination of the final indicators was preceded by extensive clinical and social consultations. 

The use of clinical indicators to assess quality is an important approach in the process of assessing the quality of cancer care. Thanks to the introduction of quality indicators, participants in the healthcare system (regulators, clinicians, patients, managers of medical institutions) can obtain reliable information that is necessary for defining priorities, modifying methods of determining benefits, benchmarking, making informed choices, and improving the quality of oncological care.

Introduction

The issue of quality in the Polish healthcare system has been increasingly gaining importance in recent years, both for patient communities and public healthcare providers, as well as those involved in making healthcare decisions. Given the ever-increasing cost of cancer care and the limited availability of many costly treatment options for cancer patients, cancer care requires a more rigorous approach to quality assessment [1]. It is worth noting that even though legislation on different kinds of healthcare quality indicators (QI) exists in Poland, the concept of quality has not been substantively defined in any legislation. Assessment of quality in healthcare with the aid of quality indicators should be ensured by the law on quality in healthcare and patient safety, which is currently being drafted and is due to be published at the beginning of 2022. Of some concern is the fact that in the draft submitted for public consultation, the definition of the term “quality” has, once more, been overlooked, while quality indicators themselves are referred to in only one clause of the Act (Article 3). Quality in healthcare is defined and measured by indicators relating to three main areas: 1) clinical; 2) consumer; and 3) healthcare management considerations [2]. Thus, so long as the quality of healthcare is not described within these specific domains, it remains an undefined concept.

The current article describes proposals for developing indicators for the quality of oncological care in Poland (from prophylaxis, through diagnostics, and therapy to broadly understood post-treatment care) on the basis of practice from other countries and considering challenges and limitations related to the process of assessing the quality of oncological care.

The process of developing quality indicators

The concept of quality in healthcare can be understood from a wide perspective, with the particular focus depending on the point of view of the surveyor. It is perceived from a specific perspective by patients whose attention is focused on relationships with staff, examinations, test results, and treatment outcomes, as well as the atmosphere and the environment in which they find themselves. On the other hand, for healthcare workers, quality is mainly related to the reliable provision of services, in accordance with accepted standards, and the availability of state-of-the-art diagnostic and treatment instruments [3]. The contemporary approach to the issue of quality in healthcare is based on the model proposed by Avedis Donabedian in the 1960s. A successful outcome in terms of patient recovery, restoration of function or survival is equated with high quality care. Presenting measurable clinical outcomes for a treatment procedure makes it possible to compare therapeutic effects among different groups of patients and the results achieved in different centers. It is also useful to present and compare the social effects (benefits) of improving the quality of life and levels of patient satisfaction. Donabedian distinguished three dimensions of the quality of medical services, which together constitute the quality of a given service. It is not possible to consider a service as being of high quality if errors or shortcomings are identified in any of the three categories.

Quality of the structure — includes the structure of the organization, the number of medical personnel, their qualifications, equipment, medical apparatus, buildings.

Quality of the process — refers to the range of actions performed or not undertaken during the process of diagnosis, treatment, nursing, and rehabilitation of the patient, including waiting time for the procedure and results of examinations. Donabedian emphasizes the fact that the best results are achieved when treatment follows a systematic course, according to tried and tested principles.

Quality of the results — includes the degree of improvement in the patients’ health and their satisfaction with the healthcare services provided. This comprises indicators such as death rates, morbidity, adverse events, etc. [4].

Donabedian also pointed out the need to assess the course of the care process and the consequent potential of the information gained for improving care management. Analysis of diagnostic and therapeutic processes and comparisons between centers of interest contributes to a better assessment of the quality of healthcare and further improvement of results. Elements that complement contemporary understanding and practice of quality assurance in healthcare include the constantly growing social awareness of responsibility for one’s own health (including the influence of lifestyle on health), alternative options for diagnostic and therapeutic procedures, the need to publish information on the needs and rights of patients, costs and effectiveness of procedures, and treatment outcomes. Quality assessment is performed using various types of indicators and criteria relating to specific standards [5, 6].

Material and methods

The article presents a cross-sectional analysis of approaches and experiences in developing quality indicators for the evaluation of oncological care in selected countries. For this purpose, both scientific reports and documents regulating the area related to quality in healthcare were searched.

This publication will present the implementation of a quality assessment system in oncology in six selected countries: Australia, Germany, Scotland, the USA, Canada (Ontario), and Japan, which seem to have the greatest expertise in assessing the quality of cancer care.

Results

Australia

The Australian Council on Healthcare Standards (ACHS) sponsored an expert-led consensus-based four-step process, based on the modified Delphi method, to define a set of clinical indicators to assess the quality of oncology services provided in Australia. This process was carried out in response to requests from service providers applying for accreditation. The multidisciplinary process steering committee was composed of clinical experts and patient representatives. Five additional participants were the stakeholder group that debated the final set of indicators [7].

The process of developing the indicators was carried out in 4 stages:

Stage I — Establishment of the Steering Committee

A steering committee of 16 key experts in cancer treatment, policy, nursing, outpatient care, radio-­ -oncology, and patients representing a wide variety of experiences and perspectives was established. During a one-day meeting, the committee identified directions, terms, and potential areas for the introduction of quality indicators.

Stage II — Literature review and search for indicators

A literature review and systematic search of the websites of international oncology societies was carried out to identify currently used indicators. The steering committee then adopted a systematic work plan for analyzing the list of identified potential indicators, evaluating each indicator and prioritizing it in an online survey, which took about 3 hours to complete. Individual indicators were ranked from 0 to 5 (lowest to highest priority) for each of the two criteria: ease of access and data collection, as well as clinical relevance (including potential feasibility for improving quality and evaluating best practice performance). On this basis, a priority list of potential clinical indicators was developed.

Stage III — Discussion of results

In this step, each of the highest rated indicators obtained in stage 2 was discussed in detail before being accepted, rejected, or modified. The nomenclature and measurement method were optimized for each selected indicator. The results were then discussed during a meeting with a reference group of 20 stakeholders, including health policy experts, key leaders from various clinical specialties, major providers of cancer services, nurses’ representatives, representatives of municipal and rural services, pharmacists, statisticians, indicator specialists and representatives of the public.

Stage IV — Development of coding rules for new indicators

At this stage, the steering committee, in cooperation with the ACHS, oversaw the development of an oncology care guide to facilitate the clinical coding of new indicators. The guide was approved by the Clinical Oncology Society of Australia (COSA), then ratified by the ACHS Board of Directors and published on the ACHS website [8].

During the process of development, the main reason for rejecting certain indicators was the concern that collecting data would be too burdensome for the participants of the system. This was because some information was not recorded at all or would probably have to be collected from a range of different sources. Despite widespread recognition that increasing access to digital medical data should facilitate its use in informing quality of care considerations, many Australian centers still have relatively simple electronic health record systems [7, 9].

Consequently, preference was given to indicators derived from data that are commonly and routinely collected in healthcare facility systems (waiting list information, electronic medical records, financial systems, etc.). On the other hand, it is recognized that the information necessary to calculate clinical indicators may prompt organizations to consider adding or redesigning their data collection to achieve full compliance.

In summary, this was the first attempt to create a comprehensive set of high-quality clinical indicators to measure the quality of care at each stage of cancer progression in patients in Australia. An expert group and a consensus-based methodology with broad stakeholder representation should ensure that this approach is easy to use and productive in obtaining quality baseline data for monitoring, evaluating, and comparing oncology care provided in different centers. Clinical indicators should be evaluated on a regular basis with the possibility of adding new ones or modifying existing ones, both in response to the experiences of reporting organizations and to adapt to the changing needs of cancer care services. Examples of Australian quality indicators in cancer care are shown in Table 1.

Table 1. Examples of Australian Quality Indicators in Cancer Care

Area	Name of indicator	Specifications
Screening	Breast screening rates (one of six indicators)	Numerator: Number of females in the target age range who received a mammogram through the national screening program over a 24-month period Denominator: Average number of female residents in the target age range during the 2-year reporting period
Diagnosis	Cancer incidence	Cancer incidence indicates the number of new cancers diagnosed during a specified period (usually one year). The major source of cancer incidence data is the Australian Cancer Database (ACD) which contains records of all primary, malignant cancers (except basal cell and squamous cell carcinomas of the skin) diagnosed in Australia since 1982 All Australian states and territories have legislation that makes cancer a notifiable disease. Various designated bodies, i.e., institutions such as hospitals, pathology laboratories, and registries of births, deaths, and marriages, are required to report cancer cases and deaths to their jurisdictional cancer registries Each registry supplies incidence data annually to the Australian Institute of Health and Welfare (AIHW) under an agreement between the registries and the AIHW. These data are compiled into the ACD, the only repository of national cancer incidence data
	Distribution of cancer stage	The distribution of cancer stage at diagnosis for the top five incidence cancers (breast [female], colorectal, lung, melanoma, and prostate cancer) Numerator: Incident cancer cases for a selected Registry-Derived Stage (RD-Stage) at diagnosis value (stage 1, stage 2, stage 3, stage 4, or unknown) for a selected cancer type. Denominator: All eligible RD-Stage records that could be matched to an incident cancer case in the ACD for the relevant cancer type. The denominator includes cases with an „Unknown” stage at diagnosis for which the registry did not have sufficient information to define the stage
	Capture of stage data	The unadjusted crude proportion of cancer cases for which stage data are available for cases with a principal diagnosis. The same top 5 incidence cancers Numerator: Incident cancer cases for a selected RD-Stage at diagnosis value (staged or unknown) for a selected cancer type Denominator: All eligible RD-Stage records that could be matched to an incident cancer case in the ACD for the relevant cancer type. The denominator includes cases with an „Unknown” stage at diagnosis for which the registry did not have sufficient information to define the stage
Treatment	Radiotherapy treatment activity	This measure shows the number of radiotherapy services processed by Medicare Australia over time and by population group Unit of analysis: The Medicare Benefits Schedule (MBS) summarized data show the number of radiotherapy “services” for which a reimbursement is claimed under the Medicare Benefits Schedule and processed by the Department of Human Services. These types of data will allow for linkage of cancer incidence and radiotherapy services provided in later phases of Stage, Treatment, and Recurrence (STaR) analyses and reporting Note: An additional source of data regarding radiotherapy treatment in Australia is the National Radiotherapy Waiting Times Database (NRWTD). Data are provided to the AIHW from jurisdictional health authorities and private radiotherapy providers. The unit of measure for the NRWTD data is the number of radiotherapy “courses” which is defined as a “series of one or more external beam radiotherapy treatments”
	Systemic anti-cancer therapy treatment activity	This measure shows the number of people receiving at least one of the cancer-related systemic therapies and a small number of supportive treatments in Australia in any one year over the period from 2012 to 2016 Unit of analysis: The number of people who were dispensed systemic anti-cancer related therapeutic items in a given individual year and for whom a reimbursement claim was processed under the Australian Government’s Pharmaceutical Benefits Scheme (PBS) To provide the context of the use of these therapies within the Australian population, the number of people who accessed these therapies was adjusted for the increase in population over the period. Data are expressed as the number of people who accessed these therapies per 100 000 population for the relevant year (Estimated Resident Population data are sourced from the Australian Bureau of Statistics)
Treatment	Surgical treatment activity — top 5 incident cancers	This measure focuses initially on the five highest incidence cancers in Australia: prostate, breast, colorectal, melanoma, and lung. Initial examination of procedure codes by principal diagnosis indicated a degree of overlap for treatment procedures recorded for colon and rectal cancers. To avoid potential confusion in reporting the data, these cancers have been analyzed as a group (i.e. colorectal cancers). It is anticipated that for later data analyses, where a confirmed incidence for these two cancers is available, separate data will be presented for colon and rectal cancers Unit of analysis: The number of hospital separations where the principal diagnosis for a relevant cancer was recorded and where there was at least one cancer-related procedure. Note that the unit of analysis is for hospital separations, not individual patients. An individual who had multiple separations in a given year will have a record for each of these separations. Therefore, an individual patient may be counted more than once in these data
	Multidisciplinary care	The proportion of new cancer cases discussed at an MDT meeting Numerator: The number of new cases discussed at an MDT meeting in the reporting period Denominator: The number of new cases referred to the cancer service in the reporting period

Authors’ summary based on National Cancer Control Indicators (canceraustralia.gov.au) [10]

Germany

The process of developing quality indicators forms part of the German Guideline Program in Oncology (GGPO) jointly launched by the German Cancer Society (DKG), German Cancer Aid (DKH), and the Association of Scientific Medical Societies (AWMF) in 2008. This program is designed to support the development, implementation and evaluation of evidence-based clinical practice guidelines. On the basis of these guidelines, indicators can be developed to evaluate the structure, process and outcome of cancer care, which provide a measure of the quality of care and adherence to recommendations. Indicators are tools for internal quality management in medical facilities and benchmarking with other institutions. Quality indicators are generally developed for areas where the authors of the guidelines and other healthcare professionals have identified potential for quality improvement [11].

The development of quality indicators was carried out in 5 stages:

Stage I — The composition of a representative working group on quality indicators (QIs) was determined by the Guideline Development Group (GDG)

The development of the QIs was discussed during the inaugural meeting of GDG. The working group had a maximum of 14 members, and the requirement was that it should be interdisciplinary, composed of people involved in the relevant areas addressed by the guidelines. The bodies routinely involved in this process include the committees of certified centers, cancer clinical registries, and other institutions in the area of quality management. The process is supervised and supported, with regard to methodology, by a representative of the GGPO office and a representative of the AWMF.

Stage II — Preparation of quality indicators

The goal of the first meeting of the Quality Indicators Working Group (QIWG) was to prepare a list of fundamental indicators based on clinical guidelines. It was emphasized that only strong recommendations should be translated into potential indicators and that these should be as specific as possible. Only recommendations with an “A” rating, those considered to have the highest priority for implementation, were taken into consideration, as it was assumed that actions based on these recommendations should be of clear benefit to most patients. Consequently, only strong recommendations were considered suited to the development of indicators, irrespective of whether they were evidence-based or consensual. To prepare a preliminary list of indicators, available databases containing scientific reports, websites of governmental institutions and scientific societies were searched.

Stage III — The first meeting of the working group on quality indicators

The first meeting of the QIWG aimed to complete a preliminary selection from the potential indicators identified in Stage II. The QIWG consensus was based on the following exclusion criteria:

A1: the recommendations cannot be operationalized (this generally means they are not measurable)

A2: the inability to improve the quality of healthcare provision

A3: problems with the interpretation of the indicator description by the QIWG and/or expenditure associated with the preparation of documentation too high in relation to the benefits

A4: other (e.g. duplication of indicators from two different guidelines).

The exclusion criteria are derived from the four basic requirements for quality indicators, defined in the German QUALIFY assessment instrument [12]:

The significance of the quality captured by the quality indicator (category “accuracy”);

The clarity of the definition of the indicator and its application (category “scientific soundness”);

Comprehensibility and interpretability (category “practicality”);

Costs incurred in collecting data (category “practicality”);

The indicators are accepted on the basis of a vote in accordance with the rules of consensus recommendation (AWMF Regelwerk), which means that a minimum of 75% of votes is required. [13]

Stage IV — Written evaluation of potential quality indicators

The selected set of potential indicators was evaluated using a standard scoring sheet that included the criteria listed above. Two additional criteria (“risk adjustment” and “implementation barriers”) were also commented on by the evaluators.

Stage V — The second meeting of the working group on quality indicators

The subject of the second QIWG meeting was to discuss the written assessments of the indicators and to select the final set of indicators. For final approval, the consent of a minimum of 75% of votes was required.

The entire process of developing quality indicators took 6 to 12 weeks and was fully documented. The final set of indicators was submitted to the coordinators of oncology registries and certification committees for their implementation [14–17].

Figure 1 presents a summary of the process of developing quality indicators. Examples of German quality indicators in breast cancer care are shown in Table 2.

Figure 1. The development process for Quality Indicators [11]; GDG — Guideline Development Group; GGPO — German Guideline Program in Oncology; Ql — quality indicators; WG — working group

Table 2. Examples of German Quality Indicators in breast cancer care

Area	Name of indicator	Specifications
Treatment	Post-operative case review	Numerator: Primary cases of denominator presented at the tumor conference Denominator: Surgical primary cases Rate: Target value ≥ 95%
	Pre-therapeutic case discussion	Numerator: Primary cases of denominator presented at the pre-therapeutic conference Denominator: Primary cases Rate: Target value: ≥ 40%
	Discussions of cases involving local recurrence/metastases	Nominator: Patients of the dominator presented at the tumor board Denominator: Patients with first local recurrence and/or first remote metastasis (without primary M1 pat.) Rate Mandatory statement of reasons* <70%
	Radiotherapy after BCT in the case of invasive mammary carcinoma (GL QI 8)	Numerator: Primary cases of the denominator in which radiotherapy was recommended Denominator: Primary cases with invasive mammary carcinoma and BCS (without primary M1 pat.) Rate: Target value ≥ 90%
	Radiotherapy after BCT in the case of DCIS	Numerator: Primary cases of the denominator in which radiotherapy was recommended Denominator: Primary cases with DCIS and BCT Rate Mandatory statement of reasons* < 80%
	Chemotherapy in the case of rec. pos. and nodal pos. result	Numerator: Primary cases of the denominator in which chemotherapy was recommended Denominator: Primary cases with invasive mammary carcinoma with rec. pos. and a nodal positive result (without primary M1 pat.) Rate: Target value ≥ 60%
	Endocrine therapy in the case of steroid rec. positive result	Numerator: Primary cases of the denominator in which endocrine therapy was recommended Denominator: Primary cases with invasive mammary carcinoma in the case of steroid rec. a positive result (without primary M1 pat.) Rate: Target value ≥ 80%
	Trastuzumab therapy over 1 year in the case of HER-2 pos. result	Numerator: Primary cases of the denominator for which trastuzumab therapy over 1 year was recommended Denominator: Primary cases with invasive mammary carcinoma with HER-2 positive result (without primary M1 pat.) Rate: Target value ≥ 95%
	Endocrine therapy for metastasis	Numerator: Patients of the denominator who were started on endocrine-based therapy in the metastasized stage as first-line therapy Denominator: Patients with steroid rec. pos. and HER2-negative inv. mammary carcinoma with 1st Remote metastasis (incl. primary M1 pat.) Rate: Target value ≥ 95%
	Psycho-oncological care (Consultation > 25 min)	Numerator: Patients who received psycho-oncological care in an inpatient or outpatient setting (duration of consultation > 25 Min.) Denominator: Primary case patients + patients with 1st local recurrence and/or remote metastasis (without primary M1 pat as they are already included in primary cases) Rate: Mandatory statement of reasons* 95%
	Counselling social services	Numerator: Patients who received counselling by social services in an inpatient or outpatient setting Denominator: Primary case patients + patients with 1st local recurrence and/or with 1st remote metastasis (without primary M1 pat as they are already included in primary cases) Rate: Mandatory statement of reasons* < 30%
	Share of study patients	Numerator: Patients who were included in a study with an ethical vote Denominator: Primary cases Rate: Target value ≥ 5%
	Pre-therapeutic histological confirmation	Numerator: Primary cases of the denominator with pre-therapeutic histological diagnosis confirmation by punch or vacuum-assisted biopsy Denominator: Primary cases with initial surgery and histology of invasive mammary carcinoma or DCIS Rate: Target value ≥ 90%
Treatment	Primary cases mammary carcinoma	Number Primary Cases Target value ≥ 100
	Number of surgical procedures for R0 resection for BCT	Numerator: Primary cases of the denominator with only one surgical procedure up to final surgical condition BCS Denominator: Surgical primary cases with BCS and RO Rate: Mandatory statement of reasons* < 70%
	Breast-conserving procedure for pT1	Numerator: Number BCT (the final surgical state with pT1 (incl. (y)pT1) Denominator: Surgical primary cases with pT1 (incl. (y)pT1) Rate: Target value 70–90%
	Mastectomies	Numerator: Mastectomies (final surgical stage) Denominator: Surgical primary cases Rate: Mandatory statement for reasons* 40%
	Lymph node removal in the case of DCIS	Numerator: Primary cases with axillary lymph node removal (primary axillary lymph node removal or sentinel lymph node removal) Denominator: Primary cases DCIS and completed surgical therapy and BCT Rate: Target value ≤ 5%
	Determination of nodal status in case of invasive mammary carcinoma	Numerator Primary cases with invasive mammary carcinoma for which the nodal status has been determined Denominator Surgical primary cases with invasive mammary carcinoma (without primary M1) Rate: Target value ≥ 95%
	Only sentinel lymphonodectomy (SLNE) for pNO (women)	Numerator: Female primary cases with sole sentinel lymph node removal (SNB) Denominator: Female primary cases of invasive mammary carcinoma and negative pN staging and without preoperative tumor-specific therapy Rate: Target value ≥ 80%
	Intraoperative sample radiography/ sonography	Numerator: Operations with intraoperative preparation X-ray or with intraoperative preparation sonography Denominator: Surgical procedures with preoperative wire marking guided by mammography or sonography Rate: Target value ≥ 95%
	Revision surgeries	Numerator: Revision surgery due to postoperative complications (only operated primary cases) Denominator: Surgical primary cases Rate: Target value ≤ 5%
	Therapy of the axillary lymphatic drainage for pN1mi	Numerator: Primary cases with therapy (axilla dissection or radiotherapy) of the axillary lymph drainage areas Denominator: Primary cases with invasive breast carcinoma, pN1mi Rate: Target value ≤ 5%

*For values outside the plausibility limit(s), the centers are required to provide a justification. Authors’ own summary based on Annual Report 2020 of the Certified Breast Cancer Centres (BCCs). The audit year 2019/the indicator year 2018 [18]; BCS — breast conserving surgery; BCT — breast conserving therapy; DCIS — Ductal carcinoma in situ; M1 — metastasis in TNM classification

Scotland

The Better Cancer Care plan, developed in 2008, included a commitment to prepare a program that would define how indicators for the quality of oncology services would be developed. To achieve this, the Scottish Cancer Taskforce established the National Cancer Quality Steering Group (NCQSG) which is responsible for:

development of small sets of quality performance indicators (QPIs) nationwide (approximately 10-15) which are specific to a given tumor

overseeing the implementation of the national management framework, which is the basis for reporting results concerning national QPIs,

ensuring the sustainability of the work of this program.

QPIs have been developed in collaboration with three Regional Cancer Networks (NOSCAN, SCAN, WOSCAN), the Information Services Division (ISD), and Healthcare Improvement Scotland. It is assumed that the QPIs will be regularly reviewed to reflect advances in scientific knowledge and changes in clinical practice. The overarching goal of the work program on the quality of cancer care was to ensure that activities at the level of the National Health Service (NHS) board were focused on the areas most important to improving the survival and quality of life of patients while ensuring safe, effective, and patient-centered oncology care [19].

The process of developing Scottish Quality Indicators is illustrated by the example of indicators for breast cancer. The stages of the process are described in Annex 1 to the document ‘Breast Cancer Clinical Quality Performance Indicators v4.0’.

Stage I — Preparatory work and determining the scope of the undertaking

Since NHS Quality Improvement Scotland Clinical Standards for Breast Cancer had been used nationally since 2001, it was agreed that instead of undertaking a lengthy QPI development process, extensive literature searches and discussion with clinicians should be conducted as part of the NHS QIS review (in 2008). These standards were used as the basis for the development of the QPIs. Preparatory work included independent peer review by development group members and an evaluation of existing NHS QIS Breast Cancer Standards against agreed criteria. Potential areas for the development of new quality indicators that were focused on results were also identified. The results of the above work were used in discussions by development groups in subsequent stages of the process.

Stage II — Development of indicators

The Breast Cancer Indicators Development Group QPIs has defined evidence-based, measurable indicators with a clear focus on improving the quality and outcomes of care provided. The group developed the QPIs on the basis of existing NHS QIS clinical standards. QPI projects were assessed by the Breast Cancer QPI Development Group according to three criteria:

General significance — does the indicator reflect measures of clinical significance, which may have a meaningful influence on the quality and results of the care delivered?

Based on scientific evidence – is the indicator based on high-quality scientific evidence?

Measurability — is the indicator measurable, i.e. are there clear criteria regarding the measurement of variables?

Stage III — The consultation process

In 2011, extensive clinical and social work was undertaken as part of the development work for quality indicators, whereby the QPIs for breast cancer were made available on the Scottish Government website, along with a pilot test for collecting a minimum core dataset and specifications for measurability. During the consultation period, clinicians and stakeholders from across NHS Scotland, breast cancer patients, and other interested parties had the opportunity to influence the development of the QPIs for breast cancer.

QPIs are designed to be clear and measurable, based on high-quality clinical evidence; they take into account other recognized standards and guidelines.

Each QPI has a short title that is used in reports, as well as a more complete description that explains exactly what each indicator measures.

The indicator provides a brief overview of the evidence base and a rationale that explains why it was important to develop the indicator.

The indicator has general and detailed specifications for measurability that highlight how the indicator will be measured in practice to enable comparison within NHS Scotland.

For each QPI, a target has been defined that reflects the expected level of quality that clinical sites should be aiming for (“value less than ...” or “value greater than ...”).

To ensure that the chosen target levels are the most appropriate and lead to continuous quality improvement as intended, they are evaluated and revised as newer scientific evidence or data become available. Due to the difficulty in accurately selecting patients, comorbidities, and the general condition of patients, a degree of tolerance has been built into QPIs, and target levels were set taking into account the above factors. If there are other factors that influence the target level of the indicator, this was also noted in the detailed QPI description [20]. Examples of Scottish Quality Indicators in Breast Cancer Care are shown in Table 3.

Table 3. Examples of Scottish Quality Indicators in Breast Cancer Care

Area	Name of indicator	Specifications
Diagnosis	Referral for Genetics Testing Patients with breast cancer should be offered referral to a specialist genetics clinic where appropriate	Numerator: Number of patients with breast cancer under 30 years of age referred to a specialist clinic for genetic testing Denominator: All patients with breast cancer who are under 30 years of age. Exclusions: No exclusions
	Genomic Testing Patients with breast cancer should be offered genomic testing where appropriate	Numerator: Number of patients with ER-positive, HER2-negative, node-negative breast cancer who have a 3–5% overall survival benefit of chemotherapy treatment predicted at 10 years that undergo genomic testing Denominator: All patients with ER-positive, HER2-negative, node-negative breast cancer who have a 3–5% overall survival benefit of chemotherapy treatment predicted at 10 years Exclusions: — patients with breast cancer taking part in clinical trials of chemotherapy treatment — patients who undergo neoadjuvant therapy
Treatment	Immediate Reconstruction Rate Patients undergoing mastectomy for breast cancer should have access to timely immediate breast reconstruction	Numerator: Number of patients with breast cancer undergoing immediate breast reconstruction at the time of mastectomy Denominator: All patients with breast cancer undergoing mastectomy Exclusions: — all patients with M1 disease — all male patients
	Minimizing Hospital Stay Patients should have the opportunity for day case/“23-hour”* breast surgery wherever appropriate	This QPI measures two distinct elements. (I) Patients with breast cancer undergoing wide excision and/or an axillary sampling procedure as day case surgery; and (II) Patients with breast cancer undergoing mastectomy (without reconstruction) with a maximum hospital stay of 1 night following their procedure (I) Numerator: Number of patients with breast cancer undergoing wide excision and/or axillary sampling procedure (sentinel node biopsy or 4 node sample) as day case surgery Denominator: All patients with breast cancer undergoing wide excision and/or axillary sampling procedure (sentinel node biopsy or 4 node sample) Exclusions: — all patients with breast cancer undergoing partial breast reconstruction/mammoplasty (II) Numerator: Number of patients with breast cancer undergoing mastectomy (without reconstruction) with a maximum hospital stay of 1 night following their procedure. Denominator: All patients with breast cancer undergoing mastectomy (without reconstruction) Exclusions: No exclusions
	HER2 Status for Decision Making HER2 status should be available to inform treatment decision-making	Numerator: Number of patients with invasive breast cancer for whom the HER2 status (as detected by immunohistochemistry [IHC] and/or FISH analysis) is reported within 2 weeks of core biopsy Denominator: All patients with invasive breast cancer Exclusions: — patients in whom no invasive carcinoma is present on core biopsy
	Radiotherapy for Breast Conservation in Older Adults Radiotherapy use should be reduced in patients ≥ 70 years of age with early-stage breast cancer and a low risk of recurrence	Numerator: Number of patients ≥ 70 years with T1 N0, ER-positive, HER2-negative, LVI-negative, Grade I to II breast cancers undergoing conservation surgery (completely excised with margin ≥ 1mm) with hormone therapy who receive radiotherapy Denominator: All patients ≥ 70 years with T1 N0, ER-positive, HER2-negative, LVI negative, Grade I to II breast cancers undergoing conservation surgery (completely excised with margin ≥ 1mm) with hormone therapy Exclusions: — all patients with breast cancer taking part in clinical trials of radiotherapy treatment
	Adjuvant Chemotherapy Patients with breast cancer should receive chemotherapy post-operatively where it will provide a survival benefit for patients	Numerator: Number of patients with hormone receptor (ERplus/minus PR) positive, HER2-negative breast cancer who have a > 5% overall survival benefit of chemotherapy treatment predicted at 10 years and/or high-risk genomic assay score that undergo adjuvant chemotherapy Denominator: All patients with hormone receptor (ER plus/minus PR) positive, HER2-negative breast cancer who have a >5% overall survival benefit of chemotherapy treatment predicted at 10 years and/or high-risk genomic assay score Exclusions: — all patients with breast cancer taking part in trials of chemotherapy treatment — all patients with breast cancer who have had neoadjuvant chemotherapy — all patients with M1 disease
	Re-excision Rates Patients undergoing surgery for breast cancer should only undergo one definitive operation where possible	Numerator: Number of patients with breast cancer (invasive or in situ) having breast conservation surgery who undergo re-excision or mastectomy following initial breast surgery Denominator: All patients with breast (invasive or in situ) cancer having breast conservation surgery as their initial or only breast surgery Exclusions: — LCIS alone
	30 Day Mortality following Systemic Anti-Cancer Therapy (SACT) 30-day mortality following Systemic Anti-Cancer Therapy (SACT) treatment for breast cancer	Numerator: Number of patients with breast cancer who undergo SACT that die within 30 days of treatment Denominator: All patients with breast cancer who undergo SACT Exclusions: No exclusions Please note: This indicator will be reported separately for neoadjuvant, adjuvant, and palliative chemotherapy, as opposed to one single figure
	Clinical Trial and Research Study Access All patients should be considered for participation in available clinical trials/research studies, wherever eligible	Numerator: Number of patients diagnosed with breast cancer who consented to a clinical trial/research study Denominator: All patients diagnosed with breast cancer Exclusions: No exclusions
	Neoadjuvant Chemotherapy Patients with breast cancer who receive chemotherapy should be offered neoadjuvant chemotherapy to achieve pathological complete response where appropriate	Numerator: Number of patients with triple-negative or HER2-positive, Stage II or III ductal breast cancer who receive chemotherapy that undergo neoadjuvant chemotherapy Denominator: All patients with triple-negative or HER2-positive, Stage II or III ductal breast cancer who receive chemotherapy Exclusions: — patients who undergo palliative chemotherapy
	Deep Inspiratory Breath Hold (DIBH) Radiotherapy Patients with left-sided breast cancer or DCIS undergoing adjuvant radiotherapy treatment should use a deep inspiratory breath-hold (DIBH) radiotherapy technique	Numerator: Number of patients with left-sided breast cancer or DCIS receiving adjuvant radiotherapy treatment who use a DIBH radiotherapy technique Denominator: All patients with left-sided breast cancer or DCIS receiving adjuvant radiotherapy treatment Exclusions: No exclusions

*Same-day surgery is defined as cases of such patients who were admitted and discharged on the same day that the procedure was performed. A 23-hour surgery is defined as surgery that involves a maximum of one night after surgery. Authors’ summary based on: Breast Cancer Clinical Quality Performance Indicators August 2019 (v4.0) [20]

USA

Following the 1999 report by the Institute of Medicine (IOM) entitled “Ensuring Quality Cancer Care” on the provision of high-quality cancer care and the growing problems in this part of the healthcare system, the American Society of Clinical Oncology (ASCO) initiated the National Initiative on Cancer Care Quality (NICCQ). A study was conducted in five large urban areas, providing key data on the quality of cancer care that, in the opinion of researchers, needed systemic improvement. NICCQ experts pointed to challenges related to the implementation of a quality monitoring system on a national scale, and above all, to the need to meet key requirements, which were considered to represent accurate measurement and reporting of the quality of oncological care at the lowest possible cost and achieving results that provide support and information about activities that promote quality improvement. Based on discussions with clinical experts, professional associations, and other stakeholders, researchers identified four key features that are critical to developing the NICCQ: representative patient sampling, patient privacy, appropriate measures of quality of care, and multiple data sources. The impossibility of self-assessment in the area of oncology, prompted Dr. Joseph Simone, working through ASCO, to propose the development of a methodology for improving quality in oncological care. A small network of oncologists was originally created to develop a methodology for implementing quality improvement in oncology, oncological care quality control measures and a data entry and reporting system. This working group of oncologists eventually formed the Quality Oncology Practice Initiative (QOPI).

The process of selecting experts to participate in the program was based on the modified Delphi method. The working group then defined the criteria for guiding measurement. In order to simplify the measurement process, the measures were originally binary i.e. yes/no, or agree/disagree. In addition, the measurements were also intended to address a number of important issues. Those occurring in guidelines for clinical practice or other similar standards of oncology care and based on high-quality evidence were considered of primary importance, but it was agreed not to restrict the choice to these alone. Thus, three basic sources of measures were taken into account:

consensus-based, determined by all participants in the QOPI program

evidence-based standards (i.e. those derived from published clinical practice guidelines and health technology assessment developed by scientific societies)

items related to patient/physician interactions required by organizations other than scientific societies.

Currently, the QOPI provides a system for measuring care processes at intervals of six months using a retrospective analysis of medical records. As part of the system, staff responsible for completing medical data select patients fulfilling the requirements for QOPI, starting with those most recently observed in clinical practice and going back in time up to 6 months to meet the minimum sample size. The minimum number of reported patients is determined by the number of full-time oncologists or hemato-oncologists at the center.

Data are transferred to the QOPI database using a structured, secure online form and are analyzed and reported back to the center. Reports are available within 4 weeks after data collection, so centers can use the results to improve the quality of cancer care. For each quality measure, the reports include detailed practice data and comparative aggregate data [21–24].

Examples of American Quality Indicators in Breast Cancer Care are shown in Table 4.

Table 4. Examples of American Quality Indicators in Breast Cancer Care

Module	Name of indicator (*QOPI® Certification Measure)
Core	Pathology report confirming malignancy*
	Staging documented within one month of first office visit*
	Pain addressed appropriately
	Pain assessed on either of the two most recent office visits*
	Documented plan for chemotherapy, including doses, route, and time intervals*
	Chemotherapy intent (curative vs. non-curative) documented before or within two weeks after administration*
	Chemotherapy intent discussion with patient documented*
	Documented plan for oral chemotherapy: Dose*
	Documented plan for oral chemotherapy: Administration schedule (start day, days of treatment/rest and planned duration)*
	Patient consent for chemotherapy*
	Smoking status/tobacco use documented in past year*
	Patient emotional well-being assessed by the second office visit*
	Action taken to address problems with emotional well-being by the second office visit*
	Height, Weight, and BSA documented prior to chemotherapy*
Breast	Combination chemotherapy received within 4 months of diagnosis by women under 70 with AJCC stage IA (T1c) and IB — III ER/PR negative breast cancer*
	Test for Her-2/neu overexpression or gene amplification*
	Tamoxifen or AI received within 1 year of diagnosis by patients with AJCC stage IA (T1c) and IB — III ER or PR positive breast cancer*

Source: ASCO QOPI® Certification Track 2021 Measures Summary, https://practice.asco.org/quality-improvement/quality-programs/quality-oncology-practice-initiative/qopi-related-measures [25]

Canada (Ontario)

The development of quality indicators involves several steps and is the responsibility of the Quality Standards Advisory Committee (QSAC). The process for developing indicators starts after draft quality statements have been approved. However, literature reviews and work related to the environmental scanning process begin before the draft statements are agreed upon. The process of indicator development is illustrated in Figure 2.

Figure 2. Development process — Canada (Ontario) [26]

The first step involves the identification of results that the QSAC prioritizes as being essential to the quality assessment process. The selected results should be in line with the quality criteria set out in Quality Matters, i.e. safety, effectiveness, patient orientation, efficiency, timeliness, and equity. Next, the QSAC selects a limited set of results that reflect and may affect the objectives set out in the quality standard. The results should be factors that can be reasonably expected to be influenced by the adoption of the quality standard in the entire province. At the start of each quality standards assessment project, the Health Quality Ontario team reviews the existing literature on the subject of quality issues and environmental analysis. The literature review primarily includes an international inventory of existing quality indicators (with associated definitions and validation information). It is important to identify functioning indicators so that the QSAC can determine priorities with minimal delay. The environmental analysis focuses on measuring, reporting, and collecting data in a given province. It describes the existing reporting activities, methods of analysis, and tools including previous activities, plans, or reports on quality improvement. It also describes the existing datasets that compile information relevant to the quality standard. On the basis of the results of the literature review and environmental analysis, the Health Quality Ontario team compiles a shortlist of potential indicators for prioritization (if necessary) by the QSAC. The draft indicators are then made available for consultation and the “technical details” for each indicator are agreed by a panel of experts (together with QSAC members). This process includes determining whether an indicator can be calculated using the datasets at the disposal of the provinces, delineating how the indicators should be measured, and specifying an alternative if it is not possible to measure a particular indicator, as well as stipulating any restrictions for each of the indicators. Once the proposal has been developed, each suggested indicator undergoes two phases of public consultations. It is first sent to organizations representing patient interests, and then the project is posted on the Health Quality Ontario website for 3 weeks of public consultation to obtain patient feedback. All the feedback received is analyzed and thematically synthesized in the consultation report. A draft consultation report is sent to the QSAC to provide background for the final meeting, in which the QSAC and a team of experts discuss any proposed changes or amendments to the quality standard. Then, a public version of the consultation report is prepared, which describes the feedback, comments and suggestions received, as well as any changes made to the quality standard along with a justification for each of the changes. The report also indicates where the QSAC chose not to make changes, along with the rationale behind that decision. Then the set of indicators is published with recommendations for their implementation by different stakeholders, in order to support the continuous process of quality improvement [26, 27]

The quality-of-care assessment carried out in Canada may vary from province to province. Apart from the regional quality assessment programs, nationwide evaluations are also carried out. An example of this kind of exercise is the evaluation of the quality of a screening program for the early detection of breast cancer. Table 5 shows the indicators used to evaluate the program, broken down into 5 main areas.

Table 5. Examples of Canadian indicators for assessing the quality of a breast cancer screening program

Area	Name of indicator	Specifications
Coverage	Participation Rate	The participation rate is the percentage of women who have a screening mammogram within 30 months, as a proportion of the target population National target (50 to 69 years): ≥ 70% of the target population within 30 months
	Retention Rate	Retention rate is the estimated percentage of women aged 50 to 67 years who returned for screening within 30 months of their previous screen National target (50 to 67 years): ≥ 75% within 30 months of an initial screen; ≥ 90% within 30 months of a subsequent screen
	Annual Screening Rate	The annual screening rate is the estimated percentage of women who returned to screen within 18 months of their previous screen. Target: None
Follow-Up	Abnormal Call Rate	Abnormal call rate is the percentage of screening mammograms that are identified as abnormal National target (50 to 69 years): < 10% of initial screens; < 5% of subsequent screens
	Diagnostic Assessment	Most women who receive an abnormal screening result do not go on to be diagnosed with breast cancer; however, additional assessment is required to reach a definitive diagnosis. This can include additional imaging, core or open biopsy, and/or fine-needle aspiration (FNA)
	Diagnostic Interval	Time from screen to notification of screen result National target (50 to 69 years): ≥ 90% within two weeks Time from abnormal screen to first diagnostic assessment National target (50 to 69 years): ≥ 90% within three weeks Time from abnormal screen to definitive diagnosis National target (50 to 69 years): ≥ 90% within five weeks if no tissue biopsy is performed; ≥ 90% within seven weeks if tissue biopsy (core or open) is performed
Quality of Screening	Non-Malignant Biopsy Rate	The percentage of non-malignant open surgical biopsies is the percentage of non-malignant biopsies which were open surgical biopsies National target: No target established
	Positive Predictive Value (PPV) of the Screening Mammography Program	The positive predictive value (PPV) of the screening mammography program is the percentage of abnormal cases diagnosed with breast cancer (invasive or in situ) after diagnostic workup National target (50 to 69 years): ≥ 5% for initial screens; ≥ 6% for subsequent screens
	Sensitivity of the Screening Mammography Program	The sensitivity of the screening mammography program is the percentage of breast cancer cases (invasive and in situ) that were correctly identified as cancer during the screening episode National target: No target established
	Post-Screen Invasive Cancer Rate	The post-screen invasive cancer rate is the number of invasive breast cancers found after a normal or benign mammography screening episode within 0 to < 12 months and 12 to 24 months of the screening date, per 10,000 person-years of follow-up. National target (50 to 69 years): < 6 per 10,000 person-years within 0 to < 12 months of the screen date; < 12 per 10,000 person-years within 12 to 24 months of the screening date
Detection	In Situ Cancer Detection Rate	In situ cancer detection rate is the number of ductal carcinomas in situ (DCIS) cancers detected per 1,000 screens National target: No target established
	Invasive Cancer Detection Rate	Invasive cancer detection rate is the number of invasive cancers detected per 1,000 screens National target (50 to 69 years): > 5 per 1,000 initial screens; > 3 per 1,000 subsequent screens
	Percent Ductal Carcinoma in Situ	Percent ductal carcinoma in situ is the percentage of all cancers detected that are DCIS National target: No target established
Disease Extent at Diagnosis	Screen-Detected Invasive Tumor Size	Screen-detected invasive tumor size is the percentage of screen-detected invasive cancers with a tumor size ≤ 15 mm in greatest diameter as determined by the best available evidence: 1) pathological, 2) radiological, and 3) clinical National target (50 to 69 years): > 50% screen-detected invasive tumors ≤ 15 mm
	The proportion of Node Negative Screen-Detected Invasive Cancers	The proportion of node-negative screen-detected invasive cancers is the percentage of screen-detected invasive cancers in which cancer has not invaded the axillary lymph nodes as determined by pathological evidence National target (50 to 69 years): > 70% of screen-detected invasive cancers

Authors’ summary based on: Canadian Partnership Against Cancer. Breast Cancer Screening in Canada: Monitoring and Evaluation of Quality Indicators – Results Report, January 2011 to December 2012. Toronto: Canadian Partnership Against Cancer; 2017 [28]

Japan

Under the Basic Cancer Control Act of 2006, efforts to ensure the quality of cancer care in many sectors have been intensified, amongst others, the role of the government in ensuring the quality of cancer care has been increased. Accordingly, a government-funded research project was launched to develop quality indicators in five main areas (breast, lung, stomach, colon, and liver cancer).

The project was primarily aimed at assessing how current best practices are applied, rather than assessing the overall usefulness of services (such as waiting times); however, the study also raised a number of issues regarding the concepts and methodology used to measure quality.

The project used a methodology developed by scientists from the University of California, Los Angeles, and the RAND Corporation. Thus, under the adopted methodology, a set of potential quality indicators was created, evidence was collected in support of these indicators and a discussion panel composed of multidisciplinary experts was summoned to conduct an analysis in two rounds of evaluation. The panel used a simple grading scale from 1 to 9 to evaluate the indicators. The first assessment was made before the meeting of the working group, while the next assessment took place following a discussion within the expert panel. As part of the final list, a set of 206 indicators were obtained, all of which described the care standards together with the target group of patients to whom they would be applied [29].

Currently, new research is being undertaken to create measures using the methodological experience gained so far, in a multi-stage approach including the selection of potential indicators based on the scientific literature, the creation of a multidisciplinary expert panel, and a two-round verification of the survey results [30]. Examples of Japanese Quality Indicators in Breast Cancer Care are shown in Table 6.

Table 6. Examples of Japanese Quality Indicators in Breast Cancer Care

Area	Name of indicator	Specifications
Diagnosis	Hormone receptor test	Numerator: Patients with known ER-positive or negative Denominator: All cases of breast cancer (excluding unknown ER)
	HER-2 test	Numerator: Patients who had HER-2 testing (in cases of HER-2 2+, positive or negative was determined by FISH) Denominator: All cases of breast cancer
Treatment	Adherence to St. Gallen consensus recommendations	Numerator: Patients who received post-operative therapy adherent to the St. Gallen consensus recommendations Denominator: Post-operative cases with invasive breast cancer
	Appropriate hormone therapy	Numerator: Post-operative hormone therapy (tamoxifen, toremifene, anastrozole, exemestane and letrozole) Denominator: Post-operative cases with invasive breast cancer positive
	Appropriate post-operative chemotherapy regimen	Numerator: Patients who received either regimen including anthracyclines, taxanes or CMF Denominator: Invasive breast cancer cases with implementation of post-operative adjuvant chemotherapy
	Radiation therapy after breast-conserving surgery	Numerator: Patients who received post-operative radiation therapy Denominator: Patients 70 years old or younger who underwent breast-conserving surgery for stage I–III breast cancer
	Lymph node dissection for ≥ N1 patients	Numerator: Patients who had lymph node dissection of level I or above Denominator: Patients who had surgery for N1–3 breast cancer

Source: Mukai H, Higashi T, Sasaki M. Quality evaluation of medical care for breast cancer in Japan, International Journal for Quality in Health Care, 2016, 28(1), 110–113 doi: 10.1093/intqhc/mzv109 [31]

From the comparison presented above, it can be seen that the process of developing quality indicators in all of the care systems analyzed is conducted in an analogous manner. An important element in the development of quality indicators is their validation and evaluation, which allow for the creation of the most optimal sets of indicators. With time, when the value of a given indicator reaches the target level of performance, it is withdrawn from the measurement process, as its importance for the assessment of the quality of care ceases to be important. Drawing on the practices of other countries would seem to be an optimal solution for the Polish healthcare system, bearing in mind that most of the programs for cancer care quality assessment are based on American experiences, which laid the foundation for quality assessment in the healthcare system. A summary of the results of the review is presented in Table 7.

Table 7. Summary of the results of the review

Comparative feature	Poland	Australia	Germany	Scotland	USA	Canada	Japan
Guidelines for the development of defined indicators	–	+	+	+	+	+	+
Clinical practice guidelines used as the main source for developing QIs	–	+	+	+	+	+	+
A review of the current quality indicators	–	+	+	+	+	+	+
A multi-disciplinary expert panel established	–	+	+	+	+	+	+
Extensive public consultations carried out	–	+	–	+	+	+	+
The results of the indicators made available to the public	–	+	+	+	+	+	+
Cyclical assessments and adjustment of indicators to changing conditions	–	+	+	+	+	+	+

Discussion

Within oncological care in the Polish healthcare system, measures are taken to monitor quality with the use of quality indicators. Currently, quality assessment of oncology care is conducted as part of the oncology package for services provided to breast-cancer patients and in the oncology network provider care pilot program.

These are the two most advanced initiatives attempting to measure quality in oncology. In the directive issued by the Minister of Health on 24 May 2019, amending the regulation on guaranteed services in the field of hospital treatment (Journal of Laws of 2019, item 1062), there are detailed conditions that healthcare providers should meet in the provision of comprehensive oncological care for patients with breast cancer. In terms of the quality of service provision, reference is made to the Act issued on 27 August 2004 on healthcare services financed from public funds (Journal of Laws 2021 item 1285) (hereinafter: the Act on Benefits):

“The provider calculates the effectiveness of cancer diagnostics and treatment in accordance with Article 32c of the Act and meets or aims to meet the designated threshold values of the indicators, if defined. A standardized, written protocol for diagnosis and therapeutic management is to be used at all stages of advancement.” [32]. Before the directive entered into force, the Minister of Health published, by means of an announcement, the assessment measures for conducting oncological diagnostics and oncological treatment for breast cancer. The announcement defines 14 measures that can be used to calculate the effectiveness indicators for cancer diagnostics and cancer treatment. The numerator and denominator as well as the rules determining their calculation were defined for each of the measures [33]. In accordance with the provisions set out in Art. 32c of paragraph 1 of the Act on Benefits:

“1. Service providers providing healthcare services in the field of oncological diagnostics and oncological treatment calculate annually the effectiveness indicators of oncological diagnostics and oncological treatment for the previous calendar year on the basis of data from statistical reports referred to in regulations issued under Art. 137 paragraph 2. The service providers shall provide the National Healthcare Fund with the effectiveness indicators for oncological diagnostics and cancer treatment no later than by the end of the second quarter of the calendar year following the year for which these indicators were calculated“ [34].

It is worth noting that the Act on Benefits clearly defines the basis on which data should be recorded and the time intervals over which it should be calculated to evaluate the results achieved by service providers in terms of quality indicators. It does not, however, define the rules for disclosing to the public information on the values achieved. In contrast to other countries, the Polish system has not yet established target values for quality indicators in cancer care [33]. This lack of criteria generally hampers the process of assessing the quality of care in individual organizations. A significant problem is also the fact that currently only those medical centers that provide services under the oncology package contracted with the public payer are obliged to calculate the effectiveness indicators in oncological care. According to information published by the National Healthcare Fund on the website ezdrowie.gov.pl, where quantitative data on surgical treatment for thirteen types of cancer can be found, it appears that in 2020 surgical treatment of breast cancer was carried out by 167 centers, 21 of which were above the threshold or potentially above the quality threshold, defined at the level of 250 procedures. It should be emphasized that this threshold of 250 procedures applies to those services that have been agreed under the terms of the oncology package. Centers in the potentially above-threshold category carried out more than 250 procedures, but to a large extent, these were performed outside the terms of the oncology package. [35]. Thus, the healthcare providers who choose not to charge for oncological services under the package are not required to calculate the effectiveness of oncological care. As a consequence, this leads to a situation in which the payer is not able to monitor the quality of oncological services provided in all centers throughout the country, nor is it possible to comprehensively assess the functioning of oncological care in Poland.

The assessment of the quality of oncological care is also carried out as part of an ongoing pilot program for evaluating the care of patients within the oncological network, which was introduced by a directive of the Minister of Health on 13 December 2018 (Journal of Laws 2018, item 2423, with later amendments). The aim of the project, whose completion is planned for 31 December 2021 [36], is to assess the organization, quality, and effects of oncological care within the oncological network in the four provinces covered by the program, i.e. Dolnośląskie, Podlaskie, Pomorskie, and Świętokrzyskie Voivodeships. The program covers 5 major organ tumors: colon, lung, breast, ovary, and prostate. The assessment of care within the framework of the oncological network is carried out in relation to patients covered by the pilot study, with respect to diagnoses, and the measures specified in the ordinance. Paragraph 9 of the directive defines 35 measures for the assessment of oncological care, including 15 general measures, 3 for colorectal cancer, 3 for prostate cancer, 4 for lung cancer, 3 for ovarian cancer, and 7 for breast cancer. The center responsible for drawing up and submitting reports to the provincial branch of the National Healthcare Fund is the Provincial Coordinating Centre (WOK) [37].

The evaluation of the results of the pilot project will be carried out by the provincial branches of the National Healthcare Fund in cooperation with the Provincial Coordinating Centre on the basis of the final report on the implementation of the pilot project. Overall assessment of the results of the program may be problematic due to difficulties caused by the quality assessment measures. Most of them are clinical measures, whose calculation from the historical data on the provision of services available to the public payer is significantly limited, because these data have not hitherto been collected by the National Health Fund. Understanding of the real benefits achieved by the pilot scheme would be easier if it was attempted to calculate the measures from the data of the hospitals involved in the pilot scheme retrospectively. However, the image of oncological care obtained in this way would be reflective only of the provinces covered by the pilot program. Achieving a more comprehensive evaluation reflecting results on a country-wide scale is not possible in a pilot scheme because measures for centers not participating in the program cannot be calculated. The lack of knowledge about the quality of oncological care in Poland prior to the start of the pilot program makes it difficult to answer the question of whether the program’s objective has been achieved. Nevertheless, the coordination of oncological care and cooperation within the oncological network has undoubtedly had a positive influence on increasing the satisfaction of cancer patients, shortening the waiting time for an appointment at oncology clinics, standardizing treatment processes, and increasing the importance of quality monitoring in oncology [38].

As already mentioned, the development of quality indicators is directly related to the guidelines for clinical practice. The long-term program “The National Oncological Strategy for 2020–2030,” adopted by the Council of Ministers and published in February 2020, assumes that by the end of 2021, national diagnostic and therapeutic guidelines and organizational standards in key malignant neoplasms will be developed, based on scientific evidence and taking into account the current conditions for financing health services from public funds. [39] It is worth noting that the overarching goal of this program is to increase the percentage of people surviving 5 years from the end of oncological therapy [40]. The guidelines published as an announcement by the Minister of Health should be a starting point for the development of cancer care quality indicators with a group of healthcare system stakeholders, which will be easy to implement in the Polish system and will contribute to quality improvement in priority areas. It would appear sensible to draw on the experiences of other countries and to consider implementing similar solutions in the Polish healthcare system. In these countries, most clinical data necessary to calculate indicators are collected directly from clinical registers, but in the Polish healthcare system, until the necessary database of clinical registers is built, it is worth considering the possibility of transferring the obligation to calculate indicators for the quality of oncological care from service providers to the payer. This would translate into complete information on the quality of care, taking into account all service providers, not only those implementing the oncology package.

The reporting system should be tightened up, and supplemented with key parameters necessary to calculate the indicators. Establishing multidisciplinary teams to develop quality indicators would allow for the identification of problematic areas in cancer care in Poland and the creation of key indicators which might have a significant impact on improving the quality of care. Such teams should include, in particular, representatives of scientific societies responsible for the development of clinical guidelines, representatives of the Minister of Health, representatives of the payer, system experts, and persons representing the interests of cancer patients. The desired endpoint of these consultations is the recommendation of quality indicators that would ensure maximum benefits from the point of view of the system, with a minimum administrative burden for both service providers and the payer. Currently, healthcare providers are subject to numerous reporting obligations, including the submission of a wide range of reports, which leads to considerable difficulties in task management and creates complications in identifying the appropriate information necessary for the process of assessing the quality of care. Reporting obligations include reports of significant complexity, which are required to satisfy the needs of a large group of various stakeholders, but also more selective reports, narrowly focused on specific problems, to meet the needs of individual groups of stakeholders, e.g. reports about the implementation of health policy, corporate governance, audit by the payer for health services, etc. Reporting by medical entities has a variety of goals, including control over spending public funds (e.g. within the budget of the National Healthcare Fund) [41].

The National Healthcare Fund as a state organization is obliged to collect reporting data on reimbursement of service providers by the National Healthcare Fund. Thus, in the context of measures or quality control, as well as the assessment of provided services, the diagnostic and therapeutic processes, or resources (e.g. human resources), it is currently, the largest administrator of relevant medical data.

Although using reporting data for measurement purposes (especially quality measurement) might seem easy to implement, there are several limitations, which make the process somewhat difficult. A significant problem, in terms of the National Healthcare Fund’s reporting system, is the scope of the data. The purpose of data collection by the National Healthcare Fund is not quality control. As a result, reported data are not sufficient to calculate quality indicators, especially where clinical exclusions are concerned. In addition, a very serious problem is the lack of access to data that would accurately reflect the issues related to the health of patients. This is closely related to a very important aspect concerning the appropriateness of data sets (continuously validated in real-time) and specially designed registers. Using the example of the NHS Scotland quality monitoring system, it can be seen that the measurement process included the creation of appropriate databases and a range of variables necessary for reporting. The process also included data validation at the service provider level, as well as periodic assessment of indicators, their correlation and compliance with the latest diagnostic and therapeutic guidelines [42, 43]. Moreover, the direct presentation of the results at the healthcare provider level may be biased due to the impossibility of excluding randomness in the information presented (unacceptably small samples of patients outside the system of basic hospital provision of healthcare services may cause large variances) [44–47].

Moreover, the results for the indicators should be modified to correct for the clinical characteristics of patients, taking into account clinical features such as age, sex, casemix, which might affect the result [48, 49], and data from the National Healthcare Fund may be insufficient in scope to provide these details. It is also worth noting that, based on National Healthcare Fund data, it is not possible to make a direct comparison between centers, and the use of measures to assess hospital performance in order to arrive at an “evidence-based” definition of centers deviating from the recommended standard level of care would require the use of statistical methods such as meta-analyses [50, 51]. Moreover, the use of quality indicators should be clearly distinguished to differentiate between those used at the service provider level (control of the internal process within one organization) and indicators used to compare different centers (benchmarking) [50, 51]. An important issue, from this perspective, is the creation of a statistical tool to use for comparative analysis of service providers (rankability).

Despite the significant limitations of the data available from the National Healthcare Fund, there are some possibilities for the measurement of the quality of healthcare in a sufficiently easy and at the same time precise manner. Of course, creating quality indicators based on available reporting data is subject to methodological limitations and requires appropriate validation, but the prospect appears promising. Ongoing (real-time) quality control is impossible because the data are submitted over pre-established reporting periods (usually monthly). However, under certain assumptions, it is realistic to observe processes or groups of patients post factum and react to any identified problems. A fairly common measure, that of readmission rates, can be used as an example. In the case of this measurement, it is first necessary to clearly define the patient population (e.g. patients with arterial hypertension, atrial fibrillation, etc.) to be investigated in relation to quality issues, with the aid of an appropriate indicator. It is possible to define a population of patients (e.g. patients admitted to a hospital) due to a specific health problem and patients re-admitted with an indication of the reason for re-hospitalization (e.g. based on the primary diagnosis). It is also possible to determine the time that has elapsed since the previous hospitalization and to exclude the possibility that the re-hospitalization is for other clinical reasons and was not planned (e.g. due to a procedure performed shortly after the previous visit) [52–54].

The above considerations are also directly related to the use of reporting data, e.g. ICD-10 or ICD-9 codes, as a method of quantifying certain phenomena. This is of great importance, for example, in the case of identifying those parties involved in the abuse of selected medical procedures or services or those avoiding certain groups of patients due to low point tariff values (called upcoding). In addition, the information collected by the National Healthcare Fund enables identification of the disease that is the direct cause of hospitalization, patient visit, examination, or surgery, using codes compliant with the currently functioning classification of diseases and health problems (indicated in the Ministry of Health Regulations). Reporting also includes codes of no more than three coexisting causes according to the ICD 10, excluding services in the field of primary healthcare [55]. The recorded data also contain information on the essential medical procedures performed according to the ICD 9, in the Polish version recognized by the payer as applicable in connection with reimbursement, and their appropriate inclusion when generating data or carrying out analyses makes it possible to obtain information that is not provided directly by healthcare providers (e.g. information concerning the next stage in cancer treatment or the probable stage of disease advancement).

Sometimes it is possible (as a result of following the entire diagnostic and therapeutic pathway), using an appropriate algorithm (e.g. flagging), to refine information on certain procedures and clinical conditions. The data resources of the National Healthcare Fund enable the determination of, among others, the patient’s sex and age, as well as the location of service provision and the patient’s place of residence. It is also possible to use data on comorbidities. However, this requires standardization of the method of data generation and the definition of an appropriate indicator expressing multiple morbidities. In addition, in the context of comparative analyses, direct comparison of centers is possible and can be used, for example, to identify “outliers” (such an approach should not, however, be used to draw conclusions about the quality of care!). This method should only be used to identify treatment centers that require observation, to identify the reasons for patients’ withdrawing from treatment in specific centers, and for defining problems throughout the entire process, thus providing an appropriate response to problems that may have arisen.

Moreover, the use of data collected for reporting purposes at the service provider (and payer) level is associated with a lower cost than creating a new register. An important limitation of this approach is the phenomenon of reporting better-paid procedures (upcoding) or “fragmented” coding of services, i.e. dividing services that could be accounted for as one into several separate procedures (unbundling) [56], where the extraction of relevant clinical data may be associated with additional costs. With this approach, the appropriate reporting regime imposed by the overseeing institution, e.g. by a public payer, should also be observed. Redirecting the obligation to calculate quality indicators to the payer also opens the possibility of creating a system of financial incentives for the best performers by shifting from financing services to paying for results. Performance based payment schemes have been developed because the traditional fee-for-service system rewards providers for the quantity and complexity of the services they deliver. The service-fee system encourages higher care intensity, but not a higher quality of care, and has contributed to an increase in healthcare costs. Performance-based payment programs encourage higher-quality healthcare while lowering costs. Typical compensation for performance programs provides a financial bonus to healthcare providers if they meet performance criteria for healthcare quality indicators [57]. In the United States, the introduction of alternative payment models (APMs) for financing health services improved the quality of oncological care while reducing costs, even despite its failure to report many important health conditions. Effective integration of quality initiatives with a healthcare reimbursement structure is likely to be the key to the long-term success of APMs in cancer care [58].

Conclusions

The efforts currently undertaken within the Polish healthcare system aimed at improving quality of care should lead to the development of a transparent methodology for the development, measurement, publication of results, and evaluation of cancer care quality indicators. Provided that adequate resources are committed, this may be facilitated by the expansion of the powers and tasks of the National Healthcare Fund within the National Oncological Network planned by the Ministry of Health. The Fund will be responsible, inter alia, for the administration and operation of an integrated IT system for the oncological network and the analysis of quantitative and qualitative parameters for oncological care achieved by individual centers included in the network. [36] The process of developing care quality indicators for individual disease entities must be designed in a manner adapted to the needs of the Polish healthcare system, though the experiences of other countries in this area may constitute the foundation. Bearing in mind the changes that will be introduced by the Act on Quality in Healthcare or the Act on the National Oncological Network [59] announced by the Ministry of Health, it is necessary to consider the possibilities for measuring quality with the use of oncological care quality indicators, developed in a systematic manner and of confirmed utility in the healthcare system. The quality assessment system in oncological care based on quality indicators allows patients to make an informed choice about the center in which their treatment will be conducted, and in the case of a public payer, it opens up new possibilities for financing services.

Given the current shape of the reporting and financial reimbursement system, in which the National Healthcare Fund collects a significant amount of information necessary to implement statutory tasks, minor modifications in the manner and scope of the data provided may contribute to the creation of an environment conducive to quality measurement with the use of indicators. Moreover, these are solutions that can be implemented “immediately”, they do not require significant financial outlays, and at the same time, can bring potentially large benefits towards optimization of the current system. For each set of quality indicators presented in this review, it is possible to select a few that are easily measurable under the Polish system, while others would require the introduction of several additional variables. According to the theory of marginal profit aggregation, introducing even small modifications (improvements) in the information collected and provided to the payer by service providers may seem insignificant from the point of view of an individual medical organization but may be extremely useful in the process of quality assessment in oncological care from the perspective of the entire healthcare system.

Funding

No funding was received from any sources.

Conflict of interest

The authors do not have any conflict of interests to declare