Abstract
Objective
To evaluate the use of quality indicators by obstetric caregivers in hospitals in the Netherlands.
Study design
An anonymous, self-administered survey was conducted in a convenience sample of obstetricians and clinical midwives in Dutch hospitals. Descriptive statistics were used to analyse the data, both for all caregivers and stratified by hospital setting and profession (obstetricians and midwives). Differences between strata were tested at a 5% significance level.
Results
The response rate to the online questionnaire was 61% (n = 171/279). Of all respondents 83% were aware of the quality indicators and 63% contributed to their registration. Caregivers received information about the indicators by mail or in meetings according to 64% (internal indicators) and 48% (external indicators) of the respondents. Of the respondents 56% (internal indicators) and 41% (external indicators) stated to use the results of indicators when designing plans to improve the quality of care.
Conclusion
We conclude that obstetric quality indicators are not widely used by obstetricians and midwives in Dutch hospitals to improve quality of care. To improve quality of care and the effective use of quality indicators we suggest to focus first on registering outcome indicators. These indicators should be implemented in quality structures that ensure that action is taken.
Keywords
Introduction
Quality assessment has become more important in health care since the Institute of Medicine published the report ‘To err is human’ [
[1]
] followed by ‘Crossing the Quality Chasm’ [[2]
]. Since then there has been increasing awareness of the importance to define and measure quality of care [[3]
]. Quality assessment tools such as audits, accreditation, patient safety systems and quality indicators have been developed [4
, 5
].In recent years, quality indicators have gained prominence on national and international agendas for health policy and research. The most widely accepted definition of quality indicators is: ‘Measurable elements of practice for which there is evidence or consensus that they reflect quality and hence help change the quality of care provided’ [
[6]
]. Quality indicators aim to detect suboptimal care and can be used as a tool in the process of quality improvement in health care [[7]
]. Care providers need a feedback loop to learn and improve the quality of care. Measuring, reporting and comparing outcomes are important steps towards improvement of outcomes [[8]
]. This implies that caregivers should use indicators to improve the quality of care they provide [[9]
]. Requirements for good indicators are that they are acceptable for the care providers, easy to capture and valid to the events and changes they are intended to detect [9
, 10
, 11
].There are two principal uses of indicator systems in the Netherlands: internal and external. Internal indicators are used in a formative mechanism for internal quality improvement. These indicators are developed to be used by clinicians. They can interpret and discuss these indicators to improve the quality of care within their control. External indicators are developed for external accountability, verification (summative mechanism) and external control. Information from these indicators may be useful for patients to compare providers. In addition they are instrumental to management or government to benchmark and to make policies for the improvement of care at the institutional or system level [
[12]
].Worldwide many quality indicators for obstetric care have been developed [
[9]
]. Awareness of quality indicators among caregivers is important. In addition, the indicators must be implemented in an active quality system with a feedback loop [[8]
]. In response to the poor ranking of Dutch fetal and neonatal mortality rates in the Euro-Peristat Project in 2004, policy efforts such as audits of perinatal deaths in term babies were instigated. This led, in 2010, to a decline in fetal mortality at or after 28 weeks gestation and neonatal mortality after 24 weeks gestation [[13]
]. This is an example of how indicators are used at policy level. However it is not yet clear whether and to what extent quality indicators are used by the care providers in the Netherlands [[14]
]. The aim of this study was to explore the awareness and the use of quality indicators by obstetric caregivers in their quality cycle in hospitals in the Netherlands.Materials and methods
Sample and data collection
The study group consisted of obstetricians and clinical midwives, who are the professionals responsible for deliveries in Dutch hospitals. In august 2014 an electronic survey was sent to 279 obstetricians and midwives who were registered in the database of the Masterprogram for Physician Assistant clinical midwifery at Rotterdam University (convenience sample). The questionnaire was anonymous and self-administered to protect confidentiality. Non-responders were not identified and no reminders were sent.
Content survey
The full questionnaire is attached as Addendum 1. The survey was constructed according to the quality steps ‘plan-do-study-act’ [15]. This structure is widely accepted in healthcare improvement and the principles underpin the PDSA-cycle for use in a pragmatic healthcare setting [16]. ‘Plan’ refers to the introduction of quality tools in the maternity wards. The participating caregivers were asked about their knowledge of quality tools such as the multidisciplinary perinatal audit to reflect on perinatal morbidity and mortality, client satisfaction measurement and quality indicators (Q7). As we were specifically interested in quality indicators the rest of the survey was about the use of internal and external quality indicators. For the internal indicators we selected the structure indicators implemented by the Dutch Society of Obstetricians and Gynaecology in 2008. For the external indicators we used the indicators as drafted by the Dutch Healthcare Inspectorate [17].
‘Do’ means implementing guidelines into protocols for single individual maternity units. To assess this, the respondents were asked if they actually implemented the suggested quality tools in their maternity ward for quality improvement (Q7). ‘Study’ is the evaluation of care provided by the caregivers. This step is designed to connect outcomes, guidelines, protocols and indicators. We assessed this step by asking in what way the quality indicators were communicated to the caregivers (Q9, 15).
In the last step, ‘act’, the caregiver is supposed to formulate recommendations for quality improvement. We asked the caregivers if the results of indicators were discussed in their professional group and if improvement plans were based on these results (Q10, 16).
To check if the respondents knew that certain quality tools, for example the presence of a protocol for treatment of acute shock/circulatory shock, are registered as indicators. We listed a selection of indicators set up by the Dutch Society of Obstetricians and Gynaecology. We asked if the structure was present in their department and if it was used as an indicator (Q8). For each step we included questions with a yes or no answer option and finished the questionnaire with an open question: ‘what else do you know or do in this step/example?’ The survey was tested with the help of a small number of midwives and obstetricians.
Statistical analysis
The survey was analysed descriptively, with numerical data presented with median and interquartile ranges (IQR P25–75) and categorical data presented as percentages. Descriptive statistics were presented both overall and stratified by hospital setting and type of caregiver. As it was a convenience sample, summary statistics describing the hospitals and caregivers were also compared to national statistics where available. Differences between strata were tested by Fisher's exact test or Pearson Chi-Square. We considered a p-value < 0.05 as statistically significant. All analyses were done in SPSS Statistics version 22.
Results
Two hundred seventy-nine obstetric caregivers were invited to fill in the questionnaire. The response rate was 61% (n = 171), which represents 10% of all obstetric caregivers (clinical midwives (80%) and registered obstetricians (20%) in Dutch hospitals (Table 1). Over 50% of the Dutch hospitals were represented and these hospitals were spread out over the entire country. Eighty three respondents (49%) worked in non-teaching hospitals and 88 (51%) in teaching hospitals. Mean practice size was 1800 (InterQuartile Range (IQR) 1250–2400) deliveries per year, the mean number of obstetric caregivers working in the maternity unit was 22 (IQR 17–26) and the mean working experience of the respondents was 14 years (IQR 7–21) (Table 1).
Table 1Characteristics of included obstetric caregivers (Q1–6).
| Obstetric caregivers | Sample n (%) | Nationwide (2014) n (%) |
|---|---|---|
| 171 | 1878 | |
| Respondents | Total number practising in the Netherlands | |
| Obstetricians | 34 (20) | 977 (52) |
| Midwives | 137 (80) | 901 (48) [27] |
| Hospital setting | ||
| Teaching | 88 (51) | 1159 (62) |
| Non-teaching | 83 (49) | 720 (38) |
| Number of deliveries by | Unknown | |
| Obstetricians | 10% | |
| Midwives | 57% | |
| Others | 33% | |
| Practice size | Mean 1800 (IQR 1250–2400) | |
| Number of obstetric caregivers per maternity unit | Mean 22 (IQR 17–26) | |
| Working experience | Mean 14 (IQR 7–21) | |
a Number of deliveries per year.
b Number of years.
c Data from 2010.
Plan
Almost all respondents were familiar with the quality tools addressed in the questionnaire (Table 2). 82% of the participants was aware of quality indicators with a statistically significant difference between obstetricians (97%) and midwives (79%) (p = 0.004). There was no difference between teaching and non-teaching hospitals. Of the respondents 18% stated not to know the term ‘quality indicator’.
Table 2Use of indicators in quality cycle (Q7, 9–11, 15, 16).
| Yes n (%) | No n (%) | ? n (%) | p-Value | |
|---|---|---|---|---|
| Plan (Q7) | ||||
| Awareness of quality tools by caregivers | ||||
| Safe reporting of incidents and sentinel events | 153 (90) | 18 (10) | – | |
| Perinatal audit | 171 (100) | – | – | |
| Client satisfaction | 163 (95) | 8 (5) | – | |
| Team training | 170 (99) | 1 (1) | – | |
| Make quality year plan | 155 (91) | 16 (9) | – | |
| Awareness of quality indicators by caregivers | 141 (82) | 30 (19) | – | |
| Obstetricians | 33 (97) | 1 (3) | – | 0.004 |
| Midwives | 108 (79) | 29 (21) | – | |
| Teaching | 72 (82) | 16 (18) | – | 0.843 |
| Non-teaching | 69 (83) | 14 (17) | – | |
| Do (Q7, 11) | ||||
| Caregivers say they actually do have | ||||
| Safe reporting of incidents and sentinel events | 150 (88) | 11 (6) | 10 (6) | |
| Perinatal Audit | 169 (99) | 2 (1) | – | |
| Client satisfaction | 147 (86) | 13 (8) | 11 (6) | |
| Team training | 166 (97) | 5 (3) | – | |
| Make quality year plan | 129 (75) | 9 (5) | 33 (19) | |
| Registration of quality indicators | 108 (63) | 10 (6) | 53 (31) | |
| Registration of quality indicators other than listed in the survey | 16 (9) | 39 (23) | 116 (68) | 0.001 |
| Obstetricians | 9 (26) | 6 (18) | 19 (56) | |
| Midwives | 7 (5) | 33 (24) | 97 (71) | 0.452 |
| Teaching | 10 (11) | 22 (25) | 56 (64) | |
| Non-teaching | 6 (7) | 17 (21) | 60 (72) | |
| Active in registration of indicators | ||||
| Obstetricians | 108 (63) | 10 (6) | 53 (31) | |
| Midwives | 30 (88) | 3 (9) | 1 (3) | 0.000 |
| Teaching | 78 (57) | 7 (5) | 52 (38) | |
| Non-teaching | 58 (66) | 4 (5) | 26 (29) | 0.620 |
| 50 (60) | 6 (7) | 27 (33) | ||
| Study (Q9, 15) | ||||
| Written or oral debriefing of results of indicators to caregivers | ||||
| Internal indicators | 110 (64) | 61 (36) | – | |
| External indicators | 82 (48) | 89 (52) | – | |
| Act (Q10, 16) | ||||
| Internal indicators (n = 171) | ||||
| Making plans for improvement | 96 (56) | 16 (9) | 59 (35) | |
| Obstetricians | 25 (74) | 4 (12) | 5 (15) | 0.017 |
| Midwives | 71 (52) | 12 (9) | 54 (40) | |
| Teaching | 53 (60) | 7 (8) | 28 (32) | 0.550 |
| Non-teaching | 43 (52) | 9 (11) | 31 (37) | |
| External indicators (n = 163) | ||||
| Making plans for improvement | 67 (41) | 16 (10) | 80 (49) | |
| Obstetricians | 23 (68) | 5 (15) | 6 (18) | 0.000 |
| Midwives | 44 (34) | 11 (9) | 74 (58) | |
| Teaching | 38 (45) | 3 (4) | 43 (51) | 0.021 |
| Non-teaching | 29 (25) | 13 (11) | 37 (64) | |
a Fisher's exact test.
b Pearson Chi-square.
Do
Most of the tools mentioned in the questionnaire were being used in the wards of the respondents (Table 2). A majority of the respondents replied that quality tools were routinely used their wards. For example, safe reporting of incidents and sentinel events was common at 88%. Only 63% of the respondents said they had a method of registration for quality indicators. Of all obstetric caregivers 9% (5% midwives vs 26% obstetricians, p = 0.001) stated to use other indicators than those in the survey. Of the respondents 116 (68%) did not know whether their hospital/department recorded other quality indicators than those in the questionnaire. 63% (n = 108) reported to register external indicators, and 31% (n = 53) did not know if their hospital registered indicators.
Study
36% of the respondents stated that they were not informed of were involved in the evaluation about the scores of their internal indicators (Table 2). The remainder of the group (64%) received a periodical debriefing by e-mail, newsletter or in meetings. Only 48% got a debriefing on external indicators.
Act
56% of the respondents stated that they would make plans for improvement or adjust existing protocols on the basis of the results of quality indicators (Table 2). Finally we observed that the structure indicators defined by the Dutch Society of Obstetrics and Gynaecology were implemented as quality tools in most obstetric units (Table 3). However the majority of these quality tools are not used as an indicator by the healthcare professionals. We also observed that the term ‘indicator’ was not known by some of the caregivers.
Table 3Quality tools presented as structure indicators (Q8).
| Quality tools | Structure is present according to respondent n (%) | Tool is used as an indicator for internal quality improvement n (%) |
|---|---|---|
| The presence of a protocol for shock treatment | 130 (77) | 46 (27) |
| The presence of a protocol for eclamptic and HELLP patients with third line referral | 103 (61) | 36 (21) |
| The presence of a protocol for epidural analgasia | 162 (96) | 46 (27) |
| The presence of a protocol for fetal monitoring | 131 (78) | 37 (22) |
| The presence of a responsible obstetrician to develop protocols | 153 (91) | 56 (33) |
| Local agreements of availability of operating theatre within 20 min | 152 (90) | 77 (46) |
| Structural discussion of complications | 134 (79) | 84 (50) |
| Structural meetings to discuss the indication for induction of labor | 41 (24) | 23 (14) |
| Structural meetings to discuss the indication for ceasarions | 60 (36) | 27 (16 |
| Skill and drill for obstetric complications (shock and eclamptic insult) | 131 (78) | 66 (40) |
| Certified lactation expert or structural program to support breastfeeding | 169 (100) | 105 (62) |
| Structural PICO meetings | 128 (76) | 39 (23) |
| Trans mural protocols | 166 (98) | 75 (44) |
| Structural Safe reporting of incidents and sentinel events | 127 (75) | 88 (52) |
| Agreement on reporting maternal death | 103 (61) | 50 (30) |
Comment
Main findings
The main purpose of this study was to investigate whether quality indicators are known and used by obstetric caregivers to evaluate and improve the quality of care. We observed that the indicators implemented at present in the Netherlands are not widely known and not routinely used in quality management in maternity wards in the Netherlands. Obstetricians have more awareness and are more actively involved in the quality improvement process than clinical midwives.
Strengths and limitations
To our knowledge this is the first study exploring the use of quality indicators by obstetric caregivers in the Netherlands. We obtained a good response rate with obstetric caregivers responsible for most (67%) of the normal deliveries in Dutch hospitals. There are more midwives in our sample than obstetricians, which reflects the distribution of the numbers of deliveries being done by these professionals [
[17]
].We acknowledge that the term ‘indicators’ is not a familiar term among the respondents. This limits the validity of the results. The answers to the open ended questions (Q9–16) were filled in by too few people giving different interpretations to the question. Therefore, the information collected from these questions could not be used. Care providers do not consider registered data of provided care as quality indicators. This suggests that the definition and use of indicators is not known by the caregivers. A potential weakness of our study is that we performed it on a convenience sample of obstetricians and midwives which may limit the validity of the results.
Interpretation
We found that there is no uniform approach to improve quality of care in the maternity units represented in this study, such as perinatal audits and team training. In our study 19% of the respondents is not aware of the existence of quality indicators. Perinatal audits in the Netherlands are mandatory according to the Dutch Health Inspectorate while other tools are not [
[18]
]. However Boulkedid et al. [[9]
] suggested that all these approaches have limited efficiency. Registration of quality indicators is an important part of improving quality of care and promoting beneficial changes in policy. Registration of quality indicators alone does not improve quality of healthcare, as pointed out by Porter et al. [[8]
], the most important step towards improving outcomes in health care is that providers take part in the feedback loop of measuring, giving feedback and making plans for improvement. For that reason, as a first step in this process awareness of indicators by all caregivers is essential. Although indicators are clearly described by the Dutch society of Obstetricians and Gynaecology, in daily practice quality indicators are hardly used by obstetricians and midwives. The most likely reason for this is that the obstetricians and midwives are not aware of these indicators. Another reason suggested in some studies is structure indicators are known not to be the most useful indicators for caregivers to reflect on. Some studies suggest that outcome related indicators are more effective [4
, 19
]. Only 9% of the respondents gave a small addition to this list. This is remarkable since the Dutch Society of Obstetricians and Gynaecology also recommends a list of process and outcome quality indicators, next to the listed structure indicators we used in this survey [[20]
].As observed in our study there is currently under-utilisation of quality indicators. To effectively make use of quality indicators, caregivers should receive active feedback on their performance by quality indicator data. Only then can clinical practice and patient outcome be improved. This requires measurements (indicators) that are interpretable, valid and useful in daily practice [
2
, 21
, 22
, 23
]. Preferably, quality indicators are made easily available in existing databases or digital patient files and no extra registration should be required to collect the indicators [24
, 25
, 26
]. Only providing quantitative indicator data does not mean that caregivers will use the data in their quality improvement policy and practice. Organized systematic feedback to the responsible caregivers of teams followed-up action on the process of care deliver the contribution to improvement of care [2
, 21
]. Thus, as De Vos et al. [[22]
] demonstrated, feedback is only effective when combined with action oriented quality tools such as guideline development or educational programs. This is in line with the findings in our study, where we found that these quality tools were applied to routine policy in the maternity wards.In our study only 56% of respondents appeared to use internal or external indicators when designing improvements of care. Similar results were found by Botje et al. [
[24]
] and Mainz et al. [[21]
] Their view is that pressure caused by external accountability does not automatically result in increased use of indicators for quality improvement activities by caregivers. Moreover, the engagement of medical specialists seems to play an important role in the use of performance indicators. Medical specialists appeared to select the indicators they considered to be useful for internal quality management, and based that selection on perceived validity and reliability [21
, 24
]. This is in line with the findings of our study: medical specialists were much more involved in quality improvement than the clinical midwives. Therefore we suggest a leading role for obstetricians in the implementation of indicators, as an effective means for quality improvement.Conclusion
We conclude that obstetric quality indicators are not widely used by obstetricians and midwives in Dutch hospitals to improve quality of care. We recommend to investigate barriers and facilitating factors to enhance the routine use of quality indicators to improve obstetric care. To improve quality of care and the effective use of quality indicators we suggest to focus first on registering outcome indicators, under leadership of medical specialists. We think more awareness of quality indicators and their potential benefits for the outcome of healthcare is necessary for successful implementation. Finally, these indicators should be implemented in quality structures that ensure that action is taken, such as audits and team-training.
Acknowledgement
We would like to thank all respondents who filled in our survey.
Appendix A. Supplementary data
The following are the supplementary data to this article:
References
- Kohn L.T. Corrigan J.M. Donaldson To err is human: building a safer health system. National Academy of Sciences (US), 2000
- Crossing the quality chasm: a new health system for the 21st century.National Academy Press, Washington, DC2001
- Research methods used in developing and applying quality indicators in primary care.Qual Saf Health Care. 2002; 11: 358-364
- Indicatorensets Ketenbrede Kwaliteitsindicatoren Acute Zorg. Soest.2013 July: 49-69
- Indicatorenset zwangerschap en bevalling. Den Haag.2013 October
- Quality indicators for general practice: which ones can general practitioners and health authority managers agree are important and how useful are they?.J Public Health Med. 1998; 20: 414-421
- Explorations in quality assessment and monitoring: the definition of quality and approaches to its assessment.Health Administration Press, Michigan1980
- What is value in health care?.N Engl J Med. 2010; 26: 2477-2481
- Quality indicator development and implementation in maternity units.Best Pract Res Clin Obstetr Gynaecol. 2013; 27: 609-619
- Clinical indicators: development and applications.Ned Tijdschr Geneeskd. 2007; 65: 15-22
- Defining and classifying clinical indicators for quality improvement.Int J Qual Health Care. 2003; 15: 5-23
- Using performance indicators to improve health care quality in the public sector: a review of the literature.Health Serv Manage Res. 2002; 15: 126-137
- European perinatal health report. The health and care of pregnant women and babies in Europe in 2010.Euro-Peristat Project with SCPE and EUROCAT. Euro Peristat, Paris2013
- Garanties voor de kwaliteit van zorg. Den Haag.2013
- The new economics for industry, government, education.2nd ed. MIT Press, Boston, MA1994
- Systematic review of the application of the plan-do-study-act method to improve quality in healthcare.BMJ Qual Saf. 2013; 0: 1-9
- Wat ons opvalt in de LVR2 Wie doet de bevalling?.Ned Tijdschr Obst Gyn. 2009; 122: 270
- Basisset kwaliteitsindicatoren ziekenhuizen, 2014. Utrecht.2015
- More quality measures versus measuring what matters: a call for balance in parsimony.BMJ Qual Saf. 2012; 21: 964-968
- Kwaliteitsindicatoren voor de obstetrie.Ontwikkeling en gebruik in Nederland. RIVM report 260101001/2008, Bilthoven2008
- National quality measurement using clinical indicators: the Danish National Indicator Project.Int J Surg Oncol. 2009; 99: 500-504
- Using quality indicators to improve hospital care: a review of the literature.Int J Qual Health Care. 2009; 21: 119-129
- Quality indicators: essential for quality improvement.Int J Qual Health Care. 2004; 16: i1-i2
- Are performance indicators used for hospital quality management: a qualitative interview study amongst health professionals and quality managers in the Netherlands.BMC Health Serv Res. 2016; 16: 574-583
- The future of quality measurement for improvement and accountability.JAMA. 2013; 309: 2215
- An evaluation of the reliability, validity and applicability of performance indicators for the quality of hospital care in the Netherlands.AMC/UvA, Amsterdam2012
- Klinisch verloskundigen terug in de schoolbanken.Tijdschrift voor Verloskundigen. 2015; : 40-42
Article info
Publication history
Published online: February 28, 2017
Accepted:
February 14,
2017
Received in revised form:
February 9,
2017
Received:
November 17,
2016
Identification
Copyright
© 2017 Elsevier B.V. All rights reserved.
