Clinical estimation of fetal weight and the Hawthorne effect

1. Introduction 

When performing and analyzing prospective clinical trials, investigators recognize the possible contribution of a placebo effect to the results of the trial [1]. The placebo effect refers to an improvement in outcomes solely from receiving treatment, even if it is an inert drug or sham surgery. A somewhat related phenomenon is the Hawthorne effect. The Hawthorne effect refers to improvement in performance solely due to the subject's knowledge that he or she is being studied [2]. The Hawthorne effect received its name from a factory called the Hawthorne Works, where a series of experiments were performed between 1924 and 1932 initially studying the effect of lighting on workers’ productivity. Whether the Hawthorne effect is real or a myth has been debated in the literature [3], [4]. However, there have been prospective studies demonstrating the Hawthorne effect on patients [5]. Additionally, the Hawthorne effect has been used to explain physician behavior in regards to patient assessment [6], prescribing patterns [7], emergency room care [8], and certain birth outcomes [9], [10].

It is possible that a physician's diagnostic accuracy can also be subject to the Hawthorne effect. Namely, if a physician were participating in a study, it is possible that he or she could knowingly or unknowingly become more accurate in diagnosing a condition based on the physical examination.

In term pregnant patients, an estimated fetal weight (EFW) is an important part of the obstetrician's assessment of the patient [11], [12]. While this can be done by ultrasound, numerous studies have shown that an obstetrician's clinical EFW, which is obtained by physical examination of the maternal abdomen, has similar accuracy compared to ultrasound [13], [14], [15], [16]. However, each of these studies was done prospectively. Therefore, it is possible that the obstetricians’ EFW accuracy was improved by the knowledge that they were being studied and that an obstetrician's EFW accuracy would be reduced in a non-investigational setting. The goal of this study was to determine if there is a Hawthorne effect in regards to a physician's clinical estimation of fetal weight in term pregnancies.

2. Methods 

Obstetricians on our Labor and Delivery unit were asked to participate in a study evaluating the accuracy of their clinical EFW performed at the beginning of labor compared to their clinical EFW performed at the end of labor. The results of this particular study are published separately [17]. The obstetricians were asked to perform a clinical EFW on their patients when the patient was admitted to the labor floor at or beyond 37 0/7 weeks gestation with spontaneous labor or ruptured membranes, or for induction of labor or a scheduled cesarean delivery. Patients with a recent sonographic EFW and multiple pregnancies were excluded. Obstetricians were asked to fill out a study data sheet and were aware that their EFW accuracy was being studied, but the obstetricians were not aware that we were studying the possibility of a Hawthorne effect as well. Management of labor was at the discretion of the covering obstetrician.

To determine if there is a Hawthorne effect in regards to a clinical EFW, we compared the accuracy of the EFW's obtained during our study when the obstetricians knew they were being studied (cases) to EFW's obtained before our study began in a typical clinical setting (controls). For the cases, we selected consecutive clinical EFW's obtained before or at the beginning of labor during the study period. For controls, we reviewed obstetricians’ clinical EFW's obtained before or at the beginning of labor for all patients on our Labor and Delivery unit during January 2007, which is 3 months prior to the onset of our EFW study. Similar to the cases, we included any patient who had presented at or beyond 37 0/7 weeks gestation for labor, ruptured membranes, induction of labor, or scheduled cesarean delivery. Patients with a recent sonographic EFW and multiple pregnancies were excluded. We then selected consecutive clinical EFW's obtained before or at the beginning of labor beginning with January 1.

Descriptive statistics were calculated for obstetrician characteristics and maternal/fetal baseline characteristics. EFW accuracy was defined in three ways: (1) As the percentage of EFW within ±10% of the actual birth weight; (2) as the mean/median absolute error (absolute value of EFW minus birth weight); and (3) as the mean/median percent error (absolute error divided by the birth weight).

To evaluate the presence of the Hawthorne Effect in the estimation of clinical EFW accuracy, the two-independent sample t-test or non-parametric Wilcoxon rank-sum test was used to compare the mean/median absolute error and mean/median percent error between pre-study (control) obstetricians and during-study (case) obstetricians, as appropriate. Furthermore, the chi-square test was used to evaluate the presence of the Hawthorne Effect on the percentage of EFW within ±10% of the actual fetal birth weight.

Based on prior data we assumed that 71% of EFW's done in a study-setting would be within 10% of the actual birth weight [13]. In order to detect a relative difference of 20% (from 71 to 57%), 187 cases and 187 controls would be needed, with 80% power and a two-sided alpha level of 5%. The study received IRB exemption as the obstetrician routinely obtain a clinical EFW and patients were not subjected to any additional risk.

All p-values are two-sided with statistical significance evaluated at the 0.05 alpha level, and 95% confidence intervals were calculated to assess the precision of the obtained estimates. All analyses were performed in STATA version 10 (College Station, TX).

3. Results 

Obstetricians contributed EFW measurements of 187 cases and 187 controls. A greater proportion of the cases were attending obstetricians as compared to the control obstetricians (89% vs. 53%, respectively, p<.0001). Otherwise, there was no difference with respect to maternal height, weight, body mass index (BMI), cervical dilation at the time of the EFW, membrane status (intact or ruptured), fetal gender, or actual newborn birth weight (Table 1).

Table 1. Baseline characteristics

		Before EFW study (controls) N=187	During EFW study (cases) N=187	p
	Mean maternal height (cm)	163.7±6.4	162.9±6.8	.27
	Mean maternal weight (kg)	75.7±13.1	76.5±14.2	.58
	Mean maternal BMI	28.3±4.8	28.8±5.2	.30
	Attending	53%	89%	<.0001
	Resident	47%	11%
	Membranes ruptured	37%	32%	.35
	Membranes intact	63%	68%
	Male fetus	51%	46%	.35
	Female fetus	49%	54%
	Mean cervical dilation (cm)	2.6±2cm	3±1.8cm	.08
	Mean birthweight (g)	3374±432g	3373±479g	.98

Means compared with two-independent sample t-test; proportions compared with chi-square test; EFW, estimated fetal weight; BMI, body mass index (kg/m2).

Comparing the EFW's before the study period (controls) to EWF's performed during the study period (cases), there were no statistically significant difference in the actual EFW or EFW accuracy. 67.9 and 68.5% of control and case obstetricians, respectively (p=.91) were within ±10% of the actual fetal birth weight (Table 2). There were no significant differences seen between the groups in regards to mean absolute error (absolute value of EFW minus birth weight), or mean percent error (absolute error divided by the birth weight). The raw data for each of these variables were normally distributed around zero, however when we converted the error to absolute error, the data were no longer normally distributed, as expected. Therefore, we also compared the median values of each using a Wilcoxon rank-sum test. No significant differences were seen between the groups (Table 2).

Table 2. EFW accuracy in cases vs. controls

		Before EFW study (controls) N=187	During EFW study (cases) N=187	p
	Mean EFW (g)	3406±308g	3349±375g	.10
	Mean absolute EFW-birthweight (g)a	282±227g	285±232g	.88
	Median (25%, 75%) absolute EFW-birthweight (g)	215g (110, 402)	245g (112, 385)	.87
	Mean absolute percent errorb	8.5±7.4%	8.6±7.2%	.96
	Median (25%, 75%) absolute percent error	6.3 (3.3, 11.5)	7.2 (3.3, 11.4)	.90
	Within 10% of birthweight	67.9%	68.5%	.91

Means compared with two-independent sample t-test; medians compared with Wilcoxon rank-sum test; proportions compared with chi-square test; EFW, estimated fetal weight.

a Absolute value of EFW-birthweight. b (Absolute value of EFW-birthweight)×100 divided by birthweight.

Since the cases had a higher proportion of attending obstetricians, we analyzed the data excluding all EFW's performed by resident obstetricians. Studying attending obstetricians only, we still could find no Hawthorne effect (Table 3).

Table 3. EFW accuracy in cases vs. controls, attending physicians only

		Before EFW study (controls) N=99	During EFW study (cases) N=166	p
	Mean absolute EFW-birthweight (g)a	294±221g	295±231g	.98
	Median (25%, 75%) absolute EFW-birthweight (g)	223g (122, 435)	260g (125, 393g)	.96
	Mean absolute percent errorb	8.8±6.9%	8.9±7.3%	.83
	Median (25%, 75%) absolute percent error	6.4 (3.6, 13.0)	7.7 (3.8, 11.6)	.91
	Within 10% of birthweight	63%	66%	.55

Means compared with two-independent sample t-test; medians compared with Wilcoxon rank-sum test; proportions compared with chi-square test; EFW, estimated fetal weight.

a Absolute value of EFW-birthweight. b (Absolute value of EFW-birthweight)×100 divided by birthweight.

There were 34 babies with birth weights ≥4000g (15 cases, 19 controls). Amongst this subset, we also could not find any Hawthorne effect, but we were underpowered for this analysis (Table 4).

Table 4. EFW accuracy in cases vs. controls for babies ≥4000g

		Before EFW study (controls) N=19	During EFW study (cases) N=15	P
	Mean EFW (g)	3814±331g	3917±356g	.39
	Mean absolute EFW-birthweight (g)a	409±249g	500±349g	.69
	Median (25%, 75%) absolute EFW-birthweight (g)	315g (220, 625)	496g (211, 600)	.83
	Mean absolute percent errorb	9.8±5.9%	11.3±7.6%	.53
	Median (25%, 75%) absolute percent error	7.8 (5.4, 15.0)	10.9 (4.9, 14.3)	.75
	Within 10% of birthweight	58%	40%	.30
	EFW ≥4000g	37%	53%	.49

Means compared with two-independent sample t-test; medians compared with Wilcoxon rank-sum test; proportions compared with chi-square test; EFW, estimated fetal weight.

a Absolute value of EFW-birthweight. b (Absolute value of EFW-birthweight)×100 divided by birthweight.

4. Discussion 

In this study, an obstetrician's clinical estimation of fetal weight before or at the beginning of labor was not improved in a study-setting compared to a typical setting. Thus, we could find no Hawthorne effect. It would seem reasonable to conclude that the published accuracies of clinical EFW's from other prospective studies are likely to be similar to EFW's obtained in clinical practice.

This study was not designed to determine if the Hawthorne effect alters the conclusion that a clinical EFW is as accurate as a sonographic EFW. In order to do this, one would need to do a similar study comparing sonographic EFW's obtained before and during a study period, because it is also plausible that the sonographer obtaining the EFW could be influenced by the knowledge that he or she is being studied. However, this study does give reassurance that the reported accuracy of a clinical EFW is reproducible in clinical practice.

Compared to controls, there was a higher proportion of attending obstetricians amongst the cases. This was to be expected. The controls were obtained from reviewing EFW's performed on all patients prior to the study period, and our resident obstetricians examine many of the patients in our unit and record an EFW. However, the study from which the cases were obtained required that the same obstetrician care for the patient over the entire course of labor. This is much more likely to be possible for attending obstetricians than for resident obstetricians so fewer residents were able to participate in the study. This probably explains the different proportions. Despite this difference, one would expect the more experienced obstetricians to perform better in regards to a clinical EFW, increasing the accuracy in the cases compared to controls. Since we still found no increased accuracy in cases despite a higher level of training, our finding that there is no Hawthorne effect is only strengthened. We also analyzed our data excluding resident physicians and found similar results. It is possible that each individual attending obstetrician may have contributed a different proportion of the case EFW's compared to control EFW's, which could introduce a bias into the study.

Our study does not negate the possibility the Hawthorne effect applies to other prospective trials. However, it appears that it does not play a role in an obstetrician's clinical estimation of fetal weight.