TY - JOUR
T1 - Corrigendum to “Paying people to protect the environment
T2 - A meta-analysis of financial incentive interventions to promote proenvironmental behaviors” [Journal of Environmental Psychology 47 (2016) 242–255] (Journal of Environmental Psychology (2016) 47 (242–255), (S0272494416300676), (10.1016/j.jenvp.2016.07.006))
AU - Maki, Alexander
AU - Burns, Rachel J.
AU - Ha, Long
AU - Rothman, Alexander J.
N1 - Funding Information:
This work was supported in part by the National Science Foundation through a Graduate Research Fellowship to Alexander Maki.
Publisher Copyright:
© 2016 Elsevier Ltd
PY - 2021/8
Y1 - 2021/8
N2 - We, the authors, identified two errors pertaining to the post-incentive effect size reported in the original version of the above-mentioned article. A reader raised an initial concern about how the post-incentive effect size for Mayer and Geller (1982–83) had been coded. Three of the authors subsequently reviewed the article, and decided that the best way to handle the coding of this article would be to remove from our meta-analysis the single effect size derived from Mayer and Geller (1982–83) as the statistical information provided in the paper conflates the effects on behavior for when the incentive was in place and after it was removed. Thus, this paper is now coded as being ineligible for inclusion in the meta-analysis (see updated Fig. 1 below). While updating the database, one other error was detected again involving a post-incentive effect size. The effect size for Foxx and Schaeffer's (1981) study was reported as positive, but should have been negative. We discovered no other errors when reviewing the database. The analyses were re-run on the updated database and the results indicate that the substantive conclusions of our meta-analysis do not change. In fact, the post-incentive effect size remains the same to two decimal places (albeit with a slightly wider confidence interval) after both errors were corrected (effect size of d+ = 0.41 after incentives were removed). The only notable changes in the findings are that after removal of an incentive, the effect of incentives on efficient driving behavior is now slightly negative (d+ = -0.11), as is the effect of cash incentives after they are removed on travel behavior. These findings are identical because they rely on the same set of studies. Finally, we are now unable to conduct any comparison of variable and fixed schedule incentives after the incentive had been removed given there is now only one effect size reflecting use of a variable schedule after the incentive was removed. All of the statistics with associated text, figures, and tables related to the updated post-incentive effect size are provided below. The updates are presented sequentially regarding the relevant findings in the paper: - Fig. 1 (Flow of information through the phases of the present review.)- Section 3.1. (Do financial incentives affect both initial and sustained proenvironmental behavior?)- Table 1 (Characteristics and effect sizes of studies included in the meta-analysis.)- Fig. 3 (Forest plot of effect sizes for the effect of incentive interventions on proenvironmental behaviors after the intervention had been removed.)- Section 3.2 (Do types of financial incentives differentially affect proenvironmental behavior?)- Table 2 (Effect sizes by type of incentive.)- Section 3.3 (Do financial incentives differentially affect types of behaviors?)- Table 3 (Effect sizes by target proenvironmental behavior.)- Table 4 (Effect sizes by target behavior category.)- Table 5 (Effect of cash versus non-cash incentives on recycling, energy conservation, and travel behaviors when the incentive was in place and after it had been removed.)- Table S1 (Characteristics of studies included in the meta-analysis.)- Updated references in these passages of text [Figure presented] 3.1 Do financial incentives affect both initial and sustained proenvironmental behavior? During the follow-up period (i.e., after the incentive had been removed), the pattern of results was similar (overall effect size d+ = 0.41, CI = 0.23 to 0.59, k = 12, N = 553, Q = 6.15, p = .86). For these two analyses, the fail-safe N indicated that 523 or 69 unpublished studies with nonsignificant findings, respectively, would be needed to overturn the results (Orwin, 1983; Rosenthal, 1979). When comparing only those studies that provided effect sizes at both time points (k = 8, N = 234), the overall effect sizes were similar when the incentive was in place (d+ = 0.47, CI = 0.20 to 0.73, Q = 3.63, p = .82) and after the incentive had been removed (d+ = 0.29, CI = 0.03 to 0.55, Q = 3.86, p = .80). These two effect sizes did not significantly differ from one another (Q = 0.93, p = .34). Although the length of follow-up ranged from two weeks to three months later (M = 7.00 weeks, SD = 4.11), it was not a significant predictor of behavior after the incentive had been removed (b = 0.03, CI = -.03 to 0.09, p = .25). A series of analyses were conducted to ensure that these findings were robust. First, Coyne, Thombs, and Hagedorn's (2010) criterion was employed to investigate whether the results differed if the analyses were limited to studies with adequate power (i.e., 55% power to detect a medium-sized effect). Thirty-three percent of the studies reporting an effect size regarding behavior during the intervention and eight percent of the studies reporting an effect size regarding behavior during follow-up had adequate power. Findings were largely unchanged when the meta-analysis was restricted to these studies. The effect when the incentive was in place (d+ = 0.39, CI = 0.18 to 0.60, k = 8, N = 2423) was comparable to the effect after the incentive had been removed (d+ = 0.47, CI = 0.16 to 0.78, k = 1, N = 169), both of which are quite comparable to the effect sizes obtained when using all of the studies. Follow-up analyses were not conducted because only one study that reported an effect size after the incentive was removed had adequate power. Second, given the low power in many of the studies, Egger's tests (Egger, Smith, Schneider, & Minder, 1997) were computed to examine whether there was a significant relationship between study effect sizes and the size of the sample from which the effect size was derived. Egger's tests were not significant while the incentive was in place (t(25) = 1.04, p = .31) or after it had been removed (t(12) = −0.82, p = .43), suggesting that effect sizes derived from small samples were not biasing the estimate of the overall effect sizes. [Figure presented] [Figure presented] After the incentive had been removed, use of cash (k = 3) and tickets (k = 5) had a medium effect on behavior, coupons (k = 2) had a small-to-medium effect on behavior, using more than one incentive (k = 1) had a small effect on behavior, and using lotteries (k = 1) had a medium-to-large negative effect on behavior. Do different reinforcement features affect proenvironmental behavior? Because only one effect size reflected use of a variable schedule after the incentive was removed, we were unable to conduct any comparisons of variable and fixed schedule incentives after the incentive had been removed. [Table presented] There was less variability in the effect of incentives on specific behaviors after the incentive had been removed, as they largely had a small-to-medium effect on most types of specific behaviors, except for the small negative effect of incentives on efficient driving behavior. After the incentive had been removed, the effect size for energy conservation behavior (k = 1) during follow-up was of medium magnitude; the effect sizes for both recycling behavior (k = 2) and travel behavior (k = 9) were of small-to-medium magnitude. Do types of financial incentives differentially affect types of behaviors? After the incentive had been removed, cash incentive interventions were used in relation to travel behavior and energy conservation, with a medium positive effect on energy conservation and a small negative effect on travel behavior. Non-cash incentive interventions were administered in studies examining travel behavior and recycling, demonstrating comparable positive effects across those categories of behavior. Only travel behavior was targeted in both cash and non-cash incentive interventions that included a follow-up time point; there was not a significant difference between non-cash incentives and cash incentives (Q = 2.16, p = .14). Do types of financial incentives differentially affect types of behaviors? After the incentive had been removed, cash incentive interventions were used in relation to travel behavior and energy conservation, with a medium positive effect on energy conservation and a small negative effect on travel behavior. Non-cash incentive interventions were administered in studies examining travel behavior and recycling, demonstrating comparable positive effects across those categories of behavior. Only travel behavior was targeted in both cash and non-cash incentive interventions that included a follow-up time point; there was not a significant difference between non-cash incentives and cash incentives (Q = 2.16, p = .14). Does study design influence the effect of financial incentives on behavior? After the incentive had been removed, within-subject comparisons revealed the strongest effects (d+ = 0.64, CI = 0.05 to 1.24, k = 3), followed by comparisons of incentives to active comparisons (d+ = 0.48, CI = −0.30 to 0.99, k = 2), and finally comparing incentives to controls (d+ = 0.37, CI = 0.17 to 0.57, k = 7). The difference between the effects observed from a within-subject and a control comparison design was not significant (Q = 0.72, p = .40); nor was the difference between the effects observed from a within-subject and an active comparison design (Q = 0.64, p = .43). Finally, effect sizes observed in the control comparison and the active comparison designs were not different (Q = 0.16, p = .69). Does combining a financial incentive intervention with other intervention components influence the effect of incentives on behavior? The direction and magnitude of this difference held after the incentive had been removed, but again was not significant (combined interventions d+ = 0.48, CI = 0.27 to 0.69, k = 8; pure incentive interventions d+ = 0.24, CI = −0.13 to 0.60, k = 4; Q = 1.52, p = .22). Does intervention length influence the effect of financial incentives on behavior? The length of the financial incentive interventions ranged from minutes up to three months, on average lasting just over nine weeks (M = 9.27 weeks, SD = 9.80). The length of the intervention was not a significant predictor of behavior while the incentive was in place (b = -.005, CI = -.02 to 0.01, p = .48) or after it had been removed (b = -.04, CI = -.14 to 0.06, p = .39). Does objective versus self-reported behavior influence the effect of incentives on behavior? After the incentive had been removed, three studies relied on self-reported behavior and nine relied on objective behavior; studies that used self-report measures (d+ = 0.51, CI = 0.25 to 0.76, k = 3) reported larger effects on behavior compared to those that used objective measures of behavior (d+ = 0.32, CI = 0.07 to 0.56, k = 9), but this difference was not statistically significant (Q = 1.11, p = .29). Does explicitly grounding an intervention in a theoretical model influence the effect of financial incentives on behavior? After the incentive had been discontinued, theory-based incentive interventions (d+ = 0.52, CI = 0.24 to 0.80, k = 2) were not significantly more effective than interventions that did not mention theory (d+ = 0.34, CI = 0.11 to 0.57, k = 10; Q = 0.95, p = .33). [Table presented] [Table presented] [Table presented] [Table presented] The authors would like to apologize to the readers for any inconvenience caused.
AB - We, the authors, identified two errors pertaining to the post-incentive effect size reported in the original version of the above-mentioned article. A reader raised an initial concern about how the post-incentive effect size for Mayer and Geller (1982–83) had been coded. Three of the authors subsequently reviewed the article, and decided that the best way to handle the coding of this article would be to remove from our meta-analysis the single effect size derived from Mayer and Geller (1982–83) as the statistical information provided in the paper conflates the effects on behavior for when the incentive was in place and after it was removed. Thus, this paper is now coded as being ineligible for inclusion in the meta-analysis (see updated Fig. 1 below). While updating the database, one other error was detected again involving a post-incentive effect size. The effect size for Foxx and Schaeffer's (1981) study was reported as positive, but should have been negative. We discovered no other errors when reviewing the database. The analyses were re-run on the updated database and the results indicate that the substantive conclusions of our meta-analysis do not change. In fact, the post-incentive effect size remains the same to two decimal places (albeit with a slightly wider confidence interval) after both errors were corrected (effect size of d+ = 0.41 after incentives were removed). The only notable changes in the findings are that after removal of an incentive, the effect of incentives on efficient driving behavior is now slightly negative (d+ = -0.11), as is the effect of cash incentives after they are removed on travel behavior. These findings are identical because they rely on the same set of studies. Finally, we are now unable to conduct any comparison of variable and fixed schedule incentives after the incentive had been removed given there is now only one effect size reflecting use of a variable schedule after the incentive was removed. All of the statistics with associated text, figures, and tables related to the updated post-incentive effect size are provided below. The updates are presented sequentially regarding the relevant findings in the paper: - Fig. 1 (Flow of information through the phases of the present review.)- Section 3.1. (Do financial incentives affect both initial and sustained proenvironmental behavior?)- Table 1 (Characteristics and effect sizes of studies included in the meta-analysis.)- Fig. 3 (Forest plot of effect sizes for the effect of incentive interventions on proenvironmental behaviors after the intervention had been removed.)- Section 3.2 (Do types of financial incentives differentially affect proenvironmental behavior?)- Table 2 (Effect sizes by type of incentive.)- Section 3.3 (Do financial incentives differentially affect types of behaviors?)- Table 3 (Effect sizes by target proenvironmental behavior.)- Table 4 (Effect sizes by target behavior category.)- Table 5 (Effect of cash versus non-cash incentives on recycling, energy conservation, and travel behaviors when the incentive was in place and after it had been removed.)- Table S1 (Characteristics of studies included in the meta-analysis.)- Updated references in these passages of text [Figure presented] 3.1 Do financial incentives affect both initial and sustained proenvironmental behavior? During the follow-up period (i.e., after the incentive had been removed), the pattern of results was similar (overall effect size d+ = 0.41, CI = 0.23 to 0.59, k = 12, N = 553, Q = 6.15, p = .86). For these two analyses, the fail-safe N indicated that 523 or 69 unpublished studies with nonsignificant findings, respectively, would be needed to overturn the results (Orwin, 1983; Rosenthal, 1979). When comparing only those studies that provided effect sizes at both time points (k = 8, N = 234), the overall effect sizes were similar when the incentive was in place (d+ = 0.47, CI = 0.20 to 0.73, Q = 3.63, p = .82) and after the incentive had been removed (d+ = 0.29, CI = 0.03 to 0.55, Q = 3.86, p = .80). These two effect sizes did not significantly differ from one another (Q = 0.93, p = .34). Although the length of follow-up ranged from two weeks to three months later (M = 7.00 weeks, SD = 4.11), it was not a significant predictor of behavior after the incentive had been removed (b = 0.03, CI = -.03 to 0.09, p = .25). A series of analyses were conducted to ensure that these findings were robust. First, Coyne, Thombs, and Hagedorn's (2010) criterion was employed to investigate whether the results differed if the analyses were limited to studies with adequate power (i.e., 55% power to detect a medium-sized effect). Thirty-three percent of the studies reporting an effect size regarding behavior during the intervention and eight percent of the studies reporting an effect size regarding behavior during follow-up had adequate power. Findings were largely unchanged when the meta-analysis was restricted to these studies. The effect when the incentive was in place (d+ = 0.39, CI = 0.18 to 0.60, k = 8, N = 2423) was comparable to the effect after the incentive had been removed (d+ = 0.47, CI = 0.16 to 0.78, k = 1, N = 169), both of which are quite comparable to the effect sizes obtained when using all of the studies. Follow-up analyses were not conducted because only one study that reported an effect size after the incentive was removed had adequate power. Second, given the low power in many of the studies, Egger's tests (Egger, Smith, Schneider, & Minder, 1997) were computed to examine whether there was a significant relationship between study effect sizes and the size of the sample from which the effect size was derived. Egger's tests were not significant while the incentive was in place (t(25) = 1.04, p = .31) or after it had been removed (t(12) = −0.82, p = .43), suggesting that effect sizes derived from small samples were not biasing the estimate of the overall effect sizes. [Figure presented] [Figure presented] After the incentive had been removed, use of cash (k = 3) and tickets (k = 5) had a medium effect on behavior, coupons (k = 2) had a small-to-medium effect on behavior, using more than one incentive (k = 1) had a small effect on behavior, and using lotteries (k = 1) had a medium-to-large negative effect on behavior. Do different reinforcement features affect proenvironmental behavior? Because only one effect size reflected use of a variable schedule after the incentive was removed, we were unable to conduct any comparisons of variable and fixed schedule incentives after the incentive had been removed. [Table presented] There was less variability in the effect of incentives on specific behaviors after the incentive had been removed, as they largely had a small-to-medium effect on most types of specific behaviors, except for the small negative effect of incentives on efficient driving behavior. After the incentive had been removed, the effect size for energy conservation behavior (k = 1) during follow-up was of medium magnitude; the effect sizes for both recycling behavior (k = 2) and travel behavior (k = 9) were of small-to-medium magnitude. Do types of financial incentives differentially affect types of behaviors? After the incentive had been removed, cash incentive interventions were used in relation to travel behavior and energy conservation, with a medium positive effect on energy conservation and a small negative effect on travel behavior. Non-cash incentive interventions were administered in studies examining travel behavior and recycling, demonstrating comparable positive effects across those categories of behavior. Only travel behavior was targeted in both cash and non-cash incentive interventions that included a follow-up time point; there was not a significant difference between non-cash incentives and cash incentives (Q = 2.16, p = .14). Do types of financial incentives differentially affect types of behaviors? After the incentive had been removed, cash incentive interventions were used in relation to travel behavior and energy conservation, with a medium positive effect on energy conservation and a small negative effect on travel behavior. Non-cash incentive interventions were administered in studies examining travel behavior and recycling, demonstrating comparable positive effects across those categories of behavior. Only travel behavior was targeted in both cash and non-cash incentive interventions that included a follow-up time point; there was not a significant difference between non-cash incentives and cash incentives (Q = 2.16, p = .14). Does study design influence the effect of financial incentives on behavior? After the incentive had been removed, within-subject comparisons revealed the strongest effects (d+ = 0.64, CI = 0.05 to 1.24, k = 3), followed by comparisons of incentives to active comparisons (d+ = 0.48, CI = −0.30 to 0.99, k = 2), and finally comparing incentives to controls (d+ = 0.37, CI = 0.17 to 0.57, k = 7). The difference between the effects observed from a within-subject and a control comparison design was not significant (Q = 0.72, p = .40); nor was the difference between the effects observed from a within-subject and an active comparison design (Q = 0.64, p = .43). Finally, effect sizes observed in the control comparison and the active comparison designs were not different (Q = 0.16, p = .69). Does combining a financial incentive intervention with other intervention components influence the effect of incentives on behavior? The direction and magnitude of this difference held after the incentive had been removed, but again was not significant (combined interventions d+ = 0.48, CI = 0.27 to 0.69, k = 8; pure incentive interventions d+ = 0.24, CI = −0.13 to 0.60, k = 4; Q = 1.52, p = .22). Does intervention length influence the effect of financial incentives on behavior? The length of the financial incentive interventions ranged from minutes up to three months, on average lasting just over nine weeks (M = 9.27 weeks, SD = 9.80). The length of the intervention was not a significant predictor of behavior while the incentive was in place (b = -.005, CI = -.02 to 0.01, p = .48) or after it had been removed (b = -.04, CI = -.14 to 0.06, p = .39). Does objective versus self-reported behavior influence the effect of incentives on behavior? After the incentive had been removed, three studies relied on self-reported behavior and nine relied on objective behavior; studies that used self-report measures (d+ = 0.51, CI = 0.25 to 0.76, k = 3) reported larger effects on behavior compared to those that used objective measures of behavior (d+ = 0.32, CI = 0.07 to 0.56, k = 9), but this difference was not statistically significant (Q = 1.11, p = .29). Does explicitly grounding an intervention in a theoretical model influence the effect of financial incentives on behavior? After the incentive had been discontinued, theory-based incentive interventions (d+ = 0.52, CI = 0.24 to 0.80, k = 2) were not significantly more effective than interventions that did not mention theory (d+ = 0.34, CI = 0.11 to 0.57, k = 10; Q = 0.95, p = .33). [Table presented] [Table presented] [Table presented] [Table presented] The authors would like to apologize to the readers for any inconvenience caused.
UR - http://www.scopus.com/inward/record.url?scp=85108783283&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85108783283&partnerID=8YFLogxK
U2 - 10.1016/j.jenvp.2021.101637
DO - 10.1016/j.jenvp.2021.101637
M3 - Comment/debate
AN - SCOPUS:85108783283
SN - 0272-4944
VL - 76
JO - Journal of Environmental Psychology
JF - Journal of Environmental Psychology
M1 - 101637
ER -