Risk Analysis, Vol. 34, No. 8, 2014
DOI: 10.1111/risa.12249
Commentary
Longitudinal Studies on Risk Research Michael Siegrist
In their informative and useful tutorial on surveys involving the risk domain, Greenberg and Weiner(1) devote only a short paragraph to repeated and longitudinal surveys. Given my experiences with reviewers and the few published longitudinal studies about risk perception and risk communication,(2) it may be helpful to include a brief supplement that focuses on some questions related to longitudinal research designs. To avoid any confusion, a longitudinal research design can be defined by three characteristics:(3) (1) data for the same variables are collected during two or more distinct time periods, (2) the subjects in the different data collection waves are the same or at least comparable, and (3) the statistical analysis involves some sort of comparisons between or among the periods. The two most often used types are the repeated cross-sectional and longitudinal panel designs. Other longitudinal designs exist,(3) but space limitations prevent me from describing them in this commentary. In the repeated cross-sectional design, independent probability samples from the population are drawn at each measurement point. For example, the Eurobarometer uses such a design. For each country in the European Union, random samples are used, and for each wave, new samples are drawn. Some Eurobarometer surveys are related to risk perception and acceptance of new technologies.(4) This design allows the measurement of trends across time. However, it is not possible to calculate individual change scores because different subjects participate in each data wave. Therefore, this design is less useful if researchers are interested in intracohort changes or in testing causality. It is not possible to examine why
people have increased or decreased risk perceptions, for example. The results of cross-sectional designs may also be misleading when examining the influence of risk perception on behavior.(2) In the longitudinal panel design, the same subjects participate in each wave of data collection. If researchers are interested in identifying or estimating the strength of causal relationships, this design should be preferred. However, panel attrition or mortality is a challenge of this design. Some participants may die between the measurement points. Other subjects may be unwilling to cooperate any more, or they may have changed their addresses and can no longer be located. These missing data can be a serious problem if they are not random but biased. In one of my research team’s studies, we measured people’s acceptance of nuclear power before and after the nuclear power accident in Fukushima, Japan.(5) We conducted a telephone survey, with a three-year interval between the two waves. Even though it was not planned as a longitudinal panel design, 46% of the first-wave respondents also participated in the second wave. However, a small bias may have influenced our study’s results. The participants who were against nuclear power prior to the Fukushima disaster were more likely to participate in the second wave, compared with those in favor of nuclear power before the incident. Nonetheless, panel mortality can be reduced. For example, in the Swiss Food Panel design that measures self-reported eating behavior, perception of food technologies, and other nutrition-related aspects, my research team mailed survey forms to 20,912 randomly selected household addresses from the Swiss telephone book.(6) We had a response rate of 30%. A year later, all the respondents were contacted, and 78% participated again. The subsequent year, 87% participated once more. Between the waves, we sent our study participants a summary
ETH Zurich, Institute for Environmental Decisions (IED), Con¨ sumer Behavior, Universitatsstrasse 22, CHN J76.3, CH-8092, Zurich, Switzerland; [email protected].
1376
C 2014 Society for Risk Analysis 0272-4332/14/0100-1376$22.00/1
Longitudinal Studies on Risk Research of some of our findings, and we asked them to inform us of any address changes. These measures have certainly helped decrease panel mortality. Most studies examining risk perception or risk communication use cross-sectional designs. This is not always the best possible type, but cost considerations and prolonged data collection time prevent researchers from using longitudinal study designs, even if they are more suitable for answering the research question. Nevertheless, various studies about the risk domain that have employed longitudinal panel designs(7–9) show that additional knowledge can be gained in utilizing such designs. These designs are especially important when examining the impact of accidents or crisis situations on risk perception(7,9) or when investigating the relationship between risk perception and health behavior.(8,10) In these studies, change scores could be computed for each participant. Therefore, it is possible to examine the factors that influence changes in risk behavior or in the acceptance of a technology. The quasi-experiment is an important research design to improve our understanding of changes in human health caused by hazards or risk regulations.(11) The nonrandom assignment of the study participants to the experimental or the control group is the main characteristic of this research design. The influence of regulatory actions or accidents is beyond the researcher’s control. As a result, true experiments are not feasible. It should be noted, however, that panel designs cannot solve all the problems in causal inference in empirical research.(12) Because most of the time different models can be estimated for a given data set, there are several models that can be accepted for explaining the data observed. In a previous commentary, I have already emphasized the need for more longitudinal studies.(2) It is surprising how little is known about the stability of people’s risk perceptions and what factors are most important for changes in risk perception. Moreover, there must be serious barriers for using such a research design. All of the various data collection methods described by Greenberg and Weiner(1) can be used for longitudinal designs, of course. How-
1377 ever, it is also true that costs and the amount of work are higher for a longitudinal design, compared with a cross-sectional design. The additional costs of a longitudinal design are heavily influenced by the mode of data collection; for instance, a telephone survey is more expensive than an Internet survey. Nonetheless, my experience in several longitudinal panel studies has convinced me that researchers tend to overestimate the extra expenses of a longitudinal research design. It may be more expensive, but its additional costs are often negligible compared with the other costs of a study. Furthermore, not the costs, but the research question determines which research design should be used. I fully agree with Menard(3) that if a longitudinal design is necessary, then the question should not be about its expense but whether the research should be conducted properly or not done at all.
REFERENCES 1. Greenberg MR, Weiner MD. Keeping surveys valid, reliable, and useful: A tutorial. Risk Analysis, 2014; 34(8):1362–1375. 2. Siegrist M. The necessity for longitudinal studies in risk perception research. Risk Analysis, 2013; 33(1):50–51. 3. Menard S. Longitudinal Research. Thousand Oaks, CA: Sage, 2002. 4. Gaskell G. The 2010 Eurobarometer on the life sciences. Nature Biotechnology, 2011; 29(2):113–114. 5. Siegrist M, Suetterlin B, Keller C. Why have some people changed their attitudes toward nuclear power after the accident in Fukushima? Energy Policy, 2014; 69:356–363. 6. Hartmann C, Dohle S, Siegrist M. Time for change? Food choices in the transition to cohabitation and parenthood. Public Health Nutrition, in press. doi:10.1017/S1368980013003297. 7. Burns WJ, Peters E, Slovic P. Risk perception and the economic crisis: A longitudinal study of the trajectory of perceived risk. Risk Analysis, 2012; 32(4):659–677. ¨ B, Sniehotta FF. Preventive health behavior 8. Renner B, Schuz and adaptive accuracy of risk perceptions. Risk Analysis, 2008; 28(3):741–748. 9. Visschers HM, Siegrist M. How a nuclear power plant accident influences acceptance of nuclear power: Results of a longitudinal study before and after the Fukushima disaster. Risk Analysis, 2013; 33(2):333–347. 10. Brewer NT, Weinstein ND, Cuite CL, Herrington J. Risk perceptions and their relation to risk behavior. Annals of Behavioral Medicine, 2004; 27:125–130. 11. Dominici F, Greenstone M, Sunstein CR. Particulate matter matters. Science, 2014; 344:257–259. 12. Finkel SE. Causal Analysis with Panel Data. Thousand Oaks, CA: Sage, 1995.
DOI: 10.1111/risa.12249
Commentary
Longitudinal Studies on Risk Research Michael Siegrist
In their informative and useful tutorial on surveys involving the risk domain, Greenberg and Weiner(1) devote only a short paragraph to repeated and longitudinal surveys. Given my experiences with reviewers and the few published longitudinal studies about risk perception and risk communication,(2) it may be helpful to include a brief supplement that focuses on some questions related to longitudinal research designs. To avoid any confusion, a longitudinal research design can be defined by three characteristics:(3) (1) data for the same variables are collected during two or more distinct time periods, (2) the subjects in the different data collection waves are the same or at least comparable, and (3) the statistical analysis involves some sort of comparisons between or among the periods. The two most often used types are the repeated cross-sectional and longitudinal panel designs. Other longitudinal designs exist,(3) but space limitations prevent me from describing them in this commentary. In the repeated cross-sectional design, independent probability samples from the population are drawn at each measurement point. For example, the Eurobarometer uses such a design. For each country in the European Union, random samples are used, and for each wave, new samples are drawn. Some Eurobarometer surveys are related to risk perception and acceptance of new technologies.(4) This design allows the measurement of trends across time. However, it is not possible to calculate individual change scores because different subjects participate in each data wave. Therefore, this design is less useful if researchers are interested in intracohort changes or in testing causality. It is not possible to examine why
people have increased or decreased risk perceptions, for example. The results of cross-sectional designs may also be misleading when examining the influence of risk perception on behavior.(2) In the longitudinal panel design, the same subjects participate in each wave of data collection. If researchers are interested in identifying or estimating the strength of causal relationships, this design should be preferred. However, panel attrition or mortality is a challenge of this design. Some participants may die between the measurement points. Other subjects may be unwilling to cooperate any more, or they may have changed their addresses and can no longer be located. These missing data can be a serious problem if they are not random but biased. In one of my research team’s studies, we measured people’s acceptance of nuclear power before and after the nuclear power accident in Fukushima, Japan.(5) We conducted a telephone survey, with a three-year interval between the two waves. Even though it was not planned as a longitudinal panel design, 46% of the first-wave respondents also participated in the second wave. However, a small bias may have influenced our study’s results. The participants who were against nuclear power prior to the Fukushima disaster were more likely to participate in the second wave, compared with those in favor of nuclear power before the incident. Nonetheless, panel mortality can be reduced. For example, in the Swiss Food Panel design that measures self-reported eating behavior, perception of food technologies, and other nutrition-related aspects, my research team mailed survey forms to 20,912 randomly selected household addresses from the Swiss telephone book.(6) We had a response rate of 30%. A year later, all the respondents were contacted, and 78% participated again. The subsequent year, 87% participated once more. Between the waves, we sent our study participants a summary
ETH Zurich, Institute for Environmental Decisions (IED), Con¨ sumer Behavior, Universitatsstrasse 22, CHN J76.3, CH-8092, Zurich, Switzerland; [email protected].
1376
C 2014 Society for Risk Analysis 0272-4332/14/0100-1376$22.00/1
Longitudinal Studies on Risk Research of some of our findings, and we asked them to inform us of any address changes. These measures have certainly helped decrease panel mortality. Most studies examining risk perception or risk communication use cross-sectional designs. This is not always the best possible type, but cost considerations and prolonged data collection time prevent researchers from using longitudinal study designs, even if they are more suitable for answering the research question. Nevertheless, various studies about the risk domain that have employed longitudinal panel designs(7–9) show that additional knowledge can be gained in utilizing such designs. These designs are especially important when examining the impact of accidents or crisis situations on risk perception(7,9) or when investigating the relationship between risk perception and health behavior.(8,10) In these studies, change scores could be computed for each participant. Therefore, it is possible to examine the factors that influence changes in risk behavior or in the acceptance of a technology. The quasi-experiment is an important research design to improve our understanding of changes in human health caused by hazards or risk regulations.(11) The nonrandom assignment of the study participants to the experimental or the control group is the main characteristic of this research design. The influence of regulatory actions or accidents is beyond the researcher’s control. As a result, true experiments are not feasible. It should be noted, however, that panel designs cannot solve all the problems in causal inference in empirical research.(12) Because most of the time different models can be estimated for a given data set, there are several models that can be accepted for explaining the data observed. In a previous commentary, I have already emphasized the need for more longitudinal studies.(2) It is surprising how little is known about the stability of people’s risk perceptions and what factors are most important for changes in risk perception. Moreover, there must be serious barriers for using such a research design. All of the various data collection methods described by Greenberg and Weiner(1) can be used for longitudinal designs, of course. How-
1377 ever, it is also true that costs and the amount of work are higher for a longitudinal design, compared with a cross-sectional design. The additional costs of a longitudinal design are heavily influenced by the mode of data collection; for instance, a telephone survey is more expensive than an Internet survey. Nonetheless, my experience in several longitudinal panel studies has convinced me that researchers tend to overestimate the extra expenses of a longitudinal research design. It may be more expensive, but its additional costs are often negligible compared with the other costs of a study. Furthermore, not the costs, but the research question determines which research design should be used. I fully agree with Menard(3) that if a longitudinal design is necessary, then the question should not be about its expense but whether the research should be conducted properly or not done at all.
REFERENCES 1. Greenberg MR, Weiner MD. Keeping surveys valid, reliable, and useful: A tutorial. Risk Analysis, 2014; 34(8):1362–1375. 2. Siegrist M. The necessity for longitudinal studies in risk perception research. Risk Analysis, 2013; 33(1):50–51. 3. Menard S. Longitudinal Research. Thousand Oaks, CA: Sage, 2002. 4. Gaskell G. The 2010 Eurobarometer on the life sciences. Nature Biotechnology, 2011; 29(2):113–114. 5. Siegrist M, Suetterlin B, Keller C. Why have some people changed their attitudes toward nuclear power after the accident in Fukushima? Energy Policy, 2014; 69:356–363. 6. Hartmann C, Dohle S, Siegrist M. Time for change? Food choices in the transition to cohabitation and parenthood. Public Health Nutrition, in press. doi:10.1017/S1368980013003297. 7. Burns WJ, Peters E, Slovic P. Risk perception and the economic crisis: A longitudinal study of the trajectory of perceived risk. Risk Analysis, 2012; 32(4):659–677. ¨ B, Sniehotta FF. Preventive health behavior 8. Renner B, Schuz and adaptive accuracy of risk perceptions. Risk Analysis, 2008; 28(3):741–748. 9. Visschers HM, Siegrist M. How a nuclear power plant accident influences acceptance of nuclear power: Results of a longitudinal study before and after the Fukushima disaster. Risk Analysis, 2013; 33(2):333–347. 10. Brewer NT, Weinstein ND, Cuite CL, Herrington J. Risk perceptions and their relation to risk behavior. Annals of Behavioral Medicine, 2004; 27:125–130. 11. Dominici F, Greenstone M, Sunstein CR. Particulate matter matters. Science, 2014; 344:257–259. 12. Finkel SE. Causal Analysis with Panel Data. Thousand Oaks, CA: Sage, 1995.
