Skip to Main Content
Skip Nav Destination

Listing decisions are a form of risk assessment supported largely by expert judgement. Expert judgements of rare events in novel circumstances are error prone. Experts are susceptible to social influences and their views are shaped by context, framing, and personal values. Expert judgment is a legitimate source of knowledge, but only if it is fallible. A range of behavioural and numerical methods may be employed to elicit, combine and communicate opinions in deliberation processes by expert groups. Expert groups provide an efficient and convenient way of synthesising a broad range of knowledge but their advice may be improved by training, feedback of performance, consistent communication formats, and awareness of motivational and other sources of bias. A suite of behavioural and numerical methods to aggregate opinions and communicate uncertainty provides new opportunities.

Akçakaya, H. R., Ferson, S., Burgman, M. A., Keith, D. A., Mace, G. M. and Todd, C. R. 2000. Making consistent IUCN classifications under uncertainty. Conservation Biology 14: 1001-1013.
ALRC. 1985. Evidence. Australian Law Reform Commission Report 26. Volume 1. Australian Government Publishing Service, Canberra. http://www.austlii.edu.au/au/other/alrc/publications/reports/26/
Anonymous. 1995. Biodiversity: the UK Steering Group report. Volume 2. Action Plans, HMSO, London.
Ayyub, B. M. 2001 Elicitation of expert opinions for uncertainty and risks. CRC Press, Boca Raton.
Baran, N, 2000. Effective survey methods for detecting plants. MSc Thesis, School of Botany, University of Melbourne.
Bier, V. 2002. Implications of the research on overconfidence for challenge problem solution strategies. Paper presented to the SANDIA Conference on Uncertainty. Albuquerque: New Mexico, August 2002.
Bier, V. M., Haimes, Y. Y., Lambert, J. H., Matalas, N. C. and Zimmerman, R. 1999. A survey of approaches for assessing and managing the risks of extremes. Risk Analysis 19: 83-94.
Briggs, J. D. and Leigh, J. H. 1996. Rare or threatened Australian plants. CSIRO, Melbourne.
Burgman, M. A. 2002. Are listed threatened plant species actually at risk? Australian Journal of Botany 50: 1-13.
Burgman, M. A. 2004 Risks and decisions for conservation and environmental management. Cambridge University Press, Cambridge (in press).
Campbell, L. M. 2002. Science and sustainable use: views of marine turtle conservation experts. Ecological Applications 12: 1229-1246.
Christensen-Szalanski, J. and Bushyhead, J. 1981. Physicians use of probabilistic information in a real clinical setting. Journal of Experimental Psychology: Human Perception and Performance 7: 928-935.
Clemen, R. T. and Winkler, R. L. 1999. Combining probability distributions from experts in risk analysis. Risk Analysis 19: 187-203.
Cooke, R. M. 1991 Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, Oxford.
Engelhardt, H. and Caplan, H. 1986. Patterns of controversy and closure: the interplay of knowledge, values, and political forces. Pp. 1-23 in Scientific controversies: case studies in the resolution and closure of disputes in science and technology, edited by H. Engelhardt and H. Caplan. Cambridge University Press, New York.
Ferson, S., Kreinovich, V., Ginzburg, L., Myers, D. and Sentz, K. 2003. Constructing probability boxes and Dempster-Shafer structures. SAND2002-4015. Sandia National Laboratories, Albuquerque: New Mexico.
Fischhoff, B. 1995. Risk perception and communication unplugged: twenty years of progress. Risk Analysis 15: 137-145.
Fischhoff, B., Slovic, P. and Lichtenstein, S. 1982. Lay foibles and expert fables in judgements about risk. American Statistician 36, 240-255.
Freckelton, I. 1995. The challenge of junk psychiatry, psychology and science: the evolving role of the forensic expert. Pp. 58-59 in Tomorrow's law, edited by H. Selby. Federation Press, Sydney.
Freudenburg, W. R. 1996. Risky thinking: irrational fears about risk and society. Annals AAPSS 545: 44-53.
Freudenburg, W. R 1999. Tools for understanding the socioeconomic and political settings for environmental decision making. Pp. 94-125. in Tools to aid environmental decision making, editied by V. H. Dale and M. R. English. Springer, New York.
Gigone, D. and Hastie, R. 1997. Proper analysis of the accuracy of group judgments. Psychological Bulletin 121: 149-167.
Gigerenzer, G. 2002 Calculated risks: how to know when numbers deceive you. Simon and Schuster, New York.
Groombridge, B. (Ed) 1994. Biodiversity data sourcebook. World Conservation Monitoring Centre, Biodiversity Series No 1. World Conservation Press, Cambridge.
Hart, A. 1986 Knowledge acquisition for expert systems. McGraw-Hill, New York.
IUCN. 1994. IUCN red list of threatened animals. International Union for the Conservation of Nature, Gland: Switzerland.
IUCN. 2001. IUCN Red List Categories. IUCN Species Survival Commission, The World Conservation Union, Gland: Switzerland.
Kahneman, D. and Tversky, A. 1979. Prospect theory: an analysis of decision under risk. Econometrica 47: 263-291.
Kahneman, D. and Tversky, A. 1984. Choices, values, and frames. American Psychologist 39: 342-347.
Keith, D. A. and Burgman, M. A. 2004. The Lazarus effect: can the dynamics of extinct species lists tell us anything about the status of biodiversity? Biological Conservation 117: 41-48.
Kerr, R. 1996. A new way to ask the experts: rating radioactive waste risks. Science 274: 913-914.
Klir, G. and Wierman, M. J. 1998 Uncertainty-based information: elements of generalized information theory. Physica-Verlag, Heidelberg.
Krinitzsky, E. L. 1993. Earthquake probability in engineering - Part 1: The use and misuse of expert opinion. Engineering Geology 33: 257-288.
Lawson, J. D. 1900 The law of expert and opinion evidence (2nd edition). T. H. Flood, Chicago.
Lehrer, K. 1997. Consensus, negotiation and mediation. Pp. 3-14. in Consensus under fuzziness, edited by J. Kacprzyk, H. Nurmi and M. Fedrizzi. Kluwer, Boston.
Lehrer, K. and Wagner, C. 1981 Rational consensus in science and society. Reidel, Dordecht.
Lunney, D., Curtin, A., Ayers, D., Cogger, H. G. and Dickman, C. R. 1996. An ecological approach to identifying the endangered fauna of New South Wales. Pacific Conservation Biology 2: 212-231.
May, R. M., Lawton, J. H. and Stork, N. E. 1995. Assessing extinction rates. Pp. 1-24. in Extinction rates, edited by J. H. Lawton and R. M. May. Oxford University Press, Oxford.
Meyer, M. A. and Booker, J. M. 1990. Eliciting and analyzing expert judgment: a practical guide. Office of Nuclear Regulatory Research, Division of Systems Research, US Nuclear Regulatory Commission, Washington, DC.
Morgan, M. G. and M. Henrion. 1990 Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press, Cambridge.
Morgan, M. G., Fischhoff, B., Lave, L. and Fischbeck, P. 1996. A proposal for ranking risk within Federal agencies. Pp. 111-147. in Comparing environmental risks, edited by Davies, J. C. Resources for the Future, Washington, DC.
Plous, S. 1993 The psychology of judgment and decision making. McGraw-Hill, New York.
Possingham, H. P. Andelman, S. J., Burgman, M. A., Medellín, R. A., Master, L. L. and Keith, D. A. 2002a. Limits to the use of threatened species lists. Trends in Ecology and Evolution 17: 503-507.
Regan, H. M., Colyvan, M. and Burgman, M. A. 2002. A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecological Applications 12: 618-628.
Regan, T. J., Burgman, M. A., McCarthy, M. A., Mace, G. M., Master, L. L., Keith, D. A. and Andelman, S. J. 2004. The consistency of extinction risk classifications. Conservation Biology (in review).
Ruckelshaus, M. H., Levin, P., Johnson, J. B., and Kareiva, P. M. 2002. The Pacific Salmon wars: what science brings to the challenge of recovering species. Annual Review of Ecology and Systematics 33: 665-706.
Sentz, K. and Ferson, S. 2002. Combination of evidence in Dempster-Shafer theory. SAND Report, SAND2002-0835. Sandia National Laboratories, Albuquerque, New Mexico.
Slovic, P. Fischhoff, B. and Lichtenstein, S. 1984. Perception and acceptability of risk from energy systems. Pp. 115-135. in Public reactions to nuclear power: are there critical masses? edited by W. R. Freundenburg and E. A. Rosa. AAAS/Westview, Boulder, Colorado.
Slovic, P., Monahan, J. and MacGregor, D. G. 2000. Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency formats. Law and Human Behavior 24: 271-296.
Trumbo, C. W. and McComas, K. A. 2003. The function of credibility in information processing for risk perception. Risk Analysis 23: 343-353.
Tufte, E. 1997 Visual explanations: images and quantities, evidence and narrative. Graphics Press, Cheshire, Connecticut.
Valverde, L. J. 2001. Expert judgment resolution in technicallyintensive policy disputes. Pp. 221-238. in Assessment and management of environmental risks, edited by I. Linkov and J. Palma-Oliveira. Kluwer Academic, Dordrecht, The Netherlands.
Vose, D. 1996 Quantitative risk analysis: a guide to Monte Carlo simulation modelling. Wiley, Chichester.
Walley, P. 1991 Statistical reasoning with imprecise probabilities. Chapman and Hall, London.
Walley, P. and DeCooman, G. 2001. A behavioral model for linguistic uncertainty. Information Sciences 134: 1-37.
Walton, D. 1997 Appeal to expert opinion: arguments from authority. Pennsylvania State University Press, Pennsylvania.
Wright, G., Bolger, F. and Rowe, G. 2002. An empirical test of the relative validity of expert and lay judgments of risk. Risk Analysis 22: 1107-1122.
This content is PDF only. Please click on the PDF icon to access.

Contents

Figures & Tables

References

Akçakaya, H. R., Ferson, S., Burgman, M. A., Keith, D. A., Mace, G. M. and Todd, C. R. 2000. Making consistent IUCN classifications under uncertainty. Conservation Biology 14: 1001-1013.
ALRC. 1985. Evidence. Australian Law Reform Commission Report 26. Volume 1. Australian Government Publishing Service, Canberra. http://www.austlii.edu.au/au/other/alrc/publications/reports/26/
Anonymous. 1995. Biodiversity: the UK Steering Group report. Volume 2. Action Plans, HMSO, London.
Ayyub, B. M. 2001 Elicitation of expert opinions for uncertainty and risks. CRC Press, Boca Raton.
Baran, N, 2000. Effective survey methods for detecting plants. MSc Thesis, School of Botany, University of Melbourne.
Bier, V. 2002. Implications of the research on overconfidence for challenge problem solution strategies. Paper presented to the SANDIA Conference on Uncertainty. Albuquerque: New Mexico, August 2002.
Bier, V. M., Haimes, Y. Y., Lambert, J. H., Matalas, N. C. and Zimmerman, R. 1999. A survey of approaches for assessing and managing the risks of extremes. Risk Analysis 19: 83-94.
Briggs, J. D. and Leigh, J. H. 1996. Rare or threatened Australian plants. CSIRO, Melbourne.
Burgman, M. A. 2002. Are listed threatened plant species actually at risk? Australian Journal of Botany 50: 1-13.
Burgman, M. A. 2004 Risks and decisions for conservation and environmental management. Cambridge University Press, Cambridge (in press).
Campbell, L. M. 2002. Science and sustainable use: views of marine turtle conservation experts. Ecological Applications 12: 1229-1246.
Christensen-Szalanski, J. and Bushyhead, J. 1981. Physicians use of probabilistic information in a real clinical setting. Journal of Experimental Psychology: Human Perception and Performance 7: 928-935.
Clemen, R. T. and Winkler, R. L. 1999. Combining probability distributions from experts in risk analysis. Risk Analysis 19: 187-203.
Cooke, R. M. 1991 Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, Oxford.
Engelhardt, H. and Caplan, H. 1986. Patterns of controversy and closure: the interplay of knowledge, values, and political forces. Pp. 1-23 in Scientific controversies: case studies in the resolution and closure of disputes in science and technology, edited by H. Engelhardt and H. Caplan. Cambridge University Press, New York.
Ferson, S., Kreinovich, V., Ginzburg, L., Myers, D. and Sentz, K. 2003. Constructing probability boxes and Dempster-Shafer structures. SAND2002-4015. Sandia National Laboratories, Albuquerque: New Mexico.
Fischhoff, B. 1995. Risk perception and communication unplugged: twenty years of progress. Risk Analysis 15: 137-145.
Fischhoff, B., Slovic, P. and Lichtenstein, S. 1982. Lay foibles and expert fables in judgements about risk. American Statistician 36, 240-255.
Freckelton, I. 1995. The challenge of junk psychiatry, psychology and science: the evolving role of the forensic expert. Pp. 58-59 in Tomorrow's law, edited by H. Selby. Federation Press, Sydney.
Freudenburg, W. R. 1996. Risky thinking: irrational fears about risk and society. Annals AAPSS 545: 44-53.
Freudenburg, W. R 1999. Tools for understanding the socioeconomic and political settings for environmental decision making. Pp. 94-125. in Tools to aid environmental decision making, editied by V. H. Dale and M. R. English. Springer, New York.
Gigone, D. and Hastie, R. 1997. Proper analysis of the accuracy of group judgments. Psychological Bulletin 121: 149-167.
Gigerenzer, G. 2002 Calculated risks: how to know when numbers deceive you. Simon and Schuster, New York.
Groombridge, B. (Ed) 1994. Biodiversity data sourcebook. World Conservation Monitoring Centre, Biodiversity Series No 1. World Conservation Press, Cambridge.
Hart, A. 1986 Knowledge acquisition for expert systems. McGraw-Hill, New York.
IUCN. 1994. IUCN red list of threatened animals. International Union for the Conservation of Nature, Gland: Switzerland.
IUCN. 2001. IUCN Red List Categories. IUCN Species Survival Commission, The World Conservation Union, Gland: Switzerland.
Kahneman, D. and Tversky, A. 1979. Prospect theory: an analysis of decision under risk. Econometrica 47: 263-291.
Kahneman, D. and Tversky, A. 1984. Choices, values, and frames. American Psychologist 39: 342-347.
Keith, D. A. and Burgman, M. A. 2004. The Lazarus effect: can the dynamics of extinct species lists tell us anything about the status of biodiversity? Biological Conservation 117: 41-48.
Kerr, R. 1996. A new way to ask the experts: rating radioactive waste risks. Science 274: 913-914.
Klir, G. and Wierman, M. J. 1998 Uncertainty-based information: elements of generalized information theory. Physica-Verlag, Heidelberg.
Krinitzsky, E. L. 1993. Earthquake probability in engineering - Part 1: The use and misuse of expert opinion. Engineering Geology 33: 257-288.
Lawson, J. D. 1900 The law of expert and opinion evidence (2nd edition). T. H. Flood, Chicago.
Lehrer, K. 1997. Consensus, negotiation and mediation. Pp. 3-14. in Consensus under fuzziness, edited by J. Kacprzyk, H. Nurmi and M. Fedrizzi. Kluwer, Boston.
Lehrer, K. and Wagner, C. 1981 Rational consensus in science and society. Reidel, Dordecht.
Lunney, D., Curtin, A., Ayers, D., Cogger, H. G. and Dickman, C. R. 1996. An ecological approach to identifying the endangered fauna of New South Wales. Pacific Conservation Biology 2: 212-231.
May, R. M., Lawton, J. H. and Stork, N. E. 1995. Assessing extinction rates. Pp. 1-24. in Extinction rates, edited by J. H. Lawton and R. M. May. Oxford University Press, Oxford.
Meyer, M. A. and Booker, J. M. 1990. Eliciting and analyzing expert judgment: a practical guide. Office of Nuclear Regulatory Research, Division of Systems Research, US Nuclear Regulatory Commission, Washington, DC.
Morgan, M. G. and M. Henrion. 1990 Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press, Cambridge.
Morgan, M. G., Fischhoff, B., Lave, L. and Fischbeck, P. 1996. A proposal for ranking risk within Federal agencies. Pp. 111-147. in Comparing environmental risks, edited by Davies, J. C. Resources for the Future, Washington, DC.
Plous, S. 1993 The psychology of judgment and decision making. McGraw-Hill, New York.
Possingham, H. P. Andelman, S. J., Burgman, M. A., Medellín, R. A., Master, L. L. and Keith, D. A. 2002a. Limits to the use of threatened species lists. Trends in Ecology and Evolution 17: 503-507.
Regan, H. M., Colyvan, M. and Burgman, M. A. 2002. A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecological Applications 12: 618-628.
Regan, T. J., Burgman, M. A., McCarthy, M. A., Mace, G. M., Master, L. L., Keith, D. A. and Andelman, S. J. 2004. The consistency of extinction risk classifications. Conservation Biology (in review).
Ruckelshaus, M. H., Levin, P., Johnson, J. B., and Kareiva, P. M. 2002. The Pacific Salmon wars: what science brings to the challenge of recovering species. Annual Review of Ecology and Systematics 33: 665-706.
Sentz, K. and Ferson, S. 2002. Combination of evidence in Dempster-Shafer theory. SAND Report, SAND2002-0835. Sandia National Laboratories, Albuquerque, New Mexico.
Slovic, P. Fischhoff, B. and Lichtenstein, S. 1984. Perception and acceptability of risk from energy systems. Pp. 115-135. in Public reactions to nuclear power: are there critical masses? edited by W. R. Freundenburg and E. A. Rosa. AAAS/Westview, Boulder, Colorado.
Slovic, P., Monahan, J. and MacGregor, D. G. 2000. Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency formats. Law and Human Behavior 24: 271-296.
Trumbo, C. W. and McComas, K. A. 2003. The function of credibility in information processing for risk perception. Risk Analysis 23: 343-353.
Tufte, E. 1997 Visual explanations: images and quantities, evidence and narrative. Graphics Press, Cheshire, Connecticut.
Valverde, L. J. 2001. Expert judgment resolution in technicallyintensive policy disputes. Pp. 221-238. in Assessment and management of environmental risks, edited by I. Linkov and J. Palma-Oliveira. Kluwer Academic, Dordrecht, The Netherlands.
Vose, D. 1996 Quantitative risk analysis: a guide to Monte Carlo simulation modelling. Wiley, Chichester.
Walley, P. 1991 Statistical reasoning with imprecise probabilities. Chapman and Hall, London.
Walley, P. and DeCooman, G. 2001. A behavioral model for linguistic uncertainty. Information Sciences 134: 1-37.
Walton, D. 1997 Appeal to expert opinion: arguments from authority. Pennsylvania State University Press, Pennsylvania.
Wright, G., Bolger, F. and Rowe, G. 2002. An empirical test of the relative validity of expert and lay judgments of risk. Risk Analysis 22: 1107-1122.
Close Modal

or Create an Account

Close Modal
Close Modal