Structured Decision-Making to Rank North American Wetlands Conservation Act Proposals Within Joint Venture Regions

Conservation practitioners face complex resource allocation decisions, with multiple (sometimes competing) objectives, various stakeholder values, and limited financial resources. With a growing number and magnitude of conservation challenges, only the highest-priority needs can be addressed given the limited financial and human resources. Therefore, decisions about resource allocation necessitate prioritization (Possingham et al. 2001). Decision analysis provides a rigorous framework for making decisions about environmental management and resource allocation (Possingham et al. 2001; Kiker et al. 2005; Lyons 2020). Decision analysis is comprised of the philosophy, theory, and methodology necessary to address decisions in a formal manner (Raiffa 1968). It includes procedures and tools for identifying, clearly representing, and formally assessing aspects of a decision and prescribing recommended actions. Foundations of decision analysis are rooted in individual decision theory, social choice theory, and game theory (von Winterfeldt and Edwards 1986). Application of decision analysis has a long history in fields such as economics (Chunchachinda et al. 1997), medicine (Hazen et al. 1998), and public policy (Keeney and von Winterfeldt 1994). However, application of decision analysis to environmental decisions is relatively recent and has gained significant momentum in the last 20 y (Kiker et al. 2005; Huang et al. 2011; Gregory et al. 2012; Runge et al. 2020). Examples of decision analysis applied to conservation decisions include resource allocation among regions (Wilson et al. 2006; Murdoch et al. 2007; Bode et al. 2008), reserve design (Possingham et al. 2000; Moore et al. 2004; Albers et al. 2016), allocation of funds for species conservation programs (Joseph et al. 2009; Gerber et al. 2018; Wu et al. 2021), monitoring programs (Hauser and McCarthy 2009; Convertino et al. 2013), and managing migratory and endangered species (Martin et al. 2007; Edwards et al. 2019).

Monetary and human resources for bird habitat conservation in North America come from many sources, including the Migratory Bird Conservation Fund (16 U.S.C. 718–718j), Land and Water Conservation Fund (16 U.S.C. 460l–460l-11), Conservation Title of the U.S. Farm Bill (16 U.S.C. 3801–3862), Endangered Species Conservation Fund (16 U.S.C. 1531–1544), and North American Wetlands Conservation Fund (16 U.S.C. 4401–4412). We focused on resource allocation decisions made under the North American Wetlands Conservation Fund, enabled by the North American Wetland Conservation Act (NAWCA; 16 U.S.C. 4401–4412) in 1989. The NAWCA Fund provides grants to protect and manage wetlands for migratory birds and other wildlife (USFWS 2021). Since its passage, NAWCA grants have totaled over US$1.9 billion and leveraged US$3.9 billion in matching contributions from partners in Mexico, Canada, and the United States (USFWS 2021). More than 3,100 NAWCA-funded projects have contributed to the conservation of nearly 31.5 million ha of wildlife habitat across North America (USFWS 2021). Additionally, NAWCA grants are a key source of funding for the implementation of the North American Waterfowl Management Plan through migratory bird Joint Ventures (JVs; Humburg et al. 2017; Brasher et al. 2019). The North American Waterfowl Management Plan was established in 1986 by the Secretary of the Interior for the United States and the Minister of Environment for Canada to pursue cooperative planning and coordinated management of waterfowl. The Plan is implemented through JVs, which are collaborative, regional partnerships of government agencies, tribes, nonprofit organizations, corporations, universities, and individuals that conserve habitat for the benefit of priority bird species, other wildlife, and people. There are 22 JVs in North America ranging in size from > 30 million ha (Western Boreal Area of Interest JV) to 1.5 million ha (San Francisco Bay JV). Each JV has a management board that is made up of key representatives from the organizations forming the partnership. The management board provides overall leadership, guidance, resources, and support to partners to ensure that the JV reaches its bird and habitat conservation goals (MBJV 2017).

Proposals for NAWCA grants are submitted annually to the U.S. Fish and Wildlife Service (USFWS) following proposal guidelines (USFWS 2022). Each JV receives copies of all proposals relevant to their geography. Proposals are first ranked by individual JV management boards. These ranks are submitted to the North American Wetlands Conservation Council (hereafter, the Council) for further evaluation. The Council includes nine members representing state and federal management agencies and nongovernmental organizations. The Council meets three times per year to select grant proposals to take before the Migratory Bird Conservation Commission (MBCC), which is the ultimate decision maker for grant allocation. The MBCC is composed of the U.S. Secretary of Agriculture, U.S. Secretary of the Interior, U.S. Environmental Protection Agency Administrator, and two members each from the U.S. Senate and the U.S. House of Representatives.

Although prioritization of project proposals occurs at the Council level, council members rely on input from JVs regarding proposals submitted from their respective regions. Currently, the level of structured and data-driven decision frameworks applied regionally for making recommendations to the Council varies greatly (J. Foth, U.S. Fish and Wildlife Service, unpublished data). To facilitate commensurate ranking of proposals by different JVs, the Council, in collaboration with USFWS and JVs, provided seven ranking criteria in June 2016 (Figure 1; D. Gordon and K. Kriese, U.S. Fish and Wildlife Service, unpublished data). Some JVs have little to no structured methodology for ranking project proposals and instead use a combination of expert opinion and the seven ranking criteria outlined by the Council to make their final funding recommendations (J. Foth, U.S. Fish and Wildlife Service, unpublished data). Others (e.g., Gulf Coast JV [GCJV]) have established working groups to review proposal benefits toward specific species or guilds, and rank proposals on the basis of their contribution toward achieving JV-specific habitat objectives (B. Wilson, U.S. Fish and Wildlife Service, personal communication). In general, JVs seek to align their regional conservation priorities and objectives with the seven broad technical assessment questions that reflect the NAWCA purpose and intent (USFWS 2022). Joint Ventures aim to rank proposals from their region on the basis of how each proposal addresses those NAWCA criteria and other JV-specific priorities and objectives, and communicate their proposal ranks to the Council (Figure 1). However, differences among JV methodologies at establishing final rankings among proposals within a funding cycle has led USFWS, Council, and JVs over time to strive for a more consistent and transparent framework for providing recommendations on proposal ranks at the JV scale (D. Gordon and K. Kriese, U.S. Fish and Wildlife Service, unpublished data).

One aim of this study was to develop applications of structured decision-making with a linear additive model for ranking NAWCA proposals within a JV region to increase transparency of the prioritization and reduce certain inherent biases that are common in unstructured prioritizations. Structured decision-making is an organized, inclusive, and transparent approach to understanding complex problems and achieving fundamental objectives (Gregory et al. 2012; Runge et al. 2020). Structured decision-making is a type of decision analysis that emphasizes a collaborative approach to 1) defining the problem, 2) identifying objectives, 3) creating alternatives, 4) predicting consequences of alternatives, and 5) making trade-offs to determine the most appropriate decision(s). The acronym PrOACT (problem, objectives, alternatives, consequences, and trade-offs) is commonly used to describe the framework to structure decisions (Hammond et al. 2002). First, the decision problem is framed by describing elements such as decision maker, potential actions, ultimate goals to be achieved, temporal and spatial extent of the problem, and any constraints; these elements are identified independently and often assembled in a problem statement (Gregory et al. 2012; Smith 2020). Second, objectives are concise statements of “what matters” (Gregory et al. 2012). Each objective may stand independently or may be composed of subobjectives. Objective hierarchies clarify the types of objectives and their relationships to one another. There are several types of objectives: biological, social, cost, and process (Keeney 2007). An objective means–ends diagram or objectives hierarchy is often used as a visual tool for organizing the stakeholder collaborative process (Conroy and Peterson 2013). Third, alternatives in a decision process are complete solutions to the problem (Gregory et al. 2012). Fourth, consequences describe the relative performance of each alternative in terms of the objectives. Fifth, trade-offs are common among decisions that have multiple objectives and require assessments about how much a decision maker would give up on one objective to achieve gains in another objective (Gregory et al. 2012). The relative preference for different objectives and acceptable trade-offs is inherently subjective and can differ among stakeholders. Structured decision-making provides many tools that aid in trade-off analysis for multiple objective problems; among the most important of these are practical value models (Keeney and von Winterfeldt 2007).

Complex decision problems with multiple, often competing, objectives require complex value trade-offs. Value models incorporate the values and value trade-offs important to decision makers and provide a means to evaluate consequences. As such, value models fit into the structured decision-making framework as a tool to identify and structure objectives, after problem framing and before identifying the alternatives (Figure 2). A value model is composed of four elements: 1) objectives, 2) performance measures, 3) weights, and 4) value functions (Kirkwood 1997; Keeney and von Winterfeldt 2007). As stated above, objectives are statements of what the decision maker is attempting to achieve; they are tailored to the specific decision context and often encompass competing interests of various stakeholder groups. A performance measure (also called performance metric or measurable attribute) is a specific attribute chosen to quantify expected outcomes in terms of one objective or subobjective (Keeney and Gregory 2005). In every decision context, alternatives will differ in their estimated consequences for at least one objective. Thus, weight assignment and an analysis of trade-offs among objectives are integral parts of the PrOACT approach. Weight elicitation is a vital step because weights represent the value stakeholders place on each objective relative to the others. Assigning weights is a rightly subjective process; therefore, weights may vary greatly depending on characteristics of the region, stakeholder values, and policy.

Within the PrOACT framework, performance measures for each objective are often represented by disparate units (e.g., dollars for cost, kilometers for distance, abundance for species, etc.). The final components of a value model, value functions, are used to normalize disparate performance measures; these functions reflect changes in stakeholder satisfaction with changes in the expected consequences of an alternative (Conroy and Peterson 2013). Value functions are defined through elicitation with the decision maker or, in the case of group decisions, appropriate stakeholders. An ordinal value function, v(y), that has larger values for preferred alternatives exists if we assume that the preferences of a decision maker or stakeholder group for consequences are complete and transitive (Keeney and Raiffa 1976; Eisenführ et al. 2010; Reichert et al. 2015). Complete preferences occur when the decision maker can compare any two possible outcomes and state a preference for one over the other, or that they are indifferent between the two outcomes (Reichert et al. 2015). Transitivity requires that if consequence y^(a) is preferred over consequence y^(b) and y^(b) over y^(c), then consequence y^(a) is preferred over y^(c) (Reichert et al. 2015). An ordinal value function can be scaled to the interval [0,1], where 0 represents the worst and 1 the best consequences. Normalized performance measures of all objectives are weighted and summed for an overall score (hereafter, conservation benefit score) using an aggregation rule. There are several types of aggregation rules, the most common of which are additive and multiplicative. The additive linear aggregation model is generally superior and easier for decision makers to use and understand and the most common method for evaluating alternatives in multicriteria decision analysis (Choo and Wedley 2008; Martin and Mazzotta 2018).

We used structured decision-making to create a transparent, repeatable, and consistent framework for ranking NAWCA proposals within JV regions. We developed two value models following the PrOACT approach for two case studies: the GCJV and the Upper Mississippi/Great Lakes JV (UMGLJV). These two JVs represent two different geographies (Figure 3). We evaluated variation in objectives and weights in the two regions and demonstrated the flexibility of value models for prioritization problems. Although we have worked with representatives of the JVs that are presented here as case studies and have done our best to represent the concerns of each JV, these value models are not a product of the JVs and do not reflect decision-making of the management boards or JV staff. They serve only as case studies to illustrate applicability of structured decision-making to regional NAWCA proposal ranking. We also conducted a sensitivity analysis for objective weights for each JV case study. We believe this value modeling approach could greatly improve the regional NAWCA proposal-ranking process by providing JV coordinators an option for refining core values and a tool for communicating with their management board and the Council. Moreover, other JVs could tailor this process and build a value model specific to their region (i.e., region-specific objectives, performance measures, value functions, and weights).

As described above, the process for ranking NAWCA proposals is a series of linked decisions by JV management boards, the Council, and the MBCC. We addressed the first of these decisions (i.e., the ranking decision made by JV management boards). Stakeholders in our decision context included JV management board members, NAWCA proposal authors, the Council, and, generally, all wetland–wildlife habitat conservation and management practitioners. An eight-member assessment group (hereafter, the expert panel) developed the framework using two JV case studies and tested the resulting proposal ranking for each region with previously submitted proposals. The expert panel consisted of members with expertise in NAWCA proposal ranking, bird conservation modeling, and wetland habitat management and conservation. Four members of the expert panel (two from each JV) represented their management boards and, for the purposes of the case study, served as a proxy for the decision makers. Structured decision-making facilitators led a 3-d workshop, which included the expert panel and additional stakeholders (see Acknowledgments), to create a rapid prototype of the decision analysis (Garrard et al. 2017). We framed the decision as a prioritization problem in which each JV is tasked with developing a ranked list of NAWCA proposals from those submitted within their respective planning boundaries.

The GCJV is a bird habitat conservation partnership with a boundary that encompasses portions of Texas, Louisiana, Mississippi, and Alabama (Figure 3). On its southern and eastern border, the JV region is characterized by coastal land cover types including prairie tidal marshes, cypress swamps, mud and sand flats between barrier islands, and seagrass beds. The interior and northern and western boundaries are characterized by rice agriculture, coastal prairies, and bottomland hardwood forests (Chabreck et al. 1989). The mission of the GCJV is to “advance the conservation of important bird habitats within the GCJV region through biological planning, implementation of habitat conservation actions, and focused monitoring and evaluation of the planning and implementation processes” (GCJV 2020). This region provides habitat for millions of wintering waterfowl and other migratory birds; NAWCA funding serves a critical role in the GCJV's conservation and management initiatives that target wetland bird habitat.

The UMGLJV is one of the largest and most diverse JV regions in the United States, with a boundary that encompasses Michigan, Wisconsin, Indiana, and parts of Ohio, Illinois, Missouri, Iowa, Minnesota, Kansas, and Nebraska (Figure 3). The JV region borders four of the five Great Lakes and cultivated cropland is a dominant cover type (40% of land area), especially in the southern half of the region. The northern half of the JV region has abundant and diverse wetland types with widespread use by waterfowl, whereas in the southern half of the region waterfowl are far more concentrated on diminished habitat availability (90% of wetlands have been drained; Dahl 1990). Additionally, nearly 10% of land is developed and the region has a large urban presence, including several large cities (e.g., Chicago, Minneapolis, Detroit, and Indianapolis). The JV region has substantial breeding waterfowl populations, but from a continental perspective, it may be most important during nonbreeding periods. Food resources in UMGLJV wetlands must support millions of ducks annually (Soulliere et al. 2007). As in the GCJV, NAWCA funding is an important component for achieving waterfowl habitat conservation and restoration targets.

These two regions represent critical wintering, migratory, and breeding geographies for waterfowl and other wetland birds. They are distinct from each other in habitat characteristics, human culture, and management targets and strategies. These differences, as well as variation in availability of spatially explicit biological and social data sets at the regional scale, mandated the development of two distinct value models. However, both JVs rely on NAWCA funding for waterfowl habitat retention and restoration. Therefore, as in most (but not all) JVs, the GCJV and UMGLJV are tasked with selecting and ranking proposals for funding recommendations to the Council. Additionally, both JV management boards must effectively communicate and justify proposal ranks to the Council (D. Gordon and K. Kriese, U.S. Fish and Wildlife Service, unpublished data). Compatibility among JVs and adherence to national NAWCA criteria can be achieved through the application of a consistent framework (i.e., structured decision making and PrOACT) while simultaneously addressing distinct regional priorities with tailored value models. Model development for each JV is described in brief below; detailed documentation of objectives, performance measures, and value functions is provided in Texts S1 and S2 (Supplemental Material) for the GCJV and UMGLJV, respectively.

The expert panel examining the two JV case studies held a workshop 26–28 June 2018 to include stakeholders in framing the decision context and identifying JV-specific objectives in prototype models, and then convened several teleconferences to refine the prototypes. Objectives were identified through roundtable discussion during these meetings by exploring current JV decision-making processes and regional stakeholder values. Eight GCJV objectives were identified and structured in a hierarchy (Table 1; Text S1, Supplemental Material). These were classified as either biological, social, process, or cost objectives (Keeney 2007). The first biological objective (objective 1.0, maximize benefit toward JV habitat objectives, Table 1) was based on habitat needs of four bird guilds of the Gulf Coast: waterfowl, land birds, shorebirds, and waterbirds. Biological planning for bird habitat conservation in the GCJV is led by their Science Team, which is composed of four working groups, one for each bird guild. Currently, Science Team working groups evaluate NAWCA proposals on the expected habitat benefits to each of the four bird guilds. Consistent with current GCJV practices, we defined biological objectives on the basis of the proposal's ability to demonstrate habitat benefits toward each of the four bird guilds and reduce threats to bird habitat. The second biological objective (objective 4.0, minimize threat, Table 1) was evidence and amelioration of threat to natural bird habitat. One social objective (objective 7.0, maximize benefit to people, Table 1), which was divided into three subobjectives, addressed benefits to resource users (i.e., hunters, bird viewers, and landowners). Three additional objectives addressed cost concerns: minimizing loss of match dollars (objective 3.0, minimize probability of lost match, Table 1), maximizing benefits returned on investments (objective 8.0, minimize cost/benefit of projects, Table 1), and tenure of benefits from proposed projects (objective 5.0, maximize tenure of conservation benefits, Table 1), all key concerns of the GCJV Management Board. Finally, two process objectives addressed degree of collaboration between partners (i.e., proposal authors, JV staff, land managers, and conservationists) in development of the proposal (objective 2.0, maximize partnership involvement, Table 1) and clarity of proposed activities (objective 6.0, maximize clarity of proposed activities, Table 1). All objectives were explicitly coupled to one or more of the Council's seven ranking criteria (Figure 1; Table 1; D. Gordon and K. Kriese, U.S. Fish and Wildlife Service, unpublished data), effectively linking regional decision making to national proposal evaluation criteria of the Council and MBCC. Justifications for the inclusion of each objective were discussed in workshops and written narratives were provided by the expert panel in collaboration with the CGJV Coordinator and other stakeholders and represent the mental models currently used by the JV's management board to evaluate proposals (Table 1).

The UMGLJV objectives hierarchy included 10 regional objectives (Table 2; Text S2, Supplemental Material). Two biological objectives were based on UMGLJV conservation planning documents that identify key locations across the region for breeding (objective 1.0, maximize abundance of breeding focal species, Table 2) and nonbreeding (objective 2.0, maximize carrying capacity for nonbreeding focal species, Table 2) focal species of waterfowl and waterbirds (Soulliere et al. 2017, 2018). Three additional biological objectives were related to bird habitat: maximizing area of four wetland classes of greatest importance to waterfowl and waterbirds (objective 3.0, maximize area of high-quality waterfowl and habitat using specific wetland classes described in JV strategy, Table 2); optimizing the configuration of wetland and associated upland plant communities (objective 5.0, maximize favorable wetland/upland configuration, Table 2); and minimizing habitat loss from conversion of natural cover (objective 7.0, minimize threat: risk of conversion from natural cover providing bird habitat, Table 2). One social objective (objective 9.0, maximize benefit to people, Table 2) was divided into three subobjectives addressing benefits to resource users and ecological services (i.e., waterfowl hunting opportunity, bird-watching opportunity, and watershed impairment). Cost-effectiveness objectives included minimizing loss of match dollars (objective 6.0, minimize probability of lost match, Table 2) and maximizing probability of project success (objective 8.0, maximize tenure of conservation benefits, Table 2). Finally, process objectives included the degree of collaboration between proposal authors and JV staff (objectives 10.0, maximize quality of science support and degree of communication with staff, Table 2) and use of appropriate technical planning documents, as well as the level of partner involvement in proposal development (objective 4.0, maximize partnership involvement). All objectives were explicitly coupled to one or more of the Council's seven ranking criteria (Figure 1; Table 2; D. Gordon and K. Kriese, U.S. Fish and Wildlife Service, unpublished data), effectively linking regional decision-making to national proposal evaluation criteria of Council and MBCC. Justifications for the inclusion of each objective were discussed in workshops and written narratives were provided by the expert panel in collaboration with the UMGLJV staff and represent the suite of primary considerations currently used by the JV management board and staff to evaluate proposals (Table 2).

The expert panel identified performance measures that were consistent with currently used methods by each JV. Performance measures may be natural, proxy, or constructed (Keeney and Gregory 2005). Natural measures are those that follow directly from the objective itself (i.e., dollars for financial impacts, hectares for bird habitat, etc.). Proxy measures use scales that do not directly measure the objective of concern, but rather a metric that is correlated with the objective of concern; proxy measures are less informative because they indirectly indicate achievement of an objective. For example, the gross domestic product is a proxy measure for quality of life and the standard of living. Constructed attributes can be developed when no natural measure exists (Gregory et al. 2012). For example, the Richter scale is a constructed measure for earthquake intensity (Gregory et al. 2012). Flexibility in selection of performance measures is critical, as data, time, and resources between the two JVs differed. We used natural, proxy, and constructed performance measures. Where possible, and where data were available, we used density, hectares, and dollars to estimate consequences for objectives that addressed avian populations, habitat, and costs of conservation, respectively. Proxy performance measures were used where spatial models have been developed to identify priority regions for breeding and nonbreeding waterfowl and waterbirds and by correlation represent areas of expected high abundances (Soulliere et al. 2017, 2018). Where natural or proxy measures were not available or conservation consequences were based on expert panel interpretation, we developed constructed measures. All our constructed performance measures were categorical, with levels, and descriptions of each level, developed by the expert panel. This method ensured that evaluations of constructed performance measures would result in congruent conclusions among end users. Identified performance measures were classified as natural, proxy, or constructed for each JV (Tables 1 and 2). All descriptions and calculations for performance metrics are documented in Text S1 (Supplemental Material) for the GCJV value model and Text S2 (Supplemental Material) for the UMGLJV value model.

Performance measures for each objective were normalized using value functions. Value functions were elicited through workshop sessions with the expert panel and stakeholders. Discussions consisted of presentations to the workshop group of various value function curves (i.e., monotonically increasing/decreasing, piecewise, and exponential curvilinear functions) for each of the objectives (subobjectives), letting the majority decide which function best reflected the stakeholder values and represented the change in satisfaction with a change in the performance metric of the objective (or subobjective). Value functions for all but two objectives were straight-line functions with minimum (0) and maximum (1) bounds equal to the minimum and maximum values of the objective-specific (or subobjective-specific) performance measure (additional details in Texts S1 and S2, Supplemental Material). In the GCJV case study, subobjective 2.5 (maximize match ratio) had a piecewise linear value function with three inflection points. The expert panel and regional stakeholders believed that once a match ratio threshold was reached, increasing matched dollars represented “wasted” match. Stakeholder satisfaction thus increased linearly from a match ratio of 1:1 to a match ratio of 1:2.5. Above a match ratio of 1:2.5, the value function decreased until a ratio of 1:5, above which the value function was equal to zero (Text S1, Supplemental Material). In the UMGLJV case study, objective 10.0 (maximize quality of science support and degree of communication with staff) had a piecewise linear value function. Level of support was assessed with four categories developed by the expert panel. Values 1–4 correspond to inflection points in the value function, where 1 was “JV objectives used improperly” (the worst outcome) and 4 was “JV objectives addressed properly following communication with JV experts” (the best outcome). The value function was not linear; it was concave, with diminishing returns as the amount of scientific support and degree of communication increased to the maximum level (Text S2, Supplemental Material).

Weights for each objective were determined through the ranking and rating procedure (Goodwin and Wright 2004). The expert panel and relevant stakeholders took part in a group elicitation exercise to determine weights for each objective of two JV value models. Objectives were assigned weights relative to other objectives. Weights were assigned first to objectives and then determined for subobjectives using the same ranking and rating process. Weights were confirmed and approved by the JV representatives for each value model. A formal elicitation process for weight determination was not performed with the JV management boards in each case study and therefore, weights do not necessarily represent the regional values of the JVs. Joint Ventures reserve the right to alter weights on the basis of future elicitation processes when running value models for alternative funding cycles. The flexibility to alter weights is built into the two value models (see Data S1 and S2, Supplemental Material)

Seven NAWCA proposals were submitted in the spring of 2017 from the GCJV geography. To maintain anonymity, proposals are referred to as GCJV Proposals 1–7. Regional NAWCA proposals are not published for public access but may be obtained by contacting the JV directly (B. Wilson, U.S. Fish and Wildlife Service, personal communication). These seven proposals served as our alternatives for the GCJV value model framework. Six NAWCA proposals were submitted in the spring of 2016 for the UMGLJV geography. Again, to preserve anonymity, these are referred to as UMGLJV Proposals 1–6. Regional NAWCA proposals are not published for public access but may be obtained by contacting the JV directly (G. Soulliere, U.S. Fish and Wildlife Service, personal communication). These six proposals served as our alternatives for the UMGLJV value model framework. Each proposal consisted of one to several tracts of land. Estimates of consequences, as assessed with the performance measures, were based on the combination from all tracts within the proposal. In the UMGLJV, proposal tracts that represented matched dollars received from previously completed projects (i.e., old match) were excluded from estimates of consequences as these costs were considered “sunk/nonrecoverable costs,” and have little bearing on net return of new NAWCA investments (Soulliere et al. 2017:79–82). The value model framework ranked proposals within each region relative to each other.

The case studies for the GCJV and the UMGLJV included 8 and 10 objectives, respectively (Tables 1 and 2). The weights (w_j) for objectives reflected the priority of decision makers in each JV (Tables 3 and 4). There were several important similarities and differences between the GCJV and UMGLJV frameworks. Both JVs addressed all seven NAWCA ranking criteria with objectives specific to their region (Figure 1). However, the regional objectives that matched each criterion differed. For example, NAWCA ranking criterion 1, importance of the project toward meeting JV bird habitat objectives and species priorities, was addressed with objective 1.0 in the GCJV model, which evaluated habitat benefits for four bird guilds. The evaluation by the GCJV Science Team was generally qualitative and assessed the degree to which each proposal addressed habitat objectives for each bird guild. Because this regional objective addressed habitat objectives and landscape-scale context, it also met NAWCA ranking criterion 3. Alternatively, the UMGLJV addressed NAWCA ranking criterion 1 with three separate objectives (objectives 1.0–3.0). Performance measures for maximizing abundance of waterfowl and waterbirds (subobjectives 1.1–1.2) and maximizing carrying capacity for nonbreeding waterfowl and waterbirds (subobjectives 2.1–2.2) were estimated from spatial data provided by Soulliere et al. (2017, 2018). Subobjectives 3.1–3.4 also addressed NAWCA ranking criterion 1 and were based on reported hectares of four different wetland types in each proposal (Table 2). Another example involved the different ways to address NAWCA ranking criterion 2, partnership involvement. In the GCJV framework, this criterion was addressed with subobjectives 2.1–2.5; the performance measures here reflected involvement of JV staff and other teams (three “Yes”/“No” measures) and partner's matching contributions (number of 10% matching partners and the match ratio). The UMGLJV, on the other hand, addressed NAWCA criterion 2 with a constructed performance measure, which was estimated using a matrix that accounts for partner familiarity (i.e., new vs. common/familiar partner) and funding level (objective 4.0; see Text S2, Supplemental Material). Both JVs addressed all NAWCA ranking criteria in their framework with regionally specific objectives and performance measures (Tables 1 and 2).

Weight distribution among regional objectives differed for the two JVs. In the GCJV, representatives determined that weights should be equal for all eight objectives if the proposals are from the fall funding cycle. In the spring funding cycle, however, probability of lost match (objective 3.0) was considered less important, as there is another opportunity in the fall for funding in that same fiscal year. The weight for probability of lost match in the spring cycle was assigned one-half the weight assigned to this objective in the fall funding cycle; the remaining seven objectives in spring were weighted equally ([1 − weight of objective 3.0]/7). The spring funding cycle weighting scheme was adopted in this case study, as GCJV proposals 1–7 were submitted in the spring 2017 funding cycle (Table 3). The GCJV thus assigned most (93.7%) of the weight equally among seven of the eight objectives (i.e., 13.4% to each) and only 6.3% to probability of lost match. The objective weights were then divided equally among any subobjectives in most cases (Table 3). However, subobjectives to minimize cost per hectare for match (objective 8.1) and grant projects (objective 8.2) did not receive equal weight. The subobjective for grant projects was considered twice as important as the subobjective for match projects, and therefore received two-thirds of the objective weight.

The UMGLJV used weights largely congruent with current JV practices in modeling spatial prioritization to achieve biological and social objectives (Soulliere et al. 2017, 2018), but with the addition of the process objectives to maximize the quality of NAWCA grant proposals. For the UMGLJV, weight distribution was not equal among objectives (Table 4). Biological objectives related to breeding abundance, nonbreeding habitat carrying capacity, and priority wetland types received 40% of the weight; social objectives related to resource users and ecological services received 30% of the weight; and cost-effectiveness and process objectives received 30% of the weight (Table 4). The weight for each objective was divided equally among any subobjectives. The UMGLJV did not adopt a weighting scheme that differed between fall and spring funding cycles.

Contribution of each objective to the overall conservation benefit score differed among proposals. In the GCJV framework, Proposals 1 and 2, which ranked as the best (first and second, respectively), included contributions (conservation benefit) from seven objectives and showed the most uniform distribution of benefits among most objectives, although these proposals did not provide much value in terms of tenure of conservation benefits (objective 5.0; Figure 4) and did not provide any value from the probability of lost match objective (objective 3.0; Figure 4). In the UMGLJV framework, Proposal 3, which ranked as the best, did not have a uniform distribution of benefits among objectives (Figure 5). In fact, Proposal 3 showed the most uneven distribution of contributions from objectives to the total conservation benefits score. Most of the conservation benefit score for Proposal 3 in the UMGLJV came from objective 9.0, maximize benefits to people. This reflected the core values of the JV in this case study, as the total weight assigned to objective 9.0 was large (30%; 10% for each of three subobjectives). Proposal 6 in the UMGLJV had the most uniform distribution of benefits by objective but ranked as fifth best; it did not provide as many benefits to people and did not have any conservation benefits from favorable wetland/upland configuration (objective 5.0; Figure 5) or from carrying capacity for nonbreeding focal species (objective 2.0; Figure 5).

In the GCJV framework, results from the value model completely corresponded with rank order assigned by the GCJV management board in the spring 2017 funding cycle (Table 5). Proposals 1, 2, and 3 were ranked first, second, and third, respectively. Proposal 4 was ranked seventh. The UMGLJV results also showed similarity in proposal ranks with actual JV management board ranks (Table 6). Proposals 4 and 6 were ranked fourth and fifth, respectively, by both the value model and the JV management board in the spring 2016 funding cycle. However, notable differences in rank order occurred for Proposals 2 and 3. Proposal 2 was ranked sixth by the value model and second by the JV management board. Alternatively, Proposal 3 was ranked first by the value model and sixth by the JV management board.

In general, sensitivity analysis showed that proposal rank order in each JV was highly sensitive to shifts in objective weights. Of 18 sensitivity analyses (8 objectives in GCJV and 10 objectives in UMGLJV), only 5 objectives (4 in the GCJV framework and 1 in the UMGLJV framework) were robust to changes in weight (see Texts S1, S2, Data S1, S2, Supplemental Material) and did not affect the rank shift of the best proposal (i.e., proposal that ranked 1). Sensitivity analysis for the UMGLJV was especially telling, where the slightest shifts on objective weights had effects on proposal rank order (see Text S2 and Data S2, Supplemental Material).

Our primary results include two Microsoft Excel-based frameworks, one for each JV, with worksheets for the derivation of performance measures for each objective (see Data S1 and S2, Supplemental Material). The models are coded to run the example frameworks for the seven GCJV and six UMGLJV proposals, respectively. The models serve as templates and may be used as tools for the GCJV and the UMGLJV to rank regional NAWCA proposals in future funding cycles and may be adapted for the same purpose by other JVs. Additionally, Texts S1 and S2 (Supplemental Material) serve two purposes: as documentation for the development of the GCJV and UMGLJV value models and roadmaps for the construction of region-specific value models in other JVs.

We provide a general framework to identify region-specific priorities of a JV and a linear additive value model to rank NAWCA proposals, which could facilitate communication between management boards and the Council. In our case studies, each value model reflected the specific concerns of the JV. For example, each JV addressed the NAWCA criterion “importance of the project toward meeting JV habitat objectives and species priorities,” but the objectives identified by the JVs were tailored to their specific concerns. The GCJV objectives focused on habitat types for four bird guilds because the GCJV has prioritized these guilds and established guild-specific working groups. On the other hand, the UMGLJV addressed the same NAWCA criterion by defining objectives that focused on habitat for breeding and nonbreeding waterfowl and waterbirds. Whereas the GCJV used ranks submitted by guild-specific Science Teams, the UMGLJV estimated habitat density according to spatial models developed by Soulliere et al. (2017, 2018). Approaches for establishing population abundance objectives and their use in conservation planning generally differ among regions (Petrie et al. 2011), as bird habitat and foraging needs vary with the annual cycle. The GCJV is critical to waterfowl and many other birds during the nonbreeding period given its location in the southern United States. Located in the northern United States, the UMGLJV has a greater focus on the breeding and migration periods. The value model approach allowed each JV to incorporate region-specific concerns and make best use of relevant data (see Texts S1 and S2, Supplemental Material). Additionally, results of analytical models, such as those presented herein, increase transparency of prioritizations and allow greater efficiency in communication.

Our proposal ranking results for the GCJV value model were the same as those ranks provided by their management board. It is likely that these results indicate that the current JV management board model mirrors the input we were provided by the JV representatives in the development of the case study. The participation of the GCJV coordinator may have contributed to a better understanding of regional objectives and subjective valuation of those objectives during the spring funding cycles. Therefore, we were able to recreate a linear additive value model that closely resembles the current mental models used by the GCJV Management Board to rank NAWCA proposals.

There were notable differences in rank order of some proposals in the UMGLJV value model compared with ranks assigned by their management board. These discrepancies illustrate the difference between qualitative ranks from informal processes and ranks from value models (i.e., a structured decision analytic approach with spatial data, more formal elicitation, and a linear additive model for multiple objectives). There are several possible reasons for the differences between model-based ranks and ranks assigned by the management board. Decision-making informed by expert opinion is common in environmental management and policy decisions (Krueger et al. 2012; Martin et al. 2012), but may introduce bias and uncertainty when the decision context is complex and elicitations are informal or unstructured (O'Hagan et al. 2006; Speirs-Bridge et al. 2010). Cognitive biases occur because of a failure to adequately process, aggregate, or integrate relevant information due to limitations on human processing ability (Wilson 1994; McBride and Burgman 2012). As more spatial and empirical data become available, it becomes more difficult for subject matter experts to evaluate expected consequences with mental models. Analytical models, which decompose objectives and integrate consequences through quantitative modeling, may help overcome inherent cognitive biases in heuristic approaches such as “roundtable discussions.” Although expert judgment was critical throughout the development of each JV case study, we used a structured approach that is transparent and guards against cognitive biases. Additionally, although all our JV coauthors have long tenures of experience with their respective JVs, these coauthors were limited in their ability to know for certain the values and relative weights used by their respective management boards, especially as engagement of individual board members evolves with each successive grant cycle. Finally, it is possible that our UMGLJV proposal ranking framework included objectives that may have been considered only implicitly by the management board ranking processes. For example, partnership involvement may have been considered informally during the ranking process by the management board, but in our linear additive value model, we assigned quantitative values to measuring success toward this objective, increasing consistency across individual evaluators. Similarly, tenure of conservation benefits was considered by the JV management board in an ad hoc process, whereas the linear additive model explicitly calculated a value for this objective (Texts S1 and S2, Supplemental Material).

Several improvements in performance measures used in the case studies may be possible in future applications of the models. The constructed scales for habitat area (objective 1.0) used by the GCJV Science Team could be replaced with natural performance measures to increase clarity and objectivity. In this case, rank-based performance measures for amount of habitat provided (subobjectives 1.1–1.4) could be replaced with hectares of bird habitat (a natural measure) for the specific guilds included in these subobjectives. For example, where data are available, habitat objectives could be parsed into breeding habitat, brood-rearing habitat, and wintering habitat for waterfowl, including special emphasis on habitat for priority species such as mottled duck Anas fulvigula if desired. Additionally, special emphasis could be added to spatial components of the subobjectives by disentangling them from the subjective narrative used to rank proposals. For example, if contiguous blocks of habitat (e.g., > 4,047 ha) are minimum requirements for species such as seaside sparrow Ammospiza maritima, this can be used to set thresholds in a value function to reflect preference for large tracts. In the UMGLJV value model, threat of development in and around urban areas (objective 7.0) is measured using a constructed scale (i.e., high, medium, low). This subjective judgment can depend on the end user's personal expertise. Transitioning from the potentially opaque measures used by the GCJV for habitat objectives and UMGLJV for threat of development to natural measures such as hectares of selected habitat types and rate of habitat conversion, respectively, could increase transparency and repeatability of the value model. Replacing the constructed categorical measures (i.e., high, medium, low) for social objectives (i.e., bird-watching and waterfowl-hunting opportunity) with empirical data such as hunting days afield or bird-watching day trips in the county could also improve model repeatability and transparency (Pickard et al. 2015; Krainyk et al. 2021).

Sensitivity analysis of both JV models showed that proposal rank order was affected by changes in weights on objectives. We did not perform a formal weight elicitation because of time constraints and our inability to work directly with the JV management boards. Analytical hierarchy process is the most widely used technique for criteria weighting in multicriteria decision analysis processes (Saaty 2008) and can be integrated with Monte Carlo simulations to deal with uncertainty in objective weighting (Feizizadeh et al. 2014). A critical weight analysis also could be useful in identifying the critical objective with respect to their weights (Triantaphyllou 2000). This procedure determines the minimum weight change on an objective to produce a rank shift; thus, the smaller the weight change needed to produce a shift, the more critical an objective was to determining rank order. A formal weight elicitation, such as analytical hierarchy process or swing weights (Saaty 2008; Gregory et al. 2012), and critical weight analysis are warranted in future applications of value models to prioritize NAWCA proposals.

We describe the structured decision-making PrOACT approach and a linear additive model for ranking proposals, present two case studies, and encourage other JVs to develop their own regional NAWCA proposal ranking performance measures and models. Keeney (2007) defines five desirable properties that pertain to the set of objectives included in a value model (complete, nonredundant, concise, specific, and understandable). Keeney and Gregory (2005) provide a set of sufficient properties for good performance measures. Structured decision-making tools are becoming indispensable in increasingly complicated resource allocation problems (Huang et al. 2011; Lyons 2020). Value models are a flexible tool that can be customized to serve the specific needs of regional decision makers. Yet, the structured decision-making framework still maintains a degree of consistency across all JV ranking processes, which reflect the seven critical NAWCA criteria.

The method we propose, using a structured decision-making process to create an analysis framework, could be appropriate for many JVs because it could make the prioritization process more transparent and easier to communicate to stakeholders and others, and perhaps save time and resources. Within JVs, project portfolios in NAWCA grant applications can vary considerably with respect to each of the seven NAWCA ranking criteria, which can make it difficult for board members to specify which criteria drove the ranking on any given grant proposal. Therefore, model results such as those presented in Figures 4 and 5 can help to disentangle the reasoning behind proposal rank order. It could be helpful for the Council to use the results of the structured decision-making process to identify more specifically which criteria drove the rank order of proposals within the JV and these models can aid staff in preparing a quantitative report describing ranking decisions, improving communication between the two sets of decision makers.

Please note: The Journal of Fish and Wildlife Management is not responsible for the content or functionality of any supplemental material. Queries should be directed to the corresponding author for the article.

Text S1. A value model and sensitivity analysis for ranking North American Wetland Conservation Act proposals submitted in the Gulf Coast Joint Venture during the 2017 funding cycle. We conducted analysis in 2018 and a detailed explanation for identifying the objectives (subobjectives), selecting performance measures, and developing value functions is included in the Gulf Coast Joint Venture value model.

Available: https://doi.org/10.3996/JFWM-21-089.S1 (1.040 MB PDF)

Text S2. A value model and sensitivity analysis for ranking North American Wetland Conservation Act proposals submitted in the Upper Mississippi/Great Lakes Joint Venture during the 2016 funding cycle. We conducted analysis in 2018 and a detailed explanation for identifying the objectives (subobjectives), selecting performance measures, and developing value functions is included in the Upper Mississippi/Great Lakes Joint Venture value model.

Available: https://doi.org/10.3996/JFWM-21-089.S2 (806 KB PDF)

Data S1. Gulf Coast Joint Venture value model. Microsoft Excel-based user interface with inputs for proposal parameters to run the ranking procedure for annual granting decisions. The model produces a proposal rank order for the 2017 funding cycle in addition to supporting figures, which make communication between Joint Ventures and the North American Wetland Conservation Act Council transparent and consistent.

Available: https://doi.org/10.3996/JFWM-21-089.S3 (1.083 MB XLSM)

Data S2. Upper Mississippi/Great Lakes Joint Venture value model. Microsoft Excel-based user interface with inputs for proposal parameters to run the ranking procedure for annual granting decisions. The model produces a proposal rank order for the 2016 funding cycle in addition to supporting figures, which make communication between Joint Ventures and the North American Wetland Conservation Act Council transparent and consistent.

Available: https://doi.org/10.3996/JFWM-21-089.S4 (6.671 MB XLSM)

Reference S1. Cowardin LM, Carter V, Golet FC, LaRoe E. 1979. Classification of wetlands and deepwater habitats of the United States. Washington D.C.: U.S. Fish and Wildlife Service.

Available: https://doi.org/10.3996/JFWM-21-089.S5 (17.389 MB PDF)

Reference S2. Dahl TE. 1990. Wetlands: losses in the United States 1780s to 1980s. Washington, D.C.: U. S. Department of the Interior, U.S. Fish and Wildlife Service.

Available: https://doi.org/10.3996/JFWM-21-089.S6 (2.381 MB PDF)

Reference S3. Petrie M, Brasher MG, Soulliere GJ, Tirpak TM, Pool DB, Reker RR. 2011. Guidelines for establishing Joint Venture waterfowl population abundance objectives. North American Waterfowl Management Plan Science Support Team Report 2011-01. Washington, D.C.: U.S. Fish and Wildlife Service.

Available: https://doi.org/10.3996/JFWM-21-089.S7 (343 KB PDF)

Reference S4. Soulliere GJ, Al-Saffar MA, Coluccy JM, Gates RJ, Hagy HM, Simpson JW, Straub JN, Pierce RL, Eichholz MW, Luukkonen DR. 2017. Upper Mississippi River and Great Lakes Region Joint Venture Waterfowl Habitat Conservation Strategy—2017 Revision. Bloomington, Minnesota: U.S. Fish and Wildlife Service.

Available: https://doi.org/10.3996/JFWM-21-089.S8 (6.791 MB PDF)

Reference S5. Soulliere GJ, Al-Saffar MA, Pierce RL, Monfils MJ, Wires LR, Loges BW, Shirkey BT, Miller NS, Schultheis RD, Nelson FA, Sidie-Slettedahl AM, Holm DJ. 2018. Upper Mississippi River and Great Lakes Region Joint Venture Waterbird Habitat Conservation Strategy—2018 Revision. Bloomington, Minnesota: U.S. Fish and Wildlife Service.

Available: https://doi.org/10.3996/JFWM-21-089.S9 (10.978 MB PDF)

Reference S6. Soulliere GJ, Potter BA, Coluccy JM, Gatti RC, Roy CL, Luukkonen DR, Brown PW, Eichholz MW. 2007. Upper Mississippi River and Great Lakes Region Joint Venture Waterfowl Habitat Conservation Strategy. Fort Snelling, Minnesota: U.S. Fish and Wildlife Service.

Available: https://doi.org/10.3996/JFWM-21-089.S10 (5.038 MB PDF)

Reference S7.[USFWS] U.S. Fish and Wildlife Service. 2022. North American Wetlands Conservation Act United States Standard Grant 2022 Proposal Instructions. Falls Church, Virginia: U.S. Fish and Wildlife Service.

Available: https://doi.org/10.3996/JFWM-21-089.S11 (2.073 MB PDF)

This study benefited greatly from the assistance and commitment of David Howerter, Joseph Fuller, Mark Petrie, Ken Kriese, Justyn Foth, Mitch Hartley, and David Gordon. The authors also thank Anne Mini, the Associate Editor, and two anonymous reviewers for helpful reviews that improved the manuscript.

Any use of trade, product, website, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the U.S. Government.