Uncertainty in management strategy evaluation (MSE):

class: right, middle, my-title, title-slide

# Uncertainty in management strategy evaluation (MSE):
## Case study of the New England Fishery Management Council Herring MSE
### Sarah Gaichas Ecosystem Dynamics and Assessment Northeast Fisheries Science Center

---

class: top, left

# Deep Uncertainty and Robust Decision Making (RDM)

.pull-left[
- Decision making under deep uncertainty (DMDU) needed when disagreement on:

- model of system relating action to consequences
    - probability distributions for model parameters
    - valuation of different outcomes
    
- Essential features of DMDU

- iterate analysis and deliberation
    - many computer simulations of assumptions and scenarios
    - identify scenarios with really bad outcomes
    - plan to adapt as things change
]
.pull-right[

![:img DMDU def slide, 80%](EDAB_images/DMDUneeded.png)
![:img KeyIdeas slide, 80%](EDAB_images/DMDUkeyideas.png)
]
---
## RDM and Management Strategy Evaluation (MSE)

.pull-left[
- Robust Decision Making (RDM) used when
    - future uncertainty difficult to characterize
    - diverse stakeholders with different priorities
    
- RDM is "deliberation with analysis"
    - model implications of different assumptions
    - support stakeholder deliberations
    - develop robust, adaptive plans
    
.center[![:img RDMslide, 80%](EDAB_images/RDMoutline.png)]
]
.pull-right[

- Management Strategy Evaluation (MSE)

>... involves assessing the consequences of a range of management strategies or options and presenting the results in a way which lays bare the tradeoffs in performance across a range of management objectives. <a name=cite-hancock_management_1994></a>([Smith, 1994](#bib-hancock_management_1994))

>...is a flexible approach that allows for a balance between
multiple objectives and identifies harvest strategies robust to various types of uncertainty. Simulation can accommodate more realistic modeling of the fishery than dynamic optimization, as well as more practically implementable policies. <a name=cite-holland_uncertainty_2009></a>([Holland and Herrera, 2009](https://www.jstor.org/stable/42629656))
]

???

---
## Fishery management in the US: participatory, with wicked problems

Eight regional Fishery Management Councils establish plans for sustainable management of stocks within their jurisdictions. All are governed by the same law, but tailor management to their regional stakeholder needs.  
.center[
![:img US map highlighting regions for each fishery management council, 75%](EDAB_images/USCouncils.png)
]

More information: http://www.fisherycouncils.org/  
https://www.fisheries.noaa.gov/topic/laws-policies#magnuson-stevens-act

---
background-image: url("EDAB_images/allcomponents.png")
background-size: 800px
background-position: right bottom

## Are any Atlantic herring harvest control rules good for both fisheries and predators?
.pull-left-30[
**Harvest control rules** are:
- plans for changing fishing based on stock status
- pre-determined
    
*"Which harvest control rules best consider herring's role as forage?"*
- DESIGN a harvest control rule (HCR):
    - balancing fishing benefits and ecological services
    - addressing diverse stakeholder interests
- TRANSPARENTLY within management time frame!
]

---
background-image: url("EDAB_images/MSECSIRO.png")
background-size: 800px
background-position: right

## What is Management Strategy Evaluation?
.pull-left-30[
- Process to develop fishery management procedures

- First used in S. Africa, Australia, and at International Whaling Commission  late 1980s - early 1990s

.contrib[
>Under this approach, management advice is based on a fully specified set of rules that have been tested in simulations of a wide variety of scenarios that specifically take uncertainty into account. The full procedure includes specifications for the data to be collected and how those data are to be used to provide management advice, in a manner that incorporates a feedback mechanism. <a name=cite-punt_developing_2007></a>([Punt and Donovan, 2007](https://academic.oup.com/icesjms/article/64/4/603/641398))
]
]

---
background-image: url("EDAB_images/MgtProcess.png")
background-size: 800px
background-position: right

## The Dream and The Reality
.pull-left-30[
*First MSE within US Council* 
<a name=cite-feeney_integrating_2018></a>([Feeney, Boelke, Deroba, Gaichas, Irwin, and Lee, 2019](http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0125))
Scope: annual stockwide HCR
Open stakeholder meetings (2)
- ID objectives, uncertainties
- ID acceptable performance
- more diverse, interactive than "normal" process

Uncertainties identified
- herring mortality (M)
- environmental effects on herring 
- predator response to herring abundance
- assessment uncertainty

]

---
## Operating models and uncertainties 
.pull-left-70[
.table[

|Operating Model Name|Herring Productivity|Herring Growth|Assessment Bias|
|:---------------------|:---------------------|:-----------------|:-----------------|
| LowFastBiased        | Low: high M, low h (0.44)  | 1976-1985: fast | 60% overestimate |
| LowSlowBiased        | Low: high M, low h (0.44)  | 2005-2014: slow | 60% overestimate |
| LowFastCorrect       | Low: high M, low h (0.44)  | 1976-1985: fast | None             |
| LowSlowCorrect       | Low: high M, low h (0.44)  | 2005-2014: slow | None             |
| HighFastBiased       | High: low M, high h (0.79) | 1976-1985: fast | 60% overestimate |
| HighSlowBiased       | High: low M, high h (0.79) | 2005-2014: slow | 60% overestimate |
| HighFastCorrect      | High: low M, high h (0.79) | 1976-1985: fast | None             |
| HighSlowCorrect      | High: low M, high h (0.79) | 2005-2014: slow | None             |
] 
&nbsp;
&nbsp;

] 
.pull-right-30[
<a name=cite-deroba_dream_2018></a>([Deroba, Gaichas, Lee, Feeney, Boelke, and Irwin, 2019](http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0128))
]

Implementation error was included as year-specific lognormal random deviations: 
`\(F_{a,y}=\bar{F}_yS_ae^{\varepsilon_{\theta,y}-\frac{\sigma_\theta^2}{2}} \;\;\;  \varepsilon_{\theta} \sim N(0,\sigma_\theta^2)\)`

Assessment error was modeled similarly, with first-order autocorrelation and an optional bias term `\(\rho\)`:
`\(\widehat{N}_{a,y}=[N_{a,y}(\rho+1)]e^{\varepsilon_{\phi,y}-\frac{\sigma_\phi^2}{2}}\;\;\;\; \varepsilon_{\phi,y}=\vartheta\varepsilon_{\phi,y-1}+\sqrt{1-\vartheta^2}\tau_y \;\;\;\;  \tau \sim N(0,\sigma_\phi^2)\)`

---
background-image: url("EDAB_images/OMdesign.png")
background-size: 750px
background-position: right

## Linked models matching stakeholder-identified objectives
.pull-left-30[
The Dream:1
Convert the effects of control rules on 4 user groups to dollars:
.contrib[
1. Users of landed herring (Demand)
 - Lobster industry, aquariums
1. Herring harvesters (Supply)
1. Direct users of herring in the ocean (not people)
 - Terns and Whales
 - Striped Bass, Dogfish
1. Indirect users of herring in the ocean (people, Derived Demand)
 - Bird- and whale-watchers
 - Recreational and Commercial Fishing
]
The Reality
.contrib[
- 8 herring operating models linked to simple predator and economic models, developed in parallel
- limited range of predator response
- limited economic effects, directed fishery only
]
([Deroba, Gaichas, Lee, et al., 2019](http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0128))
]
.footnote[
1 Credit: [Min-Yang Lee](https://s3.amazonaws.com/nefmc.org/13.Economic-Models_Lee.pdf)
]
---
background-image: url("EDAB_images/herrtopreds.png")
background-size: 640px
background-position: left

## Predators: deterministic population models and herring-predator links
.pull-left[]
.pull-right[.right[
![:img ternpoptrend, 100%](EDAB_images/ternpoptrend.png)
]
.table[
Time constraints forced:
- selection of predators with previous modeling and readily available data
- selection of single strongest herring-predator relationship
- models ignoring high variance in prey-predator relationships 
]
]
---
background-image: url("EDAB_images/herrtopreds_results.png")
background-size: 640px
background-position: left center
## Results summary
Three HCR types were rejected at the second stakeholder meeting for poor fishery and predator performance.
.pull-left[]
.pull-right[.right[
<img src="20200716_MSEUncertainty_Gaichas_files/figure-html/YAY-1.png" width="85%" />
]]

---
## Managing tradeoffs under uncertainty: What control rules give us 90% of everything we want?

- Tern productivity at 1.0 or above more than 90% of the time
- Herring biomass more than 90% of SSBmsy
- Fishery yield more than 90% of MSY
&nbsp;
- AND fishery closures (F=0) less than 1% of the time (plot on right).

.pull-left[
<img src="20200716_MSEUncertainty_Gaichas_files/figure-html/unnamed-chunk-2-1.png" width="504" />
]

.pull-right[
<img src="20200716_MSEUncertainty_Gaichas_files/figure-html/unnamed-chunk-3-1.png" width="504" />
]

---
## Lessons: testing strategies is key, but uncertainty still difficult to convey
.pull-left-30[
Complex food web, generalist predators
- Herring is one of several important prey
- Assess multiple prey together for stronger effects on predator productivity
- Tern/Tuna/Groundfish/Mammal productivity also affected by predators, weather, and other uncertain factors
- Still showed which herring control rules were poor
- Managers selected a harvest control rule considering a wide range of factors!
]
.pull-right-70[
![NEUSfw](EDAB_images/LinkNEUSfoodweb.png)
]

---
background-image: url("EDAB_images/salmonscenarios.png")
background-size: 700px
background-position: right
## MSE and uncertainty: incorporate scenario planning?
.pull-left-40[
- Standalone process with stakeholders
    - salmon example
    - right whale example
    - climate example just starting
    - many others!

- Scenarios could specify a set of MSE operating models
    - fewer operating models spanning more uncertainty
    - easier to describe 
    - limits results dimensionality
    
- Climate scenarios for a region could be used in many MSEs
]

.footnote[
Source: [Salmon Climate Scenario Planning Pilot Report](https://www.greateratlantic.fisheries.noaa.gov/policyseries/index.php/GARPS/article/view/15/14)
]
---
background-image: url("EDAB_images/boxscatterbag2.png")
background-size: 800px
background-position: center
## MSE and uncertainty: visualizing many dimensions? Advice from DMDU and RDM?
([Feeney, Boelke, Deroba, et al., 2019](http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0125))
--

![:img plots developed later, 90%](EDAB_images/radardecision.png)
---
background-image: url("EDAB_images/climatescenarioarrow.png")
background-size: 800px
background-position: right
# Conclusions

- NOAA Fisheries supports MSE
    - Dedicated FTE at each Science Center
    - National working group
    
- Climate and EBFM Roadmaps include
    - Scenario planning
    - MSE
    
- Enhance with DMDU

.footnote[
Source: [Salmon Climate Scenario Planning Pilot Report](https://www.greateratlantic.fisheries.noaa.gov/policyseries/index.php/GARPS/article/view/15/14)
]

---
## References

<a name=bib-deroba_dream_2018></a>[Deroba, J. J, S. K. Gaichas, M. Lee,
et al.](#cite-deroba_dream_2018) (2019). "The dream and the reality:
meeting decision-making time frames while incorporating ecosystem and
economic models into management strategy evaluation". In: _Canadian
Journal of Fisheries and Aquatic Sciences_. ISSN: 0706-652X. DOI:
[10.1139/cjfas-2018-0128](https://doi.org/10.1139%2Fcjfas-2018-0128).
URL:
[http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0128](http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0128)
(visited on Jul. 20, 2018).

<a name=bib-feeney_integrating_2018></a>[Feeney, R. G, D. V. Boelke, J.
J. Deroba, et al.](#cite-feeney_integrating_2018) (2019). "Integrating
Management Strategy Evaluation into fisheries management: advancing
best practices for stakeholder inclusion based on an MSE for Northeast
U.S. Atlantic herring". In: _Canadian Journal of Fisheries and Aquatic
Sciences_. ISSN: 0706-652X. DOI:
[10.1139/cjfas-2018-0125](https://doi.org/10.1139%2Fcjfas-2018-0125).
URL:
[http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0125](http://www.nrcresearchpress.com/doi/10.1139/cjfas-2018-0125)
(visited on Nov. 09, 2018).

<a name=bib-holland_uncertainty_2009></a>[Holland, D. S. and G. E.
Herrera](#cite-holland_uncertainty_2009) (2009). "Uncertainty in the
Management of Fisheries: Contradictory Implications and a New
Approach". In: _Marine Resource Economics_ 24.3. Publisher: [MRE
Foundation, Inc, The University of Chicago Press], pp. 289-299. ISSN:
0738-1360. URL:
[https://www.jstor.org/stable/42629656](https://www.jstor.org/stable/42629656)
(visited on Jul. 14, 2020).

<a name=bib-punt_developing_2007></a>[Punt, A. E. and G. P.
Donovan](#cite-punt_developing_2007) (2007). "Developing management
procedures that are robust to uncertainty: lessons from the
International Whaling Commission". En. In: _ICES Journal of Marine
Science_ 64.4. Publisher: Oxford Academic, pp. 603-612. ISSN:
1054-3139. DOI:
[10.1093/icesjms/fsm035](https://doi.org/10.1093%2Ficesjms%2Ffsm035).
URL:
[https://academic.oup.com/icesjms/article/64/4/603/641398](https://academic.oup.com/icesjms/article/64/4/603/641398)
(visited on Jul. 14, 2020).

<a name=bib-hancock_management_1994></a>[Smith, A. D.
M.](#cite-hancock_management_1994) (1994). "Management strategy
evaluation – the light on the hill". In: _Population dynamics for
fisheries management_. Ed. by D. Hancock. Perth: Australian Society for
Fish Biology, pp. 249-253.

---
## Additional resources

* [New England Atlantic herring management](https://www.nefmc.org/management-plans/herring)

* [New England MSE debrief](https://s3.amazonaws.com/nefmc.org/3_Herring-MSE-debrief-final-report.pdf)

* [MSE video](https://www.youtube.com/watch?v=5JG76VPnHS8)

* [Management procedures video](https://www.youtube.com/watch?v=V9QEG4R4-w0)

* Slides available at https://noaa-edab.github.io/presentations

* Contact: <Sarah.Gaichas@noaa.gov>

# Thank you!