+ - 0:00:00
Notes for current slide
Notes for next slide

atlantisom: simulated
ecosystem datasets for
model performance testing

NEMoW 5 Tool Demos

Christine Stawitz, Sarah Gaichas, Isaac Kaplan, and Patrick Lynch
Office of Science & Technology, Northeast and Northwest Fisheries Science Centers



Supported by Institute of Marine Research Norway,
and NOAA NMFS International Fellowship

1 / 22

We use models for a lot

NOAAlogo NOAA fisheries

2 / 22

We have a lot of models

ecomods

3 / 22

How do we know they are right?

  • Fits to historical data (hindcast)

  • Influence of data over time (retrospective diagnostics)

  • Keep as simple and focused as possible

  • Simulation testing

But What if

data are noisy?

we need to model complex interactions?

conditions change over time?

4 / 22

End-to-end ecosystem operating models as dataset generators

Atlantis modeling framework: Fulton et al. 2011, Fulton and Smith 2004

Norwegian-Barents Sea

Hansen et al. 2016, 2018

NOBA scale 90%

California Current

Marshall et al. 2017, Kaplan et al. 2017

CCAspatial scale 100%

Building on global change projections: Hodgson et al. 2018, Olsen et al. 2018

5 / 22

Why use Atlantis?

  • Mechanistic processes create internally consistent "truth"
  • Include cumulative effects of multiple processes:
    • Climate drivers
    • Species interactions
    • Spatial and seasonal variability
    • Fisheries
    • Oil spills, red tide, anything else Atlantis can do
  • Implemented for many ecosystems worldwide

Why generate datasets instead of simulating within Atlantis?

  • Not all analyses need closed loop
  • Faster!
  • Test many models or model configurations with the same dataset
  • Many dataset realizations from same "truth"; compare:
    • Different observation error and bias
    • Changing temporal and spatial survey coverage
    • Improved or degraded fishery observations
6 / 22

Climate impacts in the operating model

sardine

Hodgson, E. E., Kaplan, I. C., Marshall, K. N., Leonard, J., Essington, T. E., Busch, D. S., Fulton, E. A., et al. 2018. Consequences of spatially variable ocean acidification in the California Current: Lower pH drives strongest declines in benthic species in southern regions while greatest economic impacts occur in northern regions. Ecological Modelling, 383: 106–117.

Marshall, K. N., Kaplan, I. C., Hodgson, E. E., Hermann, A., Busch, D. S., McElhany, P., Essington, T. E., et al. 2017. Risks of ocean acidification in the California Current food web and fisheries: ecosystem model projections. Global Change Biology, 23: 1525–1539.

7 / 22

Climate + cumulative impacts in the operating model

guild

8 / 22

Specify uncertainty in assessment "data": What Would Poseidon Do?

What level of uncertainty do you want to carry through performance testing?

  1. Biological (Atlantis):

    1. appropriate sigma-R?

  2. Survey specification (atlantisom):

    1. timing and spatial coverage?

    2. which species are captured?

    3. species-specific survey efficiency ("q")?

    4. selectivity at age for each species?

  1. Survey uncertainty (atlantisom):

    1. additional observation error (survey cv for index)?

    2. effective sample size for biological samples?

  2. Fishery uncertainty (atlantisom):

    1. additional observation error (catch cv for total)?

    2. catch sampled for length/age in all areas?

    3. effective sample size for biological samples?

9 / 22

Get atlantisom https://github.com/r4atlantis/atlantisom

# install.packages("devtools")
devtools::install_github("r4atlantis/atlantisom")

atlantisom acknowlegements:

Atlantis Summit December 2015

atlantisom sgaichas$ git shortlog -sne
   203  kellijohnson <kellifayejohnson@gmail.com>
    65  sgaichas <sgaichas@gmail.com>
    47  Alexander Keth <alexander.keth@uni-hamburg.de>
    29  ChristineStawitz-NOAA <cstawitz@uw.edu>
    23  allanhicks <fishmanHicks@gmail.com>
    19  Sean Lucey <Sean.Lucey@NOAA.gov>
    13  hodgsone <emma.e.hodgson@gmail.com>
    11  gavinfay <gfay42@gmail.com>
     9  Kelli Johnson <kellifayejohnson@gmail.com>
     5  Emma Hodgson <ehodgson@Emmas-MacBook-Pro.local>
     4  christine.stawitz <christine.stawitz@noaa.gov>
     4  Christine Stawitz <cstawitz@uw.edu>
     3  unknown <rwildermuth@DT-322202-SMAST.UMDAR.umassd.edu>
     1  Gavin Fay <gfay42@gmail.com>
     1  sgaichas <sgaichas@users.noreply.github.com>
     1  thefaylab <gfay@umassd.edu>

summit

10 / 22

Make Atlantis output into assessment model input

Example atlantisom workflows:

  1. Get true biomass, abundance, age composition, length composition, weight at age, fishery catch, fishery catch at age, fishery length composition, and fishery weight age age for a "sardine-like species": https://sgaichas.github.io/poseidon-dev/FullSardineTruthEx.html

  2. Format these outputs and get other life history parameters for input into a stock assessment model (Stock Synthesis, using r4ss): https://sgaichas.github.io/poseidon-dev/CreateStockSynthesis.html

  3. Get true and observed input data, format inputs, and run the assessment model: https://sgaichas.github.io/poseidon-dev/SardinesHakeatlantisom2SStest.html

  4. In progress: compare assessment results with truth: https://sgaichas.github.io/poseidon-dev/SkillAssessInit.html

11 / 22

What can atlantisom do so far?

Survey census test NOBA

True length composition NOBA

Standard survey test CCA

Survey length composition CCA

12 / 22

A "sardine" assessment

Need: assessment model data inputs and life history parameters

(model based on actual Sardine assessment in Stock Synthesis 3)

Data:

  • survey biomass index
  • survey length composition
  • survey age composition (conditional catch at age)
  • fishery catch (tons)
  • fishery length composition
  • fishery age composition

Parameters:

  • natural mortality (from total mortality)
  • growth curve (from survey length at age)
  • maturity at age (true)
  • unfished recruitment and steepness (true)
  • weight-length curve (true)
13 / 22

A "sardine" assessment: setup

  • California Current Atlantis run with and without climate signal
  • Input data generated (e.g. sardine survey, below in green)
  • Parameters derived; simpler recruitment distribution

14 / 22

A "sardine" assessment: fits to data

surveyfit lenfit

caafit1 caafit2

15 / 22

A "sardine" assessment: skill? (proof of concept)

Biomass Fishing mortality

Recruitment

Key: True SS3 estimate

16 / 22

Full disclosure!

Still working on:

  1. Functions for older Atlantis codebases (i.e., CCA model)

    1. Splitting aggregate age groups into true ages

    2. Interpolating aggregate age groups weight at age for true ages

    3. Fishery catch weight by area

  2. Wrapper functions to generate data in fewer steps

  3. Automated skill assessment functions

  4. Inputs for models other than Stock Synthesis

workinprogress scale 100%

17 / 22

P.S. What else could we test?

xkcd_ensemble_model_2x.png

https://xkcd.com/1885/

18 / 22

External Resources

19 / 22

Questions?

Tusen Takk til Havforskningsinstituttet!

20 / 22

Extra slides

21 / 22

Test both estimation and (simple) MSE capability

22 / 22

We use models for a lot

NOAAlogo NOAA fisheries

2 / 22
Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k Go to previous slide
↓, →, Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow