A simulation study of trend detection methods for IEA

class: right, middle, my-title, title-slide

# A simulation study of trend detection methods for IEA
## WGNARS 2019
### Sean Hardison, Charles T. Perretti, Geret S. DePiper, Andrew Beet Integrated Statistics & Northeast Fisheries Science Center

---

class: top, left

**Trend Detection in Integrated Ecosystem Assessment**

* Assessing trends in time series is a key component of the IEA approach

* As analysts, we want to identify the limitations of these methods to ensure sound assessments of risk and indicator change

* Key statistical assumptions in trend analysis are often not properly accounted for

---
**Trend Detection in Integrated Ecosystem Assessment**

* Study question: What are the limitations of trend detection for IEA?

* Commonalities in IEA time series:
 * Short (often < 30 years)
 * Variety of error structures

* Time series assessed with single method

<img src="EDAB_images/indicator_vis.png" width="100%" style="display: block; margin: auto;" />
.image-cap2[
Frequency of estimated slopes (absolute values) (A), autocorrelation strengths (B), and time series residual variances (C) in 124 time series considered for inclusion in the 2017 State of the Ecosystem report. 
]

---
**AR(1) Simulation Methods**

1. Simulations:
 * Series of varying lengths (N = 10, 20, 30)
 * AR1 process strengths (&rho;1 = 0, 0.43, 0.9)
 * Trend strengths (&alpha;1 = 0, 0.026, 0.051, and 0.147)

2. Test for trend:
 * GLS model selection 
 * Mann-Kendall test1
 * Mann-Kendall test with trend-free pre-whitening2
 
<img src="EDAB_images/indicator_vis.png" width="100%" style="display: block; margin: auto;" />
 
.footnote[

[1] Mann, H. B. 1945b. Nonparametric tests against trend. Econometrica: Journal of the Econometric Society: 245–259. JSTOR.

[2] Yue, S., and Wang, C. Y. 2002. Applicability of prewhitening to eliminate the influence of serial correlation on the Mann-Kendall test. Water Resources Research, 38: 4–1–4–7. http://doi.wiley.com/10.1029/2001WR000861.

]

---
**AR(1) Simulation Results: Significance Testing**

---
**Results: Extending N under Strong AR(1)**

<img src="EDAB_images/iea_ar1_extended.png" width="100%" style="display: block; margin: auto;" />
.image-cap[
The dashed red line shows the nominal rejection rate of 0.05.
]

.box-highlight[
**Parameters**
* N = 50 - 650
* Strong AR(1) (&rho;1 = 0.9)
* No trend
]
---
**Results: The role of series variance in significance testing**

.box-highlight[
**Parameters**
* N = 30
* σ2 = 0.3 ("low") and σ2 = 0.8 ("high") 
* Weak trend (&beta; = 0.026)

]

---
**Results: Estimating slope from autocorrelated data**

---
**Results: Testing for trend in data simulated from AR(2) model**

* How do tests for trend handle data with AR(2) error?
 
.box-highlight[
**Parameters**
* N = 10, 20, 30
* AR(2) process strength: &rho;1 = 0.43, &rho;2 = 0.2
* &alpha;1 = 0, 0.026, 0.051, and 0.147
]

---
**Summary**
.summary-highlight[
Testing for trend in short time series results in high incidence of false positive and false negative findings
  * Both become more prevalent when autocorrelated error is present in data 
  
Tests designed to account for AR error structure are biased when N is small

Estimates of trend strength often mischaracterize true trend when N is small or AR error is present

Which test performed best?
  * GLS-MS; although difference between tests are small (and all performed poorly)
  * Benefit of GLS-MS is ability to characterize uncertainty in estimate from probability distribution
]

---
**Lessons?**
* Applying tests for trend broadly without considering underlying error structures will likely result in assigning spurious trends or missing key patterns

* If hypothesis testing is to be used to identify trends for IEA, a more nuanced approach should be developed to account for characteristics that could bias tests
 * R package with flexible diagnostic functions currently in development to address shortcomings
 
<img src="EDAB_images/noaa-iea-wgnars.png" width="100%" style="display: block; margin: auto;" />