MS Keyrun Data Needs

Sarah Gaichas

15 April, 2024

Decisions from meeting June 3 2020

Notes were taken on the KEYRUN Kanban board under the epic KEYRUN-4 (a password is required to access these sites, but current information is copied below).

Objective: Develop a common input dataset for use in all keyrun models

The purpose of the June 3 meeting was to describe the decisions and components required to develop the common input dataset for the project.

Epic description

(This text copied from KEYRUN-4 Description and lightly edited)

Below is a list of potential stories under this Epic with links to other Epics. Numbers indicate order tasks need to be done (priority) established at the June 3 2020 meeting.

TODO: Add each story. Add done condition for each story.

1. Document decisions on all stories below (maybe rmd) and document as we go for all datasets, Jira not searchable so establish decision document and documentation elsewhere (pull comment from done column? plugins may be available by license)

2. Establish temporal and spatial dimensions for data (years, seasons, areas).

define Georges Bank for survey and fisheries which will not match, could use habitat model

2. Establish species lists (Rpath KEYRUN-2 larger list than multispecies models KEYRUN-1 and KEYRUN-3)

3. Define environmental data needed for each model

4. Establish dataset standards (See e.g. condition work)

5. Biomass index data from surveys for all models:

6. Establish common definitions of fleets: applies to all catch data, discard data, and composition data.

7. Catch (landings) data for all models:

   Rpath in KEYRUN-2

   Hydra and MSCAA in KEYRUN-1

   All other tested models in KEYRUN-3

8. Discard data for all models, needs methods discussion/decision (check: Andy)

9. Length composition data from surveys and fisheries for Hydra in KEYRUN-1 (check: Andy)

10. Age composition data from surveys and fisheries for MSCAA in KEYRUN-1

11. Define which diet and or consumption data from surveys and fisheries for all models

12. Define which life history data from surveys and/or fisheries for all models (e.g. maturity, weight at age, maximum age for mortality estimation, etc)

Additional meeting notes

Progress on catch data:

Notes on mscatch and comlands. comlands in a package now. dataset from comland is input to mscatch which combines with observer data length samples, for each quarter/yr for species, gear, can assign length samples to each of those landings bins. done and tested. Discards not yet accounted for, difficult and needs decisions. mostly documented, decision process for filling blanks documented in log file. Canadian landings for GB may be needed.

there is a comdisc as well, computes discard to kept ratio. add to subtasks under discard story

Identify done criteria, meet them, then run by key stock assessment folks.

Progress

Stories have been added to the board as of 15 April, 2024:

## Warning: `includeHTML()` was provided a `path` that appears to be a complete HTML document.
## ✖ Path: NMFS_JIRA_2020-06-15T_0_11_28-0400.html
## ℹ Use `tags$iframe()` to include an HTML document. You can either ensure `path` is accessible in your app or document (see e.g. `shiny::addResourcePath()`) and pass the relative path to the `src` argument. Or you can read the contents of `path` and pass the contents to `srcdoc`.

NMFS JIRA

Displaying 11 issues at 15/Jun/20 10:11 AM.

Project	Key	Summary	Issue Type	Status	Priority	Resolution	Assignee	Reporter	Creator	Created	Last Viewed	Updated	Resolved	Affects Version/s	Fix Version/s	Component/s	Due Date	Votes	Watchers	Images	Original Estimate	Remaining Estimate	Time Spent	Work Ratio	Sub-Tasks	Linked Issues	Environment	Description	Security Level	Progress	Σ Progress	Σ Time Spent	Σ Remaining Estimate	Σ Original Estimate	Labels	Work Completed by	Reviewer1	Data Timeframe	NERO IT Documentation Project Due Date	Data Set List	Device Location	Purpose	Work Accepted by	Recipient Org.	Set Date Work Accepted	Submission Format	Submission Method	Submission Instructions	GARFO FDDC POC	NEFSC FDDC POC	RSA Project Code	NER CCB Regulatory Deadline	Number or Reviewers	RSA Project Title	Owner / Operator	RSA Hull Number	Vessel Name	Development	Schedule as Repeat Request?	Additional Information	Date of Request	Request Declined by	Date Needed by	Geographic Area	Set Date Work Verified	Set Date Request Declined	Sub-Task?	Work Verified by	Primary Stakeholder	Milestone Schedule	Project Overview	Cost Estimate	Resources Needed	Requested By	Constraints	Assumptions	Deploy Due	PICSAF-Text Field	Background	Timeframe	Business Objective	Project Sponsor	Executive Summary	Request Priority	Project Scope	IT Security Considerations	Project Objectives	Risks	JFP nFeed	NERO PRD HCD Due Date	Date Paperwork Submitted to OCIO	Labor Category	COR Group	Employee Last Day	Cell Phone	S&T HR Groups	Disposition	Company	EMAIL	Division	Date Work Completed	Assignment Group	NERO PRD HCD Role	NERO PRD HCD Assign	Multi Component	Reported By	Priority	Contractor Company	Scan Start Time	Scan Request Date	Account Types	Test Plan	Scan Stop Time	Security Awareness Training Course	Contract End Date	RSA Program	CAC Badge	SVN Link	NOAA Badge	NNF Data Applications	NNF FDDC Administrative Category	RSA Vessel Permit Number	Data Collection Methods	Data Types	Requesting Organizations	Expected Data Volume	SWI / ERT2 User Assigned To	Other Names Used	Date CAC Issued	Action Lead	Temporary Employment Other	Action Work Start Date	Country of Citizenship	Specifications?	PIFSC Report ID#	Data Access Constraints	Date Work Accepted	Date Work Suspended	Data Resource Name	Date Review Started	Date Work Request Cancelled	Procedure	Request Resolution	PIFSC DARTS ID#	Date of Last Action	Date Request Authorized	Date Request Declined	Developer1	Date Work Results Verified	Set Date Work Completed	Data Summarization Details	Data to Exclude	Data to Include	Date Progress Started	Data Fields Requested	Last Action	Customer Input Requested by	DR Last Action	Review Outcome Notes	ST6 Priority	Recipient	Authorizer	Date Work Projected To Start	FHII Vessel	Organizational Readiness	S&T Security Awareness Watchers	Alternatives	Federal Supervisor	Technical Readiness?	COR/Task Manager	Project Readiness Details	Epic Link	Requester	Sprint	Epic Status	Rank	COR/Task Manager	Capability Enhancement Details	Capability Enhancement Areas	ST Position Title	Project Dependencies	Termination Requested By	NSA System / Schema	NNF FDDC Summary	NSA Priority	Scheduled Start Time	NSA System / Schema Name	Background & Business Needs	Beneficiaries	NSA Labels	Sunflower Equipment Issued	NSA Rank	Scheduled Stop Time	Mandated Timeline	ST6 Web Scan Watchers	VOIP	Key Stakeholders	Mandated By	TDY	Public Relations Impact?	Work Location	Assignment Method	Failure Impact	VPN	Component	Existing functionality	Public Relations Impacts	Work Requirements	Network Account	NOAA Email	Position Sensitivity	Pitney-Bowes	Funding Sources	OCIO Dev Estimated Level of Effort	Release Specific Roles and Responsibilities, Assumptions, Constraints, Risks, IT Security Considerations	Resolution Notes	Key Stakeholders Old	Release Overview	Release Specific Requirements	Action Scope	Finger Print Status	OCIO QA/Test Estimated Level of Effort	Action Initiation Date	Employee Status	Action Active	Investigation Type	Action Priority	Hard Drive	Monitors	LDAP	NNF Priority Matrix Quadrant	IDI Trip ID	LOBSTAH-Year	NNF Final Decision	Resources	AIR Type	Cardholder	Submit Date	GARFO RPM Requested Action	Project Roles and Responsibilities	GARFO RPM Original Permit ID	Metadata Identifier	Organization	GARFO RPM Date Permit Reissued	Package Location	SEFSC Security Impacts	ASP Data Requirements	Impact	ASP Metadata Requirements	GARFO RPM Name of Requestor	Steward	Data Reviewer	SEFSC Other Impacts	Metadata Reviewer	SEFSC Change Type	Accounting Code	SEFSC Change Rollback Date/Time	Performance End Date	SEFSC Business/User Impacts	SEFSC Change Reason	SEFSC Change Rollback Details	SEFSC Software/Module Impacts	SEFSC Network Impacts	Environment	Personal Telephone Number	Security Awareness Training Completion Date	Federal Supervisor's Phone Number	Board Review Status	Board Review Commentary	Position Title	Environment - select	SEFSC Change End Date and Time	Position Sensitivity	SEFSC Service Interruption	Suffix	Key Stakeholders NPR	Business Need	ST6 Watchers	Implementation date	Personal Email Address	Contract Number	Notification plan	Peer Sign-off	Cascading Select	Change checklist	Previous DOC Assignments	Previous Federal/DOC Work Dates	Verification	Federal Supervisor	Integration	NERO MAST TIME Total	NERO MAST ACTUAL TIME	Backout plan	Workstation Re-Imaged Date	CD#	FHII Vessel Not Used	Risk plan	VoIP Removal Date	Acceptance plan	Departure Date	Reviewed by	Gender	Team Name	Team Members	Problem Analysis	Current State	Date Account Deactivation Form Submitted	Need	Date Employee Server Accounts Removed	Implementation Plan	Ramifications	Trust Wave	Date User Profile Backed Up	Future State	Implementation procedure	Pay.Gov	CD/Hard Drive Surrender Date	Date Request Completed	Prerequisites	Date Equipment Removed from Sunflower	Metrics	Approved By	OITC Name	OITC Approved Request	Test Results	Termination Date	Termination Checklist	Date CAC/NOAA Badge Retrieved	Memory (in GB)	Disk Space (in GB)	NOAA Contact	CPU (in cores)	Work Results Verified by	Specifications	ASP DOI Requirements Message	IP Address	Domain Name	Watchers	Distinct Error Count	SF Supervisor	Team Lead	Sequence	Manager	GAPSD Customer	GAPSD Species	Date Work Rejected	GAPSD Project Type	SEFSC Change Location	SEFSC Change Date and Time	SEFSC System Type	Effort	Incorporated Version	Founder Sign-off	Morale	QA Completed On	Total Days Spent	NOAALink Worksheet	Testing Notes status	Link to play	Frustration	SEFSC Test Description	CAM	JIRA	SVN	Database	SEFSC Test Scope	Confluence	SEFSC Test Results	SAMBA	Linux	SFTP	Start Date	Name	Employee Location	Division	Phone Type Needed	Temporary Type Employee	NNF Type of Change	NNF Priority Ranking	NNF Quadrant Ranking	Data Collection Being Addressed	Flagged	NNF Documentation Required	Story Points	Epic/Theme	NNF Labels	FDDC Administration Category	Business Value	NNF FDDC Administration	NNF Data Collections	Epic Color	Epic Name	IDI Rec ID
Multispecies model key run review	KEYRUN-24	Develop common biomass index data from surveys	Story	Backlog	Medium	Unresolved	Unassigned	sean.lucey	sean.lucey	12/Jun/20 2:51 PM	15/Jun/20 10:11 AM	13/Jun/20 8:48 AM						0	1							KEYRUN-15, KEYRUN-16		All of the models selected as part of this project need a time series of biomass.  The source of biomass will be the NEFSC bottom trawl survey.  Decisions will need to be made on whether to include both the fall and spring survey or even the winter and summer surveys which have much sorter time frames than the fall and spring.  A common strata set will need to be selected from which an index can be created.  For the full foodweb models, use of species specific surveys (clam, scallop, and shrimp) should be considered.																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				12/Jun/20 2:51 PM			12/Jun/20 2:51 PM		Completed															Request Created								sean.lucey						Common keyrun dataset				0|u061ag:																			sean.lucey																														No		Not Prioritized																			N/A							N/A		System - Modification		12/Jun/20 2:51 PM		N/A			N/A	N/A											No				sean.lucey																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sean.lucey								12/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-23	KEYRUN-5 Test and refine atlantisom diet functions	Sub-task	Selected for Development	Medium	Unresolved	sarah.gaichas	sarah.gaichas	sarah.gaichas	11/Jun/20 2:16 PM	11/Jun/20 2:17 PM	11/Jun/20 2:17 PM						0	1							KEYRUN-5		total consumption time series needed for each predator ages/lengths in diets needed? (diet proportion needed for food web models)																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				11/Jun/20 2:16 PM			11/Jun/20 2:16 PM		Completed															Request Created								sarah.gaichas										0|u060m0:																			sarah.gaichas																														No		Not Prioritized																			N/A							N/A		System - Modification		11/Jun/20 2:16 PM		N/A			N/A	N/A											No				sarah.gaichas																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sarah.gaichas								11/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-22	Discard data for all models, needs methods discussion/decision	Story	Backlog	Medium	Unresolved	Unassigned	andrew.beet	andrew.beet	03/Jun/20 11:53 AM	04/Jun/20 10:44 AM	03/Jun/20 11:53 AM						0	1									Discard estimates will be required for all species, space, time, fleet combinations. * revisit Seans comdisc.r script to dealing with discards * obtain documentation relating to how pop dynamics handle this (S.Wigley scripts?) * add discard scripts to [comlandr|[https://github.com/NOAA-EDAB/comlandr]] or [mscatch|[https://github.com/NOAA-EDAB/mscatch|https://github.com/NOAA-EDAB/comlandr]] or create a new repo   Done when: Methodology finalized and code written to implement methods																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 11:53 AM			03/Jun/20 11:53 AM		Completed															Request Created								andrew.beet						Common keyrun dataset				0|u05wg0:																			andrew.beet																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 11:53 AM		N/A			N/A	N/A											No				andrew.beet																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			andrew.beet								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-21	KEYRUN-14 Initialize github KEYRUN repo	Sub-task	Done	Medium	Done	sarah.gaichas	sarah.gaichas	sarah.gaichas	03/Jun/20 11:46 AM	04/Jun/20 10:39 AM	03/Jun/20 12:50 PM	03/Jun/20 12:50 PM					0	1							KEYRUN-14		Start bookdown? or simpler with standalone rmds/docs folder for: # dataset dimension decisions  # dataset source and methods documentation # dataset descriptions, linked to datasets themselves in different repo folder																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 11:46 AM			03/Jun/20 11:46 AM		Completed															Request Created								sarah.gaichas										0|u05wfk:																			sarah.gaichas																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 11:46 AM		N/A			N/A	N/A											No				sarah.gaichas																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sarah.gaichas								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-20	Length composition data from surveys and fisheries for Hydra in KEYRUN-1	Story	Backlog	Medium	Unresolved	Unassigned	andrew.beet	andrew.beet	03/Jun/20 11:40 AM		03/Jun/20 11:41 AM						0	1									Finalize dependent length data [mscatch|[https://github.com/NOAA-EDAB/mscatch]] (subtasks) : * include discards * aggregate length samples to semester * aggregate length samples to annual * test with comlandr data pulls * seek approval from popdy * include Canadian landings data (for GB)   Lengths composition from survey: * ??? *   Done when: have sign off from pop dynamics																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 11:40 AM			03/Jun/20 11:40 AM		Completed															Request Created								andrew.beet						Common keyrun dataset				0|u05wew:																			andrew.beet																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 11:40 AM		N/A			N/A	N/A											No				andrew.beet																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			andrew.beet								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-19	Establish common definitions of fleets	Story	Backlog	High	Unresolved	Unassigned	robert.gamble	robert.gamble	03/Jun/20 10:25 AM		03/Jun/20 10:25 AM						0	1									Establish common definitions of fleets.  This applies to all: * catch data * discard data * composition data Done: Agreed upon list of fleets and how input data to them should be aggregated.  As much as possible, these definitions should be consistent across models with the understanding that some models may have different levels of aggregation and/or species lists.																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 10:25 AM			03/Jun/20 10:25 AM		Completed															Request Created								robert.gamble						Common keyrun dataset				0|u05wdk:																			robert.gamble																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 10:25 AM		N/A			N/A	N/A											No				robert.gamble																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			robert.gamble								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-18	Establish dataset standards	Story	Backlog	Medium	Unresolved	Unassigned	sarah.gaichas	sarah.gaichas	03/Jun/20 10:09 AM	03/Jun/20 10:09 AM	03/Jun/20 10:09 AM						0	1									Objective: interoperable base datasets for all models Define filetype (available to all operating systems and software) Define format (which attributes required, long vs wide) Establish metadata describing units, source, etc Establish minimum documentation standards Define accessible location for all project participants																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 10:09 AM			03/Jun/20 10:09 AM		Completed															Request Created								sarah.gaichas						Common keyrun dataset				0|u05wd4:																			sarah.gaichas																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 10:09 AM		N/A			N/A	N/A											No				sarah.gaichas																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sarah.gaichas								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-17	Define environmental data needed for each model	Story	Backlog	Medium	Unresolved	Unassigned	sarah.gaichas	sarah.gaichas	03/Jun/20 10:05 AM	03/Jun/20 10:05 AM	03/Jun/20 10:05 AM						0	1									Which data (if any) are needed for model keyruns? List, ensure that it is available at spatial and temporal scales defined for KEYRUN-4																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 10:05 AM			03/Jun/20 10:05 AM		Completed															Request Created								sarah.gaichas						Common keyrun dataset				0|u05wcw:																			sarah.gaichas																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 10:05 AM		N/A			N/A	N/A											No				sarah.gaichas																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sarah.gaichas								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-16	Establish species lists for each model	Story	Backlog	Medium	Unresolved	Unassigned	sarah.gaichas	sarah.gaichas	03/Jun/20 10:03 AM	03/Jun/20 10:03 AM	13/Jun/20 8:48 AM						0	1							KEYRUN-24		longer list for Rpath food web model in KEYRUN-2 shorter list for structured and aggregate multispecies models in KEYRUN-1 and KEYRUN-3																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 10:03 AM			03/Jun/20 10:03 AM		Completed															Request Created								sarah.gaichas						Common keyrun dataset				0|u05wco:																			sarah.gaichas																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 10:03 AM		N/A			N/A	N/A											No				sarah.gaichas																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sarah.gaichas								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-15	Establish temporal and spatial dimensions for data	Story	Backlog	Medium	Unresolved	Unassigned	sarah.gaichas	sarah.gaichas	03/Jun/20 10:00 AM	15/Jun/20 10:03 AM	13/Jun/20 8:48 AM						0	1							KEYRUN-24		Define start and end year for time series Define which seasons Define spatial footprint: what is Georges Bank for survey and for fishery?																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 10:00 AM			03/Jun/20 10:00 AM		Completed															Request Created								sarah.gaichas						Common keyrun dataset				0|u05wc8:																			sarah.gaichas																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 10:00 AM		N/A			N/A	N/A											No				sarah.gaichas																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sarah.gaichas								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections
Multispecies model key run review	KEYRUN-14	Document decision process	Story	Backlog	High	Unresolved	Unassigned	sarah.gaichas	sarah.gaichas	03/Jun/20 9:42 AM	04/Jun/20 10:57 AM	03/Jun/20 11:46 AM						0	1						KEYRUN-21	KEYRUN-21		Have an rmd or other accessible doc where all decisions on other stories are recorded.																															not planned																																																										No Access		No Access	Data Applications														To be reviewed				03/Jun/20 9:42 AM			03/Jun/20 9:42 AM		Completed															Request Created								sarah.gaichas						Common keyrun dataset				0|u05wbs:																			sarah.gaichas																														No		Not Prioritized																			N/A							N/A		System - Modification		03/Jun/20 9:42 AM		N/A			N/A	N/A											No				sarah.gaichas																																																									Need to determine what message will be displayed? Should be a description of what needs to be done as part of preparing DOIs for archival			sarah.gaichas								03/Jun/20											None				N/A					N/A		N/A																			FDDC Administration		FDDC Administration	Data Collections

Generated at Mon Jun 15 10:11:28 EDT 2020 by sarah.gaichas using Jira 8.3.4#803005-sha1:1f96e09b3c60279a408a2ae47be3c745f571388b.