1 Overview

The aim of this Delphi study is to generate consensus on how to best share psychoneuroendocrine data openly. The results of this study will inform us about a) whether or not to proceed with the development of a standard data format for hormonal data and, if we should proceed, B) which things to address when developing the format. Before releasing the newly developed standard data format, we will make sure to externally validate templates and provide infrastructure.

For more information on the NODES project please visit: https://www.nodes-pne.eu.

Please note:

The following two code chunks can be traced for transparency. To read them, expand the chunks by clicking “Code > Show all code” or “Show”. They describe which entrys were kept in the final sample.

## Warning: package 'ggplot2' was built under R version 4.4.3
# drop unecessary columns
data_full <- data_full[, c(11, 43:114)]

# Relevant columns = columns that are not open text (i.e., sample description, statement rating on Likert scale)
relevant_cols <- c("G5R_flexibleModules", "F4R_replicates", "F5R_replicates",
                   "F8R_replicates", 
                   "M5R_timestamps", "C2R_integration", "C5R_existing",
                   "DR_optional", "N_BMI", "N_Weight", "N_Interventions",
                   "N_SamplingAdherence", "N_Caffeine", "N_SleepPattern", 
                   "N_AcuteSickness", "N_AcuteExercise", "N_Lactation",
                   "N_explain", "N_word", "N_contextSociety", "N_Puberty",
                   "N_Bedtime", "N_Season", "N_Year", "N_timeCovars", 
                   "N_Nesting", "N_species", "N_strain", "N_Housing",
                   "N_circadianTime", "N_environment", "N3_explain",
                   "N3_wording")

# preregistration: Data of experts that fill in more than 30 % of the survey will be included in the study.
min_answers <- round(length(relevant_cols)*0.3)

data_full <- data_full[-which(rowSums(is.na(data_full[, relevant_cols])) > min_answers), ]
# average time to fill out survey
# mean(data_full$S16/60)
data_full <- data_full[, -1]
write.csv(data_full, file = "processed/NODES_data_r2_clean.csv")

2 The survey

The survey contained 11 items that were rated on a Likert scale, 11 open text fields that explained participants’ Likert ratings and 10 open text fields that suggested alternative wordings.

3 The sample

Overall, N = 50 participants filled in the survey. Of those, N = 26 participants had participated in the first Delphi round and N = 24 were newly recruited.

Newly recruited participants had an average amount of 11.79 years of experience in the field (SD = 8.64, range: 1, 30) and all (N = 24) indicated that they were “PNE researchers”.

3.1 Newly recruited participtans (N = 24) - sample description

The following Table depicts the answers to the question How many articles have you published in the field of PNE or related fields?:

## 
##       none      1 - 5     6 - 10    11 - 20 20 or more 
##          1         10          3          4          6

The following Table depicts the answers to the question Which species are you working with?:

## 
##         human animal models          both 
##            20             3             1

The following Table depicts the answers to the question How often have you shared (raw or preprocessed) data openly alongside publications, e.g. on an open repository like the Open Science Framework, in the past?:

## 
##                   always in the majority of cases         every other time 
##                        1                        2                        1 
## in the minority of cases                    never 
##                       11                        9

4 Results

Consensus was defined as more than 70% ‘(dis)agree’ or ‘strongly (dis)agree’ with a statement (cf. preregistration: https://osf.io/c3y4p). For the calculations of the percentages that form the consensus index, we excluded ratings of participants that stated they had “no expertise” on this item.

In the second round of the Delphi study, we reached a consensus on the majority of items, with only four items not reaching the criterion.

4.1 Revised questions

Click here to show original statements.
  1. A standardized format for PNE research should consist of flexible modules that can accommodate diverse sample origins and species-specific variables (e.g., human, animal, and in vitro samples).
  2. A standardized data format for PNE should include the hormone values obtained from each assay replicate (e.g., duplicates, triplicates), rather than only the averaged values.
  3. Optionally and where appropriate, other data beyond the scope of the standardized data format (e.g., semi-structured interview data, qualitative data) should be stored in additional files with a codebook included.
  4. Adopting and extending elements of an existing format with proven suitability and good documentation is preferable to developing a new standard from scratch.
  5. The standardized data format for PNE should include a variable that clearly explains what the sample collection timepoints represent, and/or in which format timepoints are indicated (e.g., relative to the start of the study or the intervention, relative to a timepoint in the circadian rhythm such as 30 min after awakening or 3PM, or in a continuous format such as seconds since a particular point in time [cf. POSIX format]), etc.).
  6. The possibility to integrate the new standard with existing infrastructures (e.g., BIDS, GitHub, OpenNeuro etc.) is nice-to-have, but not a must.
  7. My willingness to learn new skills to be able to adopt the new standard data structure highly depends on effort, benefits, and available guidance.
  8. Optionally, researchers may provide variables indicating subject information like education, occupation, working hours or nation.

The consensus rates are depicted below:

All statements reached the consensus criterion.

4.2 New items

The consensus rates are depicted below:

The following statements did not reach the consensus criterion.

  • Statement 2: Weight. Agreement that BMI will be included (cf. statement 1). Decision of Steering Group: weight can be dropped.

  • Statement 6: Sleeping pattern. Concerns in open text on how to operationalise it. Decision of Steering Group: Discussion cannot be resolved within the Delphi study.

  • Statement 10: Societal context. Concerns in open text on how to operationalise it. Decision of Steering Group: Discussion cannot be resolved within the Delphi study.

  • Statement 12: Bedtime. Might be redundant dependent on other info given (e.g. sleep duration and awakening).

Overall conclusion: Due to the low number of items that need resolvement, no third round will be conducted.