The aim of this Delphi study is to generate consensus on how to best share psychoneuroendocrine data openly. The results of this study will inform us about a) whether or not to proceed with the development of a standard data format for hormonal data and, if we should proceed, B) which things to address when developing the format. Before releasing the newly developed standard data format, we will make sure to externally validate templates and provide infrastructure.
For more information on the NODES project please visit: https://www.nodes-pne.eu.
Please note:
The following two code chunks can be traced for transparency. To read them, expand the chunks by clicking “Code > Show all code” or “Show”. They describe which entrys were kept in the final sample.
# drop maria's pilot
data_full <- data_full %>%
filter(any(
is.na(PNE_hormones_text),
PNE_hormones_text != "Pilot maria")
)
# Relevant columns = columns that are not open text (i.e., sample description, statement rating on Likert scale)
relevant_cols <- c("PNE_researcher", "PNE_experience", "PNE_articles", "PNE_species", "PNE_datashare", "G1_hormones", "G2_questionnaire", "G3_physio", "G4_wet", "G5_uni_sample", "G6_uni_hormones", "G7_react_basal", "G8_exp_observe", "F1_file_type", "F2_codebook_varnames", "F3_rawdata", "F4_average", "F5_addonfile", "F6_unit", "F7_standreldtime", "F8_EUtemp", "F9_missings", "D1_sex", "D2_gender", "D3_age", "D4_educat", "D5_occu", "D6_work", "D7_ethn", "D8_nation", "D9_geno", "D10_daynight", "D11_mens_cycle", "D12_oral_contra", "D13_oral_contra_type", "D14_nicotine_t", "D15_alcohol_t", "D16_drugs_t", "D17_fasting", "D18_nicotine_s", "D19_alcohol_s", "D20_drugs_s", "D21_meds_disease", "D22_time_sampling", "D23_time_awakening", "M1_number", "M2_timestamp_global", "M3_timestamp_stand", "M4_exp_baseline_name", "M5_obs_baseline_def", "M6_metadata_core", "M7_manipulation", "M8_assay", "M9_assayCV", "M10_recruitment", "M11_in_exclusion", "M12_biospecimen", "M13_sampling_proc", "M14_hairwash", "M15_hairchem", "M16_hairvol", "M17_storage", "M18_species")
# preregistration: Data of experts that fill in more than 30 % of the survey will be included in the study.
min_answers <- round(length(relevant_cols)*0.3)
data_full <- data_full[-which(rowSums(is.na(data_full[, relevant_cols])) > min_answers), ]
write.csv(data_full, file = "processed/NODES_data_r1_clean.csv")
The questionnaire covered the following topics:
The survey contained 65 items that were rated on a Likert scale, 45 open text fields that explained participants’ Likert ratings and 45 open text fields that suggested alternative wordings.
Overall, N = 52 participants filled in the survey. They had an average amount of 14.5 years of experience in the field (SD = 9.19, range: 1, 44, missings: 0).
All participants (N = 52) answered the question Have you ever worked, or are you currently working in the field of psychoneuroendocrinology or related fields that work with hormones (in the following, PNE)? wit “yes”.
The following Figure depicts the answers to the question How many articles have you published in the field of PNE or related fields?:
The following Figure depicts the answers to the question Which species are you working with?:
The following Figure depicts the answers to the question How often have you shared (raw or preprocessed) data openly alongside publications, e.g. on an open repository like the Open Science Framework, in the past?:
When asked Which hormones and markers do you commonly work with? (multiple selection possible), participants answered the following:
The following hormones were additionally named in the open text box:
Consensus was defined as more than 70% ‘(dis)agree’ or ‘strongly (dis)agree’ with a statement (cf. preregistration: https://osf.io/c3y4p). For the calculations of the percentages that form the consensus index, we excluded ratings of participants that stated they had “no expertise” on this item.
In the first round of the Delphi study, we reached a consensus on the majority of items, with only 11 items not reaching the criterion.
The consensus rates are depicted below:
Statement 5 did not reach the consensus criterion and was rephrased for the second round of the Delphi study.
The original statement read:
A standardized format for PNE research should be universal for all kinds of sample origins and species (e.g., samples from humans, animals and in vitro research).
The following comments were made:
Neutral:
Agreeing:
Disgreeing:
Based on the comments provided in the open text, we rephrased the statement to:
“A standardized format for PNE research should consist of flexible modules that can accommodate diverse sample origins and species-specific variables (e.g., human, animal, and in vitro samples).”
The consensus rates are depicted below:
Statement 4, 5 and 8 did not reach the consensus criterion and are thus rephrased in the second round of the Delphi study.
The original statement read:
A standardized data format for PNE should include the averaged hormone values obtained from replicates (e.g., duplicates, triplicates).
The following comments were made:
Neutral:
Agreeing:
Disgreeing:
Based on the comments provided in the open text, we rephrased the statement to:
“A standardized data format for PNE should include the hormone values obtained from each assay replicate (e.g., duplicates, triplicates), rather than only the averaged values.”
The original statement read:
Other data beyond the scope of the standardized format should be stored in additional files with a codebook included.
The following comments were made:
Neutral:
Agreeing:
Disgreeing:
Based on the comments provided in the open text, we rephrased the statement to:
“Optionally and where appropriate, other data beyond the scope of the standardized data format (e.g., semi-structured interview data, qualitative data) should be stored in additional files with a codebook included.”
The original statement read:
Adopting and extending an existing format (e.g., the Stress-EU template) is preferable to developing a new standard from scratch.
The following comments were made:
Neutral:
Agreeing:
Disgreeing:
none
Based on the comments provided in the open text, we rephrased the statement to:
“Adopting and extending elements of an existing format with proven suitability and good documentation is preferable to developing a new standard from scratch.”
The respective statements asked participants to rate which subject info should be included in the standard data format.
The consensus rates are depicted below:
Statements 4 (education), 5 (occupation), 6 (work), and 8 (nation) did not reach the consensus criterion and are thus rephrased in the second round of the Delphi study to be included optionally.
The rephrased statement reads:
“Optionally, researchers may provide variables indicating subject information like education, occupation, working hours or nation.”
These questions were mainly asked to get a sense of which meta data may be necessary to be able to interpret the data correctly.
The consensus rates are depicted below:
Statement 5 did not reach the consensus criterion and is thus rephrased in the second round of the Delphi study.
The original statement read:
For observational studies involving, e.g., the measurement of diurnal profiles, the time point (t0) should be defined as the first sample collected during the study.
The following comments were made:
Neutral:
Agreeing:
Disgreeing:
Based on the comments provided in the open text, we rephrased the statement to:
“The standardized data format for PNE should include a variable with a clear description of what the respective timepoint represents relative to interventions, circadian rhythm, etc. (e.g. first sample before intervention, first sample of day 1, sample of day 2 assessed at 2 PM, etc.).”
These questions were asked to get a sense of which level of “convenience” is needed for the user to adopt the new format and which support measures should be developed.
The consensus rates are depicted below:
Statements 2 and 5 did not reach the consensus criterion and are thus rephrased in the second round of the Delphi study.
The original statement read:
I would use a PNE standardized data format only if it can be integrated with existing infrastructures (e.g., BIDS, GitHub, OpenNeuro etc.).
The following comments were made:
Neutral:
Agreeing:
Disgreeing:
none
Based on the comments provided in the open text, we rephrased the statement to:
“The possibility to integrate the new standard with existing infrastructures (e.g., BIDS, GitHub, OpenNeuro etc.) is nice-to-have, but not a must.”
The original statement read:
I would learn new software/coding to use a PNE standardized data format.
The following comments were made:
Neutral:
Agreeing:
Disgreeing:
Based on the comments provided in the open text, we rephrased the statement to:
“My willingness to learn new skills to be able to adopt the new standard data structure highly depends on effort, benefits, and available guidance.”
Additionally, based on the open text fields, the following variables are now included in the questionnaire:
The subject information should include: