SPARC Overall Standards Strategy

Standards in SPARC should achieve the following:

• Ensure that SPARC data is FAIR through adherence to the FAIR Data Principles, which include rich metadata, FAIR vocabularies, and adherence to community standards.
  • Community standards in use outside of SPARC are preferred when possible. We are particularly monitoring what is coming out of the US BRAIN Initiative, but we should also be looking at other Common Fund projects given the desire to integrate across Common Fund Projects through the Common Fund Data Ecosystem (CFDE).
  • Standards will be developed as necessary for SPARC when none are available. Depending on the nature of the standard, we may create an ad hoc committee of SPARC investigators to work through a problem.
• Improve the user experience by ensuring that SPARC data is harmonized with respect to formats, metadata, etc., to the degree possible, and allows users to write computational tools against SPARC data.

Complying with standards can be hard, particularly without computational support, and frequent updating of standards can place a significant burden on both the data submitter and the infrastructure. Decisions about what standards to use and when will therefore be based on the maturity and support for the standard. The INCF has developed a set of criteria that can help evaluate standards under consideration (Abrams et al. 2019).

Overview of Process

Because standards and best practices drive the type of integration that SPARC strive to obtain, standards are identified based on user and SPARC Data and Resources Center (DRC) requirements. Anyone within the DRC may propose a standard for SPARC. Decisions will be made internally, but the standard will be open to community discussion. Accepted standards will be posted publicly on docs.sparc.science.

To Submit a Standard

Submitters of a standard should create a brief overview of the standard and provide the following information.

• Basic information about the Standard:
  • Title
  • Brief description
  • Person/group requesting Standard
  • If appropriate:
    • URL
    • Steward
    • Relevant publication
• Why is this standard needed by SPARC?
  • What role will this standard fill in SPARC?
• Who will be affected by the standard and who should be involved in the review process?
  • Will this be surfaced/accessed outside of the DRC? If so, what communities/users will access it? SPARC investigators, tool developers, a broader community, etc.
  • If internal, which Cores, will be affected?
• Are there other similar standards and if so, why was this one chosen?
• Verification/illustration that the standard works for the proposed purpose:
  • If there’s a conversion required, it needs to be tested for validity (a “round-trip” conversion)
  • Is there a reference implementation?
• Recommended level of rigor of review by SPARC:
  • No review necessary
  • Sign-off by certain individuals
  • Rigorous review process

Notification of the DRC PIs of a proposed standard can be through either of two receiving DRC entities: K-Core or the Tech Leads (alerted through the RA/QA) via a submission request at https://sparc.science/contact-us?source_url=/&type=feedback. The proposed standard for SPARC should include the information outlined above.

The receiving DRC entity vets submission for appropriateness and includes other appropriate members of the DRC in this process. The receiving DRC entity will take the request along with their recommendation to the PIs for discussion. The PIs will evaluate the criteria and determine:

• Who should be involved in the review process
• The recommended level of rigor of review:
  • No review necessary
  • Sign-off by certain individuals
  • Rigorous review process

The standard will then go through the process for the recommended level of rigor. Details about each are below.

No Review Necessary

A standard that does not affect anyone outside of the group proposing the standard AND the Verification/illustration that the standard works for the proposed purpose is sufficient does not require a rigorous review. In this case, the DRC should be notified, with the overview of the standard along with a request for comments about the standard by the PMs. The DRC will be given 30 days to comment. If any major concerns are raised during this process, the rigor of the review process should be elevated.

Internal discussions around the standard will be kept internally; an overview of the standard will be posted publicly on docs.sparc.science.

Sign-off by Certain Individuals

A standard that affects more than a single group should require relevant stakeholders to weigh in on the recommended standard. The group putting forward the standard will lead it through this process.

Representative stakeholders should be identified by the DRC (e.g. the submitter of the standard, the Tech leads, members of K-Core, the PIs). Those individuals will be asked to review and sign-off on the standard when they are satisfied that their concerns are addressed. The individual will be given 30 days for their initial comments. PIs will be appraised of any major developments, including when sign-off occurs.

Once sign-off occurs, the DRC will be notified of the adoption of the standard by the PMs. It will also be documented and posted publicly on docs.sparc.science.

Rigorous Review Process

A standard that will affect multiple groups OR performance of the SPARC Portal will go through a more rigorous review process.

• PIs will identify/approve two experts who will be responsible for doing the in-depth review of the proposed standard.
• If appropriate expertise is held within the DRC, those members will be asked to do an in-depth review of the Standard. If expertise is not held within the DRC, external experts will need to be consulted.
• The process will be given a form that includes a set of questions about the standard (see here). Along with this, they will make a recommendation for:
  • Rejection
  • Continuation through the review process
  • Revisions required before continuing through the review process

The results of the in-depth review will be presented back to the PIs.

• The PIs will determine if it should continue the vetting process or be rejected.
• If the process should be continued and revisions are required, these will be requested from the submitters of the standard until the reviewers and PIs are satisfied.
• If a standard is rejected, this information will not be surfaced beyond the DRC.

Once a standard has reached “continuation” status:

• The standard will be posted on docs.sparc.science and relevant community members will be invited to give feedback.
• This public review period will be 60 days and include multiple outreach attempts requesting feedback.

The two expert reviewers will appraise the results of the public review and submit to PIs a final recommendation on if the standard should be adopted by SPARC. PIs will determine if the standard should be rejected or accepted.

Post Review Procedure

For all ACCEPTED standards

An overview of the standard and results of any review process will be documented and posted publicly on docs.sparc.science. The ability for the community to comment will remain open as long as the standard is in use by SPARC.

For all REJECTED standards

All related information and reasons for rejection will be documented and kept internally. If a standard is rejected, this information may not be surfaced beyond the DRC.