Reproducibility Interest Group - TAB Review

Interest Group Title:  Reproducibility

Proposers:  Victoria Stodden and Bernard Schutz

Date Received by TAB:  November 6, 2014

Date Review Completed:  November 20, 2014

TAB Reviewers:  Jamie Shiers and Andrew Treloar

Summary:  

 

TAB recommends that the IG charter be approved.

 

Focus and Fit:  

(Are the Interest Group objectives aligned with the RDA mission ?  Is the scope too large for effective progress, too small for an RDA effort, or not appropriate for the RDA?  Overall, is this a worthwhile effort for the RDA to take on?  Is this an effort that adds value over and above what is currently being done within the community?)

The objectives are definitely aligned with the driver of reducing barriers to interoperability (interoperating with someone else's data requires attention to the kinds of things that are in scope for this group)

Furthermore, the funding agencies are now requiring reproducibility to be addressed, so this is a very topical issue.

The scope is large, but the right people are involved.

Overall, this is a worthwhile effort and would serve to coordinate lots of ongoing but uncoordinated activity.

 

Capacity:

(Does the initial membership list include sufficient expertise, and disciplinary and international representation?  Are the people involved in the Interest Group sufficient to make tangible progress?  What individuals or organizations are missing?) 

The initial membership is an embarrassment of riches. No suggestions for additions.

 

Impact and Engagement:

(Is it likely that the Interest Group will engage the intended community?  Is there evidence that the research community wants this?  Will the outcome(s) of the Interest Group foster data sharing and/or exchange?)

Based on those involved, there seems little risk that the community will not be engaged. There is increasing interest in a number of research communities in the subject of reproducibility, and so this is very timely.

 

See also the above - funding agencies are requiring this (or at least a good let-out).

 

Recommendation:  

Charter is Sufficient X; Charter Requires Revision __; Charter is Rejected __

Comments:

 

<if conditional, state revisions … e.g., TAB recommends that the IG charter be conditionally approved subject to the following change to the IG charter:>



  • Kathleen Fontaine's picture

    Author: Kathleen Fontaine

    Date: 21 Nov, 2014

    btw on point 2, the list is of confirmed names.  Victoria asked all of them, and they said yes.  Now, whether they will all participate is another issue, but after the initial Community round, she did the confirmation.

    I can look into moving the conversation to TAB.

    That just leaves point 1.  That's something for the TAB call; I am their Sec liaison....I don't think they have a TAB liaison yet.

    How would you like to proceed with point 1?

    cheers

    k

    On 11/21/14, 1:14 PM, Mark Parsons wrote:

    I don’t know how to move the conversation, but perhaps a quick touching of base on the topic at the TAB meeting is appropriate. 

     
    cheers,
     
    -m. 
    On Nov 20, 2014, at 9:49 PM, Andrew Treloar <andrew.treloar@ands.org.au> wrote:


    I agree with my learned colleagues that this charter (not case statement, as this is an IG) is fine, with two caveats:

    1. they talk about developing a "research agenda" - for a WG this would be a concern (as the focus should be on developing actual solutions, not on research), but is probably acceptable for an IG. And, in any case, it might just be a choice of words that did not have those connotations to those who wrote it
    2. the list of names is almost certainly aspirational, rather than actual. But if we aren't going to delay the approval while asking them to confirm, then I see no value in asking them. It just makes us look bureaucratic and process heavy.

    So, happy to recomment to Council that we approve.

    By the way, according to my understanding of the current process, this conversation should be taking place under TAB, not Secretariat. I can't see how to move it - can someone from Secretariat please take responsibility for doing this?

    Thanks!

    --
    Full post: https://www.rd-alliance.org/group/rda-secretariat/post/reproducibility-ig-charter-ready-review.html
    Manage my subscriptions: https://www.rd-alliance.org/mailinglist
    Stop emails for this post: https://www.rd-alliance.org/mailinglist/unsubscribe/46346

     

    --
    Full post: https://www.rd-alliance.org/group/rda-secretariat/post/reproducibility-ig-charter-ready-review.html
    Manage my subscriptions: https://www.rd-alliance.org/mailinglist
    Stop emails for this post: https://www.rd-alliance.org/mailinglist/unsubscribe/46346

    --
    Dr. Kathleen S. Fontaine (Kathy)
    Managing Director, Research Data Alliance/US (RDA/US)
    RDA Secretariat Member

    Amos Eaton Building, Room 211
    Rensselaer Polytechnic Institute (RPI)
    110 8th Street
    Troy, NY 12180-3590

    Cell: 410-991-6728
    Office: 518-276-2829

    Email: fontak@rpi.edu
    Skype: ksfontaine

     

     

     

     

    I agree with my learned colleagues that this charter (not case statement, as this is an IG) is fine, with two caveats:

    1. they talk about developing a "research agenda" - for a WG this would be a concern (as the focus should be on developing actual solutions, not on research), but is probably acceptable for an IG. And, in any case, it might just be a choice of words that did not have those connotations to those who wrote it
    2. the list of names is almost certainly aspirational, rather than actual. But if we aren't going to delay the approval while asking them to confirm, then I see no value in asking them. It just makes us look bureaucratic and process heavy.

    So, happy to recomment to Council that we approve.

    By the way, according to my understanding of the current process, this conversation should be taking place under TAB, not Secretariat. I can't see how to move it - can someone from Secretariat please take responsibility for doing this?

    Thanks!

    --
    Full post: https://www.rd-alliance.org/group/rda-secretariat/post/reproducibility-i...
    Manage my subscriptions: https://www.rd-alliance.org/mailinglist
    Stop emails for this post: https://www.rd-alliance.org/mailinglist/unsubscribe/46346

     

    On Nov 6, 2014, at 5:32 AM, Jamie Shiers <Jamie.Shiers@cern.ch> wrote:


    >From my point of view, this revised case statement is fine. There is a mismatch between the possible initial members and the actual members of the corresponding RDA list, but that's a detail.
    The list  in the charter has some big names. I suspect they have not actually asked many of the people to join. We should ask them to confirm their participants. (That should not delay approval, though).
     
    cheers,
     
    -m. 

     

     

     

     

     

     

    Hi Kathy,

     
    From my point of view, this revised case statement is fine. There is a mismatch between the possible initial members and the actual members of the corresponding RDA list, but that's a detail.
     
    Unfortunately, I was not able to attend their BoF session due to conflicts. However, I spent quite some time talking to Victoria and Bernard and have also looked into their activities. They seem to be a very competent pair of co-chairs for this activity which I view as important.
     
    I like the fact that they explicitly mention asynchronous working methods (in addition to synchronous ones) - as we know, getting everyone together at the same time is essentially impossible.
     
    In other words, I would support accepting this case statement as is.
     
    Cheers, Jamie
    On Nov 5, 2014, at 10:56 PM, fontaine <fontak@rpi.edu>
     wrote:


    Hi Andrew and Jamie -

    The revised Reproducibility IG Charter is ready for TAB review.  You two had volunteered before P4, and so are still on my list, as it were.

    I have uploaded their revision (nov 2) onto their Charter page, and revised the link so that it automatically downloads the latest version.  Go here for the info:  https://www.rd-alliance.org/group/reproducibility-ig/case-statement/reproducibility-ig-case-statement.html

    The document received no comments at all during the on-line Community period, but took into account comments during their face-to-face meeting in Amsterdam, and have reworked the draft accordingly. 

    So - the clock starts on this one on November 6.  Two weeks from then is November 20.  Will that be enough time?

    Thanks

    Cheers

    k

    --
    Dr. Kathleen S. Fontaine (Kathy)
    Managing Director, Research Data Alliance/US (RDA/US)
    RDA Secretariat Member

    Amos Eaton Building, Room 211
    Rensselaer Polytechnic Institute (RPI)
    110 8th Street
    Troy, NY 12180-3590

    Cell: 410-991-6728
    Office: 518-276-2829

    Email: fontak@rpi.edu
    Skype: ksfontaine

    --
    Full post: https://www.rd-alliance.org/group/rda-secretariat/post/reproducibility-ig-charter-ready-review.html
    Manage my subscriptions: https://www.rd-alliance.org/mailinglist
    Stop emails for this post: https://www.rd-alliance.org/mailinglist/unsubscribe/46346

     

     

    Hi Andrew and Jamie -

    The revised Reproducibility IG Charter is ready for TAB review.  You two had volunteered before P4, and so are still on my list, as it were.

    I have uploaded their revision (nov 2) onto their Charter page, and revised the link so that it automatically downloads the latest version.  Go here for the info:  https://www.rd-alliance.org/group/reproducibility-ig/case-statement/reproducibility-ig-case-statement.html

    The document received no comments at all during the on-line Community period, but took into account comments during their face-to-face meeting in Amsterdam, and have reworked the draft accordingly. 

    So - the clock starts on this one on November 6.  Two weeks from then is November 20.  Will that be enough time?

    Thanks

    Cheers

    k

    --
    Dr. Kathleen S. Fontaine (Kathy)
    Managing Director, Research Data Alliance/US (RDA/US)
    RDA Secretariat Member

    Amos Eaton Building, Room 211
    Rensselaer Polytechnic Institute (RPI)
    110 8th Street
    Troy, NY 12180-3590

    Cell: 410-991-6728
    Office: 518-276-2829

    Email: fontak@rpi.edu
    Skype: ksfontaine

     

  • Rainer Stotzka's picture

    Author: Rainer Stotzka

    Date: 25 Nov, 2014

    I would be very interested to act as a Liaison, but not as a reviewer due to conflicts of interest.

    RAiner

     

submit a comment