ADMIRAL 20101207 meeting CH
From ImageWeb
ADMIRAL progress meeting with Silk Group
- When: 7-Dec-2010, 10:30-13:00
- Where: Room B17, Zoology department
- Present: CH, DS, GK, BA
Goals
- Review use of ADMIRAL by Silk group
- Review recent progress on ADMIRAL
- Solicit feedback on submission interface
- Broach plans to lodge real research datasets with Databank
- Determine next steps
Agenda
- Review project to date
- Review use of ADMIRAL by Silk group
- Introduce library services' Databank system
- Demonstrate submission interface and solicit feedback
- Demonstrate dataset viewing interface and solicit feedback
- Discuss submission of real dataset to Databank service
- Plan for deploying new ADMIRAL software to the Silk group
- Review of priorities
Details
Review project to date
- New version of ADMIRAL base system developed; revised system generation scripts and test cases; slightly revised directory layout for data
- Working with library services to create interface for submission of datasets, and viewing of submitted datasets
I forgot to mention in the meeting that we now have a plan to keep ADMIRAL development and support alive for an additional 2-3 years, through our partnership in a new EU e-science workflow project, using AQDMIRAL as an evaluation platform for ideas around Research Objects, dataset provenance and workflow sharing.
Review use of ADMIRAL by Silk group
The Silk group had rapidly filled up the available disk space, and stopped using ADMIRAL at that point. We had previously discussed providing more disk capacity via an additional NAS box, but had not prioritized deployment as we were unaware that the available space had been exhausted.
We agreed to prioritize deployment of additional storage, and move the Silk group onto the latest deployment of the ADMIRAL system at the same time. Total space required is in the range 2-4 Tb.
In addition to the "private", "shared" and "collab" areas, they also asked for a "groupshare" area with read/write access for all group members (for common software, etc.), and a "groupexchange" area to be used for temporary file exchange, and which would be periodically cleared out.
There was some discussion of what happens to data areas of members who leave the group; no form conclusion yet. (During the meeting, I didn't mention that the current plan is to transfer ownership of these to the RG leader - we need to discuss further)
It was confirmed and reinforced that the automatic daily backup is an important selling point for getting researchers to use ADMIRAL.
It was suggested that zoology email addresses should be used for usernames (with or without the @zoo.ox.ac.uk to be confirmed). (We still don't have user tools for user allocation, password setting, etc.)
Introduce library services' Databank system
- Repository for "Research Objects" - aggregations of information and data supporting a research goal [1]
- Supports identification, versioning, preservation, publication
- Dataset / Research Object API submission API used by ADMIRAL
- Supports incremental improvement through re-submission of updated data
- View submitted datasets
There was some discussion of the "Research Object" (RO) terminology - initially, this was felt to imply a single resource rather than an aggregation, and dataset was preferred. But on further discussion of some alternatives, CH warmed to the term "Research Object".
Demonstrate submission interface and solicit feedback
- Browse button or tree display on submission form (try tree display first, default collapsed to top level)
- Dataset versioning should start at 1, not 2. Suggest initial empty dataset is version 0.
- Use dataset name as given for unpacked data; add "-packed" suffix for ZIP file version
- Confirmation page: "back to front page" link should be clearer "return toi ADMIRAL front page"
- Confirmation page: option to resubmit dataset
- Databank view links: put these on the ADMIRAL view of the dataset, rather than on the conformation page
- Last modified (include?)
- "Created by" -> "Submitted by"
- "Last modified" -> "Submission date" and include version alongside
- Add confirmation page showing dataset content (like view page) before actually performing submission
Embargo setting: default value (near term or distant future?); authorization to release; pre-authorized future releases. Needs a proposal. As k Cambridge how they handle this.
Need some clarification of roles: who can submit a dataset; who can release a dataset for publication? WHo can respond to an embargo-release prompt?
Lots of detail in the Databank web page display should not be visible to typical ADMIRAL user. (might help to get Chris involved in meeting with Anusha?)
Comments on Databank dataset display interface
- "Item's Embargo state" should be "Item's embargo state" (consistent case)
- "Item's RDF Manifest" should be "Item's RDF manifest"
- "Change RDF Manifest" should be "Change RDF manifest"
- "Add File to ..." should be "Add file to ..."
Demonstrate dataset viewing interface and solicit feedback
Dataset list interface:
- Show only the unpacked datasets in the list of datasets (How to detect? Exclude entries with single ore:aggregates property where the aggregated item itself has a dcterms:hasVersion property.)
- By default, CH wants to see only personally submitted datasets, with an option to see all datasets submitted by his group. Suggestion that only RG leader can see all datasets (I think this may be harder to do)
- Also show: version number, when submitted, who submitted. Include additional metadata naming submitter (ADMIRAL username or personal name? If username = email address, that would work well enough?)
- Possibly want to see all people who have submitted any version of a dataset - need to work through access issues with library services.
- Option to expose directory map (tree) in list display?
- Place links to
- Granularity of silo alllocation will affect what datasets are potentially visible. Favour granularity at research group level (which I think is what we've discussed with library services)
- ADMIRAL banner on web page (help to distinguish ADMIRAl displays from Databank displays)
Dataset view interface:
- Include links to library service databank entries for packed and unpacked versions
- ADMIRAL banner on web page (help to distinguish ADMIRAl displays from Databank displays)
- Default embargo data (a) in far future, or (b) short date provided that further confirmation is required before data is released.
- Inconsistent header case
General:
- Use dataset name for unpacked dataset; add "-packed" suffix for ZIP file version.
Discuss submission of real dataset to Databank service
- Additional metadata in manifest, identifying submitter
Plan for deploying new ADMIRAL software to the Silk group
Broad plan agreed in meeting is to deploy new data volume with new installation of Silk group system.
The following details were added later:
- Source and install larger disk array based on cheaper SATA rather than SAS; 8Tb raw.
- Create new data disk on SATA array
This will require old system to be offline temporarily
- Create new software system, using new directory layout and new data volume on SATA array
- Test new system
This will require a coordination with Silk group to ensure a smooth transition
- Recreate user accounts on new system
- Connect old data volume
- Copy old data to new arrray
- Disconnect old data volume
Test with users. If all is well, switch; keep old system around until we have confidence in the new system
Review of priorities
- Order new disk hardware and configure into ADMIRAL hosting environment
- User interface improvements
- Create new system deployment for Silk Group, and copy data
- Discuss issues raised (versioning, authentication, ...) with Anusha. Also timeframe for live deployment of Databank.
- Read-only account for ADMIRAL to have access to all submitted data?
- CH: nominate exemplar dataset
- Proposal for embargo handling (e.g. shorter date requires further authorization or reset with long date for automatic release?) (5 year with rolling 1-year extension?)
