DefiningImageAccess/PostProjectReview
From ImageWeb
Contents |
Post-project review notes
These are a collection of notes to consider as lessons to be learned from the conduct of this project.
The idea of reviewing and recording project successes and failures is an important part of ongoing quality management, part of the concept sometimes nominated by the Japanesse term 'Kaizen' (http://en.wikipedia.org/wiki/Kaizen). (Just reviewing that page it strikes me that there is a strong resonance here with many tenets of agile programming.)
Project planning
- All day meetings - effort required is considerably more than the meeting itself: I find an extra day is tied up with planning, travel arrangements, post meeting review and write up. The last is particularly important if long-term value is to be realized from attending the meeting.
- Whether planned or not, it's very difficult to keep project management and general administration below 1 day/week for this kind of project. Maybe a more heads-down development project may be different.
- Detailed work package breakdowns and schedules loose their significance after 2-3 months. This may particularly true for a scoping/survey project.
- Allow time/budget for participation in JISC community activities.
- In the end, we spent very much more more time on the related work survey than the 10 days originally planned - possibly 30-40 days or more, and this was without going especially deeply into most of the work surveyed. In a sense, the related work survey never stopped - I don't think there has been a week go by without some additional notes on related work being added to the wiki. This is probably particularly the case for a scoping/survey project like this, and survey would be more contained in a pure development or focused investigation project.
- Software evaluation: it has been quite difficult to apply purely objective criteria to the selection of software for detailed evaluation. In hindsight, the selection of about 10 packages evaluated seems quite idiosyncratic, with no clear single unifying theme in their choice. That said, the evaluations performed do (so far) seem to have given us a good degree of confidence in our specific proposals for moving forward.
Gathering survey information
The survey schema were not really used as intended (cf. Meetings/20070124/SurveySchemas). The nearest we got to this was the meeting with OULS, where a set of questions was prepared and emailed in advance (but based on the Cambridge meeting minutes rather than the suggested schema). The meeting then used this as a guide for discussion, and did succeed in covering most if not all of the questions raised, with room for some exploratory free-ranging discussion. The schema approach may have been too prescriptive to assist us in our goals of learning about the repository landscape.
Use of Semantic Media Wiki
Our use of this tool was exploratory. The wiki has worked very well for us, though it's less clear how much value we have obtained by using the semantic markup features - this is an aspect we should review. Apart from the initial setup, which took about a day, it has cost us very little as the semantic markup doesn't intrude unless it is actually used, so using an SMW-enhanced media wiki has been a good choice. Now we have a clearer idea of what SMW can do, maybe in future projects we can plan better how to get benefit from its features.
The semantic media wiki features might be ideal as a tool for project progess tracking - if time permits, this is an area that I think we should explore. (E.g. introduce a template for recording actions, and another for noting that an action is complete. Can we then use a query to find all outstanding actions?)
In the final phases of the project, the semantic media wiki started to be useful, helping us to draw together some information for the final report. The Semantic Media Wiki search facility, which appears quite primitive, is actually quite usable for finding topics (pages) similar to a given topic by some selected criterion.
SMW templates make it relatively easy to record common bunches of metadata. It was probably rather too late in the project that we really learned how easy these were to use. Some time spent at the start of the project thinking about templates might have improved the qualoity of semantic information. (See also: http://www.ch.ic.ac.uk/wiki2/index.php/Ic-w3c-07.)
Working in a public wiki
- Capturing all information into a publicly visible has gereally worked very well. The main benefit of being public on the web is the "Martini" effect: it's easy to add notes anytime, anyplace, anywhere that there's a connected web browser.
- Working in public is not something that everyone is comfortable with, and there are some pitfalls. I made the mistake of copying some passages (with attribution) from an EU project report that had not been formally published, without first asking permission from the person who had provided the report: when using a public workspace, it is easy to forget (a) that it is public, and (b) not all researchers are so open with their outputs. Media wiki has some very limited access controls, just about adequate to deal with these situations, but this is an area where other wiki software provides better control.
Project infrastructure
- Generally, the combination of Semantic Medis Wiki and Drupal, running on a VMWare-hosted Scientific Linux system, has worked well for us. Drupal is quite flexible, but not always easy to set up, and some administrative operations can be cumbersome to perform.
- There is a problem of attempted "spam" sign-ups to the Drupal (blogging) system: I adopted the practice of configuring administrator approval required for new accounts, and then ignoring the unrecognized new signup messages. This imposes an unfortunate delay of legitimate new sign-ups. A shared sign-on system covering Media Wiki and Drupal would be preferable.
- VMWare with a Windows host. This was not the best possible choice (though somewhat dictated by available systems). Windows updates often require the host to be rebooted, which in turn requires restarting the Linux guest system. An interactionbetween ZoneAlarm and VMware meant that certain security alerts could not be dealt with remotely (via VNC), which would prevent certain maintenance operations being peformed remotely (especially involving restarting the guest system within VMware, IIRC).
- VMware-based system reliability - we have experienced an ImageWeb outage caused by running out of disk space for the virtual system disks. For a live system, it would probably be better (more reliable) to preallocate virtual disk space rather than configure VMware to extend as needed.
Successes and Failures
- Successes
- we learnt a lot about the repositories and repository systems
- we learnt a lot about web-based tools
- we clarified our vision that a data web approach would be feasible
- we validated the idea of Web as a platform
- we brought new knowledge to our project, such as SPARQL
- we built contacts with other people, organized successful meetings and got people known us better (such as the Google retrievals)
- we validated the wiki philosophy
- we achieved a risk assessment against the project goals
- we have been writing up our work as the project goes on
- Failures and difficulties
- could have done more surveys
- could have got to know more people
- recruitment delays leading to lack of man power
- unable to demonstrate one of the key goals, viz. searching across multiple repository contents
Miscellaneous
These may later be reorganized in other categories.
