From ImageWeb

Contents 1 Configuration notes for hosting live ADMIRAL services 1.1 Virtualization 1.2 Compute requirements 1.3 Storage requirements

Configuration notes for hosting live ADMIRAL services

Virtualization

I've been proceeding on the assumption of deploying a separate virtual machine for each research group. I was challenged about this the other day: if each group is to use roughly the same services, would it be simpler and more efficient to use the same environment to look after research data for all the groups? My position on using virtualization has evolved over several years - I want to minimize the amount of hardware I have to look after, preferably to zero - so it's timely that I review my position vis-a-vis hosting ADMIRAL services.

On reflection, I think the original approach is the right one. Set against the undoubted (though modest) efficiency and complexity costs of adding a virtualization layer are:

• Confidentiality of data: some research groups create commercially sensitive data whose confidentiality needs to be assured. Having a separate host environment for each research group minimizes opportunities for leakage of data between research groups, and generally allows different groups to operate different levels of stringency in their access control policies.
• Flexibility of resource allocation: shared hardware resources (especially disk capacity) can be allocated to virtual machines according to the research group needs, avoiding the possibility that one group overusing its available capacity will cause problems for another.
• Expansion and migration: new hardware capacity can be allocated to different research groups as required, and if it becomes necessary to move one group to an enlarged facility this can be achieved with minimal disruption.
• Administrative control - each hosted environment can be under the full administrative control of its respective research group, avoiding the need to depend on an external agency for managing access permissions.
• Different evolving requirements. Although we plan to start by deploying the same services for each research group, it is entirely possible that, over time, they may require a different arrangement of services. Having separate environments makes this easier to achieve.
• Local IT support and operational oversight: the ADMIRAL facilities are intended to outlive the ADMIRAL project. The local IT support are familiar with running a virtualization environment, so will be able to assist with operational maintenance issues (e.g. dealing with and recovering from hardware failures) even if we are not around to help. (This is apart from supporting the software systems, which has been discussed elsewhere.)

Compute requirements

I am working on the assumption that each research group will require services hosted using the equivalent of 2x2GHz 64-bit processors and 4Gb RAM. Avoiding over-commitment of resources, this suggests a server with dual quad-core processors and at least 16Gb RAM to support 4 groups. If economically possible, additional RAM (32Gb) should allow increased performance by reducing levels of disk I/O when systems are loaded.

Storage requirements

Different research groups have very different storage requirements. Allowing (say) 200Gb as a core requirement, our surveys to date suggest this will easily serve day-to-day needs of all research groups. But two of the groups also create large volumes of video and other image data, which does not need to be accessed frequently or at high speed.

Thus, we are contemplating that the server system will be provisioned with about 1Tb of fast SAS (Serial attached SCSI) disk storage, which will be sufficient for the virtual hosting environment, guest operating systems, installed software and day-to-day software use.

Additional, lower performance, disk capacity can be added at much lower cost using Networked Attached Storage systems using Serial ATA disks.

All disks will be configured as RAID5 3+1 arrays (thereby yielding formatted capacity a little under 75% of the raw disk capacity, and providing redundancy and online recoverability of data in the face of single disk failures)

Budgetary costs are £4-5K for the base server, and £2K for 8Tb of networked attached storage.

An alternative to slower disk capacity is to use the University's HFS to create a long-term archive of the research data (as opposed to shory-term backups, which we already pan to do). This should probably be done in any case (i.e. even if it is held on local disk storage). Data held on HFS archives can be retrieved with a few days notice - it remains to be seen if this will be considered adequate by our test users.