next up previous contents
Next: THOR:Structure Up: Guidelines for offline processing Previous: Offline quality checking

Backup of data

The THOR package was created for a system with only one 4 GB disk for data processing. The available disk space have now been increased with an other 4 GB disk and one 9 GB disk. The amount of data received have on the other hand also increased drastically, so backup of the data has to be done quite often in order not to fill up the disks.

In the old times the data was spread out over the various disks available and a backup was made when a disk became full. This is not a very elegant solution as the other permanent non data files will be backed up every time as well. This means that there will be less space for the data files and a loss on the efficiency per backup tape. Furthermore there will be no structure in the data backup with this procedure and is therefore hard to automize.

All the VSAT data should be backed up twice in case one cassette goes bad. An exabyte tape can hold about 3.6 Mega Blocks(about 1.8 Gigabyte) of data with good security. Two of the existing VSAT disks are smaller than this and can therefore not be used as backup disks. This is yet another argument not to use the old procedure, were backups of those disks resulted in half full tapes. With the present(probably the final) disk space situation the disk space should be divided according to the following scheme:


		 WSDED2$DKA200 		 3.8 		 Data space for the backup to an exabyte tape 

WSDED2$DKA200 3.8 Space for unprocessed DELANA files.

WSDED2$DKA200 0.4 Space for THOR with programs.

WSDED2$DKA300 1.0 Temp scratch area.

WSDED2$DKA400 5.0 Histograms and summary ntuples for all years.

WSDED2$DKA400 3.0 Temp scratch area.

WSDED2$DKA500 17.0 Area for present and previous FX-files.

WSDEA9$DKB0 2.0 User area(shared with users).

WSDE99$DKB500 2.7 User area(shared with STIC and users).


If anyone of these disk should be full, old garbage should be cleaned away. It is not possible to demand that the VSAT shifter should know which files that are important or not. A list of the files on the disks should therefore be sent out to all the involved persons and they should mark the files they want to save. A cleaning program should then be launched which deletes all files that have not been marked. For security it should be demanded that this program does not erase files, that not have been successfully backed up on an exabyte tape.

The procedure of backing up and deleting data should be as automatic as possible. The exabyte station can however not load tapes automatically, so this must be done with real action from the VSAT shifter. A mail should arrive to this person clarifying exactly what to do and how to do it. Inserting a tape in a station is a task that everyone can perform, so the offline procedure for the VSAT detector should be able to run even if there is no shifter at CERN.


next up previous contents
Next: THOR:Structure Up: Guidelines for offline processing Previous: Offline quality checking
Andreas Nygren
1999-11-11