What is CASTOR?

Basic usage

• In order to store your file in CASTOR, use the command rfcp:

        rfcp myfile.data /castor/cern.ch/user/u/username/file.1.data
      

• The usage of CASTOR files is normally very simple. Many CERN applications, in particular CERNLIB (thus PAW, HBOOK etc.) can read CASTOR files directly, so that you can use them - e.g.

  - in PAW:

  h/fil 1 /castor/cern.ch/user/u/username/file.1.paw

  For applications that are not (yet...) CASTOR-aware, you again use rfcp and create a local file:

        rfcp /castor/cern.ch/user/u/username/archive/myfiles.tar recov.tar
      

Main CASTOR commands

nsls

- lists the contents of your CASTOR home directory. Option -l gives a long listing (like standard ls, also some other ls options are available - although not all).

nsrm

- removes a CASTOR file. Options -R will remove a directory and everything below it.

nsmkdir

- creates a subdirectory

nsrename

- changes the file name

nschmod

- changes mode bits of file/directory. Useful if you share a directory with your colleagues.

Good old rfcp transfers a file to CASTOR - but make sure that the file has Unix read permission for "others"! In case of doubt precede this command by chmod a+r myfile.

More information can be found on IT Castor pages. In particular you may want to see the User Guide.

Copying CASTOR-files to/from non-CERN machines

• For not too big amounts of data, and specially for files from your CASTOR home directory (where no other method is possible), the simplest way is to use temporary files in the /tmp directory on a CERN machine and the scp command locally on your home machine. If you use the public cluster at CERN (like LXPLUS), you will have to remember on which particular machine you stored your file!
  • Login to the CERN machine and store your file in /tmp (we will suppose that in the first step below you connected to lxplus013)
    • ssh lxplus.cern.ch
    • rfcp /castor/cern.ch/user/u/username/myfile /tmp/myfile
  • Once your files has been read from CASTOR, in another window on your home machine initiate the transfer
    • scp -pv myuserid@lxplus013.cern.ch:/tmp/myfile /local/myfile
  • When the transfer finished, on the CERN machine delete the temporary file and possibly do the same operation with the next file to be transferred.
    • rm /tmp/myfile
    • logout
  • Please note that:
    • You should rather avoid using AFS (like $SCRATCH or $SCRATCH_WEEK areas) for temporary storage - whenever possible, use a local file (like /tmp): AFS often has very serious problems with handling big files, and this can even lead to the AFS server crash!
    • You must take care not to fill the /tmp area completely, because this may very likely crash your host, or else your files may be removed by the system clean-up: after ~50% of /tmp being used (command df -h /tmp), transfer the files and remove them from /tmp.
    • In the scp command above you may use the wild-characters for the remote files, but they must be enclosed in double-quotes:
      • scp -pv myuserid@lxplus013.cern.ch:/tmp/myfile"*" /localdir/
      The rfcp command does not accept any wild-characters (see however the previous chapter on a trick that emulates them).
• For big amounts of data from /castor/cern.ch/ CASTOR directory, you will probably have to use the new service, GridFTP. Please see the IT prepared documentation for details. Please note that for the authorisation (see the list of bodies issuing the certificates), you should contact the "GRID authority" of your laboratory and your GRID country representative (which will give the authorisation for the LHC-connected activities).

Please note that CASTOR is a complex system, developing very rapidly. The recommendations made on this page reflect our knowledge, but it may well be that users will run into problems that we have not realized. Please give us feedback!