dCache 1.8 pool upgrade

About NDGF

About
People
Meetings
Forums
Steering Board

Documents

Presentations
Technical
Managemental
Links

Activities

Planning
Operation
Middleware
Tier-1

e-Science Projects

CERN
BioGrid
CO2
CC-VO
MatSC


Added by Mattias Wadenstein, last edited by Gerd Behrmann on Oct 29, 2007  (view change)

Labels:

Enter labels to add to this page:
Wait Image 
Looking for a label? Just start typing.

Information about the 1.8 pool upgrade

The upgrade to dCache 1.8 brings a couple of big new changes which means there will be a few more things to configure than just a normal minor upgrade. Because of this we recommend that you copy the dCacheSetup.template again and re-apply your local changes, since there is quite a number of things that have changed there.

 Download 1.8

The 1.8.0 release is available at:  http://www.dcache.org/downloads/1.8.0/index.shtml

 Direct link to the appropriate rpm: http://www.dcache.org/downloads/1.8.0/dcache-server-1.8.0-0.noarch.rpm

  dCache upgrade steps

For more detailed information about the how and why of all the setup steps, please see the "normal" page: http://wiki.ndgf.org/display/ndgfwiki/Technical-DCache+PoolInstallation

  • Make sure dCache is shut down
  • Optional but recommended: install java6
  • Install (or unpack and copy) the RPM, please see note about batch file below
  • If $PREFIX does not equal /opt/d-cache/, then please follow the instructions above for installing in a non-default directory.
  • Check/reset file ownership
  • Get a new dCacheSetup from the template and re-apply your changes
  • Configure the metadata repository and import settings in dCacheSetup
  • Make sure you do not have a pool_path file (unless you want to add new pools)
  • Run $PREFIX/install/install.sh
  • Check that the $PREFIX/config/.domains and $PREFIX/config/.poollist files are correct (install.sh should not change them, so everything ought to be fine)
  • Start the pool
  • Once everything is working fine, disable the metadata import from "file" in dCacheSetup

  New dCacheSetup.template

Please copy the new dCacheSetup.template from the etc directory and follow the install instructions (or a backup copy of your current dCacheSetup) to re-apply all your local settings. Lots of things have changed, so it is best to get all the new defaults and comments about settings.

pool.batch

Notice that the pool.batch file in the config directory has changed in this upgrade. If you did not modify you local copy, you should automatically get the new version when installing the RPM. If you did changes, you should take the new copy and merge your changes into it.

Please check that you have the line

create diskCacheV111.pools.MultiProtocolPoolV3 ${0} \

This indicates that you have the new version of the file (the V3 is the important part).

 New meta data repository

As of dCache 1.8 there is a new way of storing the metadata about files out on the pools. Instead of two files in control/ for every file stored in data/ there is a Java berkeley db implementation that stores this data. This should help the issue of slow pool startups that we have seen for pools with lots of files in them. This is enabled in dCacheSetup by setting:

 metaDataRepository=org.dcache.pool.repository.meta.db.BerkeleyDBMetaDataRepository

There is also an import section which is needed for the upgrade, but should be removed after the upgrade so in case of meta data loss dCache will go to the central node and ask pnfs for the metadata about each file, instead of trying to import it from a possibly stale location. The import will read all the files in control/ and then write the information to the berkeley db, which may take some time (but shouldn't take significantly longer than a "normal" pool startup).

So, for the upgrade set:

metaDataRepository=org.dcache.pool.repository.meta.db.BerkeleyDBMetaDataRepository
metaDataRepositoryImport=org.dcache.pool.repository.meta.file.FileMetaDataRepository

After the pool has started and come back online (you can follow the progress of the metadata import in the pool log file), comment out the import section:

metaDataRepository=org.dcache.pool.repository.meta.db.BerkeleyDBMetaDataRepository
# metaDataRepositoryImport=org.dcache.pool.repository.meta.file.FileMetaDataRepository

Just so that you won't forget the import line active.