• Component list
• Quality Assurance Checks
• RPM list
• Release notes
• Documentation
• Testing

Component list

The following components are released:

Servers:

• edg-local-replica-catalog
• edg-replica-location-index
• edg-replica-metadata-catalog
• edg-replica-optimization
• edg-spitfire-server

• edg-replica-manager
• edg-local-replica-catalog-client
• edg-replica-location-index-client
• edg-replica-metadata-catalog-client
• edg-replica-optimization-client
• edg-spitfire-client

• edg-java-security
• edg-java-data-util
• edg-gsoap-base

• edg-lcfg-rm
• edg-lcfg-lrc
• edg-lcfg-rli
• edg-lcfg-rmc
• edg-lcfg-ros

Quality Assurance Checks

The following QAG check documents have been prepared

• edg-local-replica-catalog [doc]
• edg-replica-location-index [doc]
• edg-replica-metadata-catalog [doc]
• edg-replica-optimization [doc]
• edg-replica-manager [doc]

RPM list

Since Release 2.1 will move to gcc3.2, we only consider the gcc3.2 RPMs. This is only relevant for the i386 RPMs anyway and most of our code is architecture-independent, marked by noarch. In the table below X needs to be substituded by the version at the right-hand-side - except for the interface versions (Y) which are separately versioned.
 

CVS Module	Generated RPMs	Version
edg-gsoap-base	edg-gsoap-base-X.i386.rpm	1.0.8-1
edg-java-security	edg-java-security-X.noarch.rpm edg-java-security-client-X.noarch.rpm edg-java-security-tomcat4-X.noarch.rpm edg-java-security-test-X.noarch.rpm	1.5.2-1
edg-java-data-util	edg-java-data-util-X.noarch.rpm edg-java-data-util-config-X.noarch.rpm	1.3.3-1
edg-rls-server	edg-local-replica-catalog-interface-Y.noarch.rpm edg-local-replica-catalog-X.noarch.rpm edg-local-replica-catalog-config-X.noarch.rpm edg-local-replica-catalog-client-X.noarch.rpm edg-local-replica-catalog-test-X.noarch.rpm	Y: 2.1.0-1 X: 2.1.2-1
edg-metadata-catalog	edg-replica-metadata-catalog-interface-Y.noarch.rpm edg-replica-metadata-catalog-X.noarch.rpm edg-replica-metadata-catalog-config-X.noarch.rpm edg-replica-metadata-catalog-client-X.noarch.rpm edg-replica-metadata-catalog-test-X.noarch.rpm	Y: 2.1.0-1 X: 2.1.2-1
edg-replica-location-index	edg-replica-location-index-interface-Y.noarch.rpm edg-replica-location-index-X.noarch.rpm edg-replica-location-index-config-X.noarch.rpm edg-replica-location-index-client-X.noarch.rpm edg-replica-location-index-test-X.noarch.rpm	Y: 2.1.0-1 X: 2.1.2-1
edg-ros	edg-replica-optimization-interface-Y.noarch.rpm edg-replica-optimization-X.noarch.rpm edg-replica-optimization-config-X.noarch.rpm edg-replica-optimization-client-X.noarch.rpm edg-replica-optimization-test-X.noarch.rpm	Y: 2.1.0-1 X: 2.1.2-1
edg-reptor	edg-replica-manager-interface-Y.noarch.rpm edg-replica-manager-X.noarch.rpm edg-replica-manager-config-X.noarch.rpm edg-replica-manager-test-X.noarch.rpm  edg-replica-manager-gridftp-client_gcc3_2_2-X.i386.rpm	Y: 1.5.0-1 X: 1.5.1-1
edg-rls-client	edg-replica-location-client-c++_gcc3_2_2-X.i386.rpm	2.1.1-1
edg-metadata-catalog-client	edg-replica-metadata-catalog-client-c++_gcc3_2_2-X.i386.rpm	2.1.0-1
edg-ros-client	edg-replica-optimization-client-c++_gcc3_2_2-X.i386.rpm	2.1.0-1
edg-replica-location-index-client	edg-replica-location-index-client-c++_gcc3_2_2-X.i386.rpm	2.1.0-1
edg-reptor-client	edg-replica-manager-client-c++_gcc3_2_2-X.i386.rpm	2.1.0-1
edg-spitfire-server	edg-spitfire-server-X.noarch.rpm edg-spitfire-client-java-X.noarch.rpm edg-spitfire-test-X.noarch.rpm	2.1.7-1
edg-spitfire-client	edg-spitfire-client-c++-X.noarch.rpm	2.1.6-1

LCFGng
 

CVS Module	Generated RPMs	Version
edg-lcfg-lrc	edg-lcfg-lrc-X-edg1.noarch.rpm edg-lcfg-lrc-defaults-s1-X-edg1.noarch.rpm	2.1.6
edg-lcfg-javasecurity	edg-lcfg-javasecurity-X-edg1.noarch.rpm edg-lcfg-javasecurity-defaults-s1-X-edg1.noarch.rpm	1.3.2
edg-lcfg-rmc	edg-lcfg-rmc-X-edg1.noarch.rpm edg-lcfg-rmc-defaults-s1-X-edg1.noarch.rpm	2.0.6
edg-lcfg-ros	edg-lcfg-ros-X-edg1.noarch.rpm edg-lcfg-ros-defaults-s1-X-edg1.noarch.rpm	3.0.6
edg-lcfg-rli	edg-lcfg-rli-X-edg1.noarch.rpm edg-lcfg-rli-defaults-s1-X-edg1.noarch.rpm	1.0.3
edg-lcfg-rm	edg-lcfg-rm-X-edg1.noarch.rpm edg-lcfg-rm-defaults-s1-X-edg1.noarch.rpm	1.0.9

Release notes

Compatibility

Some slides discussing the different versions of the software and compatibility issues.

Replica Manager Configuration File

The replica manager configuration files are now not in $EDG_LOCATION/etc/edg-replica-manager/ but $EDG_LOCATION_VAR/etc/edg-replica-manager.

The format of the edg-replica-manager.conf file also has changed. Affected parameters are:
 

old name	old values	new name	new values
class.globusURLCopy	org.edg.data.reptor.storage.GlobusURLCopyNativeImpl org.edg.data.reptor.storage.GlobusURLCopyCoGImpl	impl.globusURLCopy	Native CoG
class.gridFTP	org.edg.data.reptor.storage.GridFTPNativeImpl org.edg.data.reptor.storage.GridFTPCoGImpl	impl.gridFTP	Native CoG
class.infoService	org.edg.data.reptor.info.InfoServiceRGMA org.edg.data.reptor.info.InfoServiceMDS org.edg.data.reptor.info.InfoServiceStub	impl.infoService	RGMA MDS Stub

The default value has changed for
 

name	old value	new value
library.gridFTP	edg_rm_gridftp	edg_rm_gridftp_gcc3_2_2

There is a new option of relevance, specifying whether we use the distributed RLS or the single central one
 

name	values
impl.ReplicaCatalog	LrcOnly WithRli

 

RMC, LRC schema upgrade

The upgrade to release 2.1 of the LRC and the RMC involves a schema change. This means that a migration script needs to be run on these databases. We provide a script for each of these databases. The change involves the different handling of the attributes. The way this migration is carried out, it is still possible to roll back if the upgrade would fail so it is recommended to run it on each database. We also provide scripts for both MySQL and Oracle.

• MySQL

LRC migration script lrc-mysql-migrate-2.1.pl

RMC migration script rmc-mysql-migrate-2.1.pl

These scripts are written in Perl/DBI so you need to run it on a node (it needn't be the same machine where the database resides) where you have DBD::mysql installed. Each script takes 4 arguments:

host  The name of the host where the DB runs
db     The db name, usually edg_VO_lrcdb
user The name of the user, usually edg_VO_lrc
pwd The password given to the user.

If you don't remember the names/passwords they can usually be fished out of the xml files on the server residing in /var/tomcat4/webapps.
• Oracle

  LRC migration script lrc-oracle-migrate-2.1.sql

  RMC migration script rmc-oracle-migrate-2.1.sql

  These scripts are written in PL/SQL. Copy these scripts in the appropriate place and run them :

  sqlplus user/pwd@db < lrc-oracle-migrate-2.1.sql
  sqlplus user/pwd@db < rmc-oracle-migrate-2.1.sql

  with the right values :

  db     The db name, usually edg_VO_lrcdb
  user  The name of the user, usually edg_VO_lrc
  pwd  The password given to the user.

LRC SRM to SFN migration

The upgrade to release 2.1 involves a change in the naming of SURLs to accomadate requirmenrts from GFAL. The URI schema 'srm:' will be reserved for SRM compliant implementations and for WP5 EDG-SE. All other SE types (by which we currently mean GridFTP servers or 'classicSE's should be changed to use the scheme 'sfn:' in the database. The script below does this.

We provide scripts for both MySQL and Oracle.

• MySQL

LRC SURL migration script lrc-mysql-migrate-srm-sfn.pl

These script is again written in Perl/DBI so you need to run it on a node (it needn't be the same machine where the database resides) where you have DBD::mysql installed. The script takes 5 arguments:

host  The name of the host where the DB runs
db     The db name, usually edg_VO_lrcdb
user The name of the user, usually edg_VO_lrc
pwd The password given to the user.
classicSEList A file containing the list of CLASSIC SEs.

If you don't remember the names/passwords they can usually be fished out of the xml files on the server residing in /var/tomcat4/webapps.

The classicSEList file should have one SE hostname per line. SURL entries in the catalog that match these hosts will have their scheme changed from 'srm:' to 'sfn:'. The other SURL entries will not be chnaged.

Note: the RMC is not involved in this operation at all.

• Oracle

  LRC SURL migration script lrc-oracle-migrate-srm-sfn.pl

  These scripts are written in PL/SQL. Copy these script into the appropriate place and run it :

sqlplus user/pwd@db < lrc-oracle-migrate-srm-sfn-2.1.sql

with the right values :

db     The db name, usually edg_VO_lrcdb
user  The name of the user, usually edg_VO_lrc
pwd  The password given to the user.
classicSEList A file containing the list of CLASSIC SEs.

The classicSEList file should have one SE hostname per line. SURL entries in the catalog that match these hosts will have their scheme changed from 'srm:' to 'sfn:'. The other SURL entries will not be chnaged.

Note: the RMC is not involved in this operation at all.

Security

The replica manager will consider the VOMS certificate's VO content so the --vo option is not needed if you work using a VOMS certificate. We suggest to install all services in INSECURE mode because at this time the c++ client interface still has issues talking in secure mode with the services. In order to have some tests using secure mode we allow for dual secure/insecure mode installation (configure both secure and insecure options in LCFG). There will be two endpoints to the same database, one secure and one insecure. However, this mode should only be used for testing and preferably not in production mode.

Notes on LRC + RMC for versions >=2.2.1

There are some additions to the API:

• getMappingByAttribute() - support for guid, alias and pfn queries
  
  The getMappingByAttribute takes an SQL query to select mappings. this now can include matches against the guid/alias/pfns themselves.
  
  The syntax is as follows:
  
  GUID : "guid.guid_guid LIKE 'guid:foo%'"
  PFN : "pfn.pfn_pfn LIKE 'pfn:foo%'"
  ALIAS : "alias.alias_alias LIKE 'alias:foo%'"
  
  These can also be exact matches ("guid.guid_guid = 'guid:foo'") and also can be more complicated queries : "guid.guid_guid LIKE 'guid:foo%' AND guid.size > 1024 AND guid.tag = 'foo'");

• New get mapping with attributed commands
  
  There are now new commands for getting mappings along with their Attributes in the C++/Java API. Note that these operations are not yet available in the commandline client.
  
  The signatures are
  
  org.edg.data.util.attribute.Mapping[] getMappingsWithAttribute(String SQLQuery, int length);
  
  and
  
  org.edg.data.util.attribute.Mapping[] getMappingsWithAttribute(String SQLQuery, int length, int start);
  
  So where you would do code as follows:
  
  String[] mappings = m_stub.getMappings("guid.guid_guid = 'guid:foo'", 1);
  String guid = mappings[0];
  String pfn = mappings[1];
  AttributeDefinition[] defns = m_stub.getAttributeDefinitions();
  for(int i = 0; i < defns.length; ++i) {
  // get pfn attribute value using one of the
  // getPfn...Attribute() methods
  }
  
  you can now just do:
  
  Mapping[] mappings = m_stub.getMappingsWithAttribute("guid.guid_guid =
  'guid:foo'", 1);
  String guid = mappings[0].getGuid();
  String pfn = mappings[0].getEntry();
  String[] pfnAttributes = mappings[0].getAttributes();
  String[] pfnAttributeNames = mappings[0].getPfnAttributes();
  
  
  It works identically for the RMC, where the getEntry() method now returns the alias, not the pfn and you can also get at the guid attributes through the getGuidAttributes() and getGuidAttributeNames() methods.
  

Documentation

All documentation is autobuilt also in Lyon and is available there.

Testing

For each of the packages, there exit a few test suites. Some example are given below:

• EDG Replica Manager Integration & API tests:

> /opt/edg/sbin/test/edg-replica-manager-testJavaAPI cms
Start Replica Manager test suite

test copyFile
test registerFile
test listReplicas
test replicateFile
test listBestFile
test getBestFile
test getTurl
test deleteFile
test copyAndRegisterFile
test copyAndRegisterFile
test listGUID
test addAlias
test listGUID
test removeAlias

Test Result
===========
copyFile             : OK
registerFile         : OK
listReplicas         : OK
replicateFile        : OK
listBestFile         : OK
getBestFile          : OK
getTurl              : OK
deleteFile           : OK
copyAndRegisterFile  : OK
listGUID             : OK
addAlias             : OK
removeAlias          : OK

Integration tests
  - Interaction with Replica Location Service     : OK
  - Interaction with Replica Metadata Catalogue   : OK
  - Interaction with Replica Optimization Service : OK
  - Interaction with Storage Elements             : OK
  - Interaction with the Information Service      : OK
 

> /opt/edg/sbin/test/edg-replica-manager-cli-test/edg-replica-manager-testCLI --vo=cms

 edg-rm printInfo test: this must pass to run the rest of the test suite
 4 CEs and 13 SEs found: printInfo test passed

edg-rm cli tests started - output in /tmp/edg-replica-manager-testcli-46221-4556.log

edg-rm-addAlias-test : OK
edg-rm-copyAndRegisterFile-test : OK
edg-rm-copyFile-test : OK
edg-rm-deleteFile-test : OK
edg-rm-getAccessCost-test : OK
edg-rm-getBestFile-test : OK
edg-rm-getVersion-test : OK
edg-rm-listBestFile-test : OK
edg-rm-listGUID-test : OK
edg-rm-listReplicas-test : OK
edg-rm-registerFile-test : OK
edg-rm-removeAlias-test : OK
edg-rm-replicateFile-test : OK

edg-rm cli tests finished 
 

/opt/edg/sbin/test/edg-local-replica-catalog-testCLI -i --vo=wpsix --host lxshare0343.cern.ch
edg-lrc cli tests started - output in /tmp/edg-local-replica-catalog-testcli-505-16942.log

edg-lrc-addTooLong-error-test : OK
edg-lrc-addAttrDefn-test : OK
edg-lrc-addMapping-test : OK
edg-lrc-existence-error-test : OK
edg-lrc-attrDefnExists-test : OK
edg-lrc-getResultLength-test : OK
edg-lrc-getPfnAttr-test : OK
edg-lrc-listAttrDefns-test : OK
edg-lrc-getVersion-test : OK
edg-lrc-guidExists-test : OK
edg-lrc-guidForPfn-test : OK
edg-lrc-mappingsByPfnAttr-test : OK
edg-lrc-mappingsByGuid-test : OK
edg-lrc-mappingsByPfn-test : OK
edg-lrc-pfnsForGuid-test : OK
edg-lrc-maxLength-test : OK
edg-lrc-pfnExists-test : OK
edg-lrc-removeAttrDefn-test : OK
edg-lrc-ping-test : OK
edg-lrc-removePfnAttr-test : OK
edg-lrc-removePfn-test : OK
edg-lrc-setResultLength-test : OK
edg-lrc-setPfnAttr-test : OK

edg-lrc cli tests finished 
 

 /opt/edg/sbin/test/edg-replica-metadata-catalog-testCLI -i --vo=wpsix --host lxshare0343.cern.ch
edg-rmc cli tests started - output in /tmp/edg-replica-metadata-catalog-testcli-505-4441.log

edg-rmc-addAliasAttrDefn-test : OK
edg-rmc-addAlias-test : OK
edg-rmc-aliasAttrDefnExists-test : OK
edg-rmc-addGuidAttrDefn-test : OK
edg-rmc-addTooLong-error-test : OK
edg-rmc-existence-error-test : OK
edg-rmc-aliasExists-test : OK
edg-rmc-aliasesForGuid-test : OK
edg-rmc-getResultLength-test : OK
edg-rmc-getAliasAttr-test : OK
edg-rmc-getGuidAttr-test : OK
edg-rmc-guidAttrDefnExists-test : OK
edg-rmc-getVersion-test : OK
edg-rmc-guidForAlias-test : OK
edg-rmc-guidExists-test : OK
edg-rmc-mappingsByAliasAttr-test : OK
edg-rmc-listAliasAttrDefns-test : OK
edg-rmc-listGuidAttrDefns-test : OK
edg-rmc-mappingsByAlias-test : OK
edg-rmc-mappingsByGuidAttr-test : OK
edg-rmc-mappingsByGuid-test : OK
edg-rmc-removeAlias-test : OK
edg-rmc-maxLength-test : OK
edg-rmc-ping-test : OK
edg-rmc-removeAliasAttrDefn-test : OK
edg-rmc-removeAliasAttr-test : OK
edg-rmc-removeGuidAttrDefn-test : OK
edg-rmc-removeGuidAttr-test : OK
edg-rmc-setResultLength-test : OK
edg-rmc-setAliasAttr-test : OK
edg-rmc-setGuidAttr-test : OK

edg-rmc cli tests finished 
 

> /opt/edg/sbin/test/edg-replica-optimization-testCLI -i --host=lxshare0382.cern.ch --vo=cms
edg-ros cli tests started - output in /tmp/edg-replica-optimization-testcli-46221-17899.log

edg-ros-getNetworkCost-test : OK
edg-ros-getVersion-test : OK
edg-ros-ping-test : OK

edg-ros cli tests finished 

.