Replacement of Nearline Storage System

Overview

• Nearline storage is configured to have backup of objects on online cache storage system. The objects shall get deleted from the online cache storage system, once objects have been exported to nearline storage system.
• Now assume that the nearline storage (from a different provider, eg. S3) system is also getting full day by day, which calls for replacement of this nearline storage system with a new one, so that the online cache storage system can continue to backup the objects without any disruptions.

Note : The use case here is very similar to Backup of Nearline Storage System. The only difference is that, instead of sending the objects to an external archive (which acts as long term archive) from where they can be retrieved later, the old nearline storage system is replaced altogether.

Configuration

• Assume you already have an online cache storage and nearline storage configuration as shown below.

• Note here that online cache storage is acting as retrieve cache for nearline storage system (dcmRetrieveCacheStorageID: cache1).

• Nearline storage acts as export storage for online cache storage (dcmExportStorageID: nearline1).

• Also, cache1 is referred as Object Storage ID for the DCM4CHEE Application Entity.

    version: 1
    dn: dcmStorageID=cache1,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmInstanceAvailability: ONLINE
    dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
    dcmProperty: checkMountFile=NO_MOUNT
    objectClass: dcmStorage
    dcmStorageID: cache1
    dcmStorageDuration: CACHE
    dcmExportStorageID: nearline1
    dcmURI: ${jboss.server.data.url}/cache1/
    
    dn: dcmStorageID=nearline1,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmInstanceAvailability: NEARLINE
    dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
    dcmProperty: checkMountFile=NO_MOUNT
    objectClass: dcmStorage
    dcmStorageID: nearline1
    dcmStorageDuration: PERMANENT
    dcmRetrieveCacheStorageID: cache1
    dcmURI: ${jboss.server.data.url}/nearline1/
  
    dn: dicomAETitle=DCM4CHEE,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmArchiveNetworkAE
    objectClass: dcmNetworkAE
    objectClass: dicomNetworkAE
    dicomAETitle: DCM4CHEE
    dicomAssociationAcceptor: TRUE
    dicomAssociationInitiator: TRUE
    dicomNetworkConnectionReference: cn=dicom,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dicomNetworkConnectionReference: cn=dicom-tls,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmAcceptedUserRole: user
    dcmAcceptedUserRole: admin
    dcmMetadataStorageID: metadata
    dcmObjectStorageID: cache1
    dcmQueryRetrieveViewID: hideRejected
    dcmUpdateLocationStatusOnRetrieve: FALSE
    dicomDescription: Hide instances rejected for Quality Reasons
  

Replacement of a nearline storage system is explained below as a 2 step process.

Add a new nearline storage

• To do this, we add a new online cache storage (call it cache1b), a new nearline storage and a separate retrieve cache storage system.
• Refer the new online cache storage on DCM4CHEE Application Entity (dcmObjectStorageID: cache1b)
• Newly created Retrieve cache storage is acting as retrieve cache for the new nearline storage system (dcmRetrieveCacheStorageID: rc).
• New nearline storage acts as export storage for the new online cache storage (dcmExportStorageID: nearline2).

Refer to below configurations.

    version: 1
    dn: dcmStorageID=cache1b,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmInstanceAvailability: ONLINE
    dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
    dcmProperty: checkMountFile=NO_MOUNT
    objectClass: dcmStorage
    dcmStorageID: cache1b
    dcmStorageDuration: CACHE
    dcmExportStorageID: nearline2
    dcmURI: ${jboss.server.data.url}/cache1b/
    
    dn: dcmStorageID=rc,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmInstanceAvailability: ONLINE
    dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
    dcmProperty: checkMountFile=NO_MOUNT
    objectClass: dcmStorage
    dcmStorageID: cache1b
    dcmStorageDuration: CACHE
    dcmURI: ${jboss.server.data.url}/rc/        
    
    dn: dcmStorageID=nearline2,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmInstanceAvailability: NEARLINE
    dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
    dcmProperty: checkMountFile=NO_MOUNT
    objectClass: dcmStorage
    dcmStorageID: nearline2
    dcmStorageDuration: PERMANENT
    dcmRetrieveCacheStorageID: rc
    dcmURI: ${jboss.server.data.url}/nearline2/

    dn: dicomAETitle=DCM4CHEE,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmArchiveNetworkAE
    objectClass: dcmNetworkAE
    objectClass: dicomNetworkAE
    dicomAETitle: DCM4CHEE
    dicomAssociationAcceptor: TRUE
    dicomAssociationInitiator: TRUE
    dicomNetworkConnectionReference: cn=dicom,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dicomNetworkConnectionReference: cn=dicom-tls,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmAcceptedUserRole: user
    dcmAcceptedUserRole: admin
    dcmMetadataStorageID: metadata
    dcmObjectStorageID: cache1b
    dcmQueryRetrieveViewID: hideRejected
    dcmUpdateLocationStatusOnRetrieve: FALSE
    dicomDescription: Hide instances rejected for Quality Reasons

Also, ensure to change the value of retrieve cache storage ID on nearline1 storage system to reflect newly created retrieve cache (dcmRetrieveCacheStorageID: rc), as shown below :

    dn: dcmStorageID=nearline1,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    dcmInstanceAvailability: NEARLINE
    dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
    dcmProperty: checkMountFile=NO_MOUNT
    objectClass: dcmStorage
    dcmStorageID: nearline1
    dcmRetrieveCacheStorageID: rc
    dcmURI: ${jboss.server.data.url}/nearline1/

Replacing old nearline storage system

• Once the new online cache storage, nearline storage and retrieve cache storage have been added, create an exporter to
  export the objects from nearline1 to nearline2.

• One may additionally choose to delete objects on old nearline storage system (i.e nearline1) by adding dcmDeleteStudyFromStorageID: nearline as shown below.

    dn: dcmExporterID=CopyToNearlineStorage2,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmExporter
    dcmExporterID: CopyToNearlineStorage2
    dcmQueueName: Export5
    dcmURI: storage:nearline2
    dcmDeleteStudyFromStorageID: nearline1
    dicomAETitle: DCM4CHEE
    dicomDescription: Copy to NEARLINE Storage2
  

• Export the objects from nearline1 to nearline2 using RESTful service Export Matching Studies and passing the query parameter storageID=nearline1