BREAKING CHANGE: Upcoming change to blob storage

[jovinson-ms]

Because of our priority of safeguarding personal health information (PHI), we've made the decision to change the way that we store DICOM files. Currently, DICOM files are stored as blobs named using DICOM UIDs - study, series, and instance for images, and instance for workitems. While this is commonly done within DICOM implementations, UIDs often contain information about the creating organization as the org root, and may include personal information about the context of their creation, such as patient information or identifiers. This makes UIDs a risk for exposing protected information inadvertently.

Starting with #1549, we are beginning the process of changing to a new format that has two benefits: we will use system-generated identifiers to protect personal information, and we will add a prefix to achieve better performance when writing to blob storage under certain conditions. This process will involve the following steps:

1. We will introduce the ability to read and write blobs using both formats. This ability will not be enabled by default. All existing OSS deployments will continue to work normally.
2. We will introduce a migration utility that will copy all blobs from the old format to the new format. This utility will run on Azure Functions using the same infrastructure as extended query tag reindexing operations.
3. In a future version, we will bump the major version of our Nuget package, and only use the new format from that point. All OSS deployments will need to have run the migration operation or deleted files stored in the old format in order to upgrade.

Note: we do not plan to provide a migration path for UPS-RS workitems since that feature is still in development.

While this is a breaking change, we do believe that the benefits are substantial and worth the cost in engineering and data migration. We'd like to hear your thoughts and questions below!

[bcarthic]

With V10.0.0, all new blobs will be using new blob format, and this break the service. In order to migrate old blobs, please follow blob migration documentation.