YADE should support file transfer to Azure blob storage

Current Situation

• The YADE can be used to transfer files from and to Azure blob storage if no authentication is required and blobs are publicly available.
• The YADE does not support Azure authentication methods for access to blob storage.

Feature

Consider the YADE support for Azure Blob Storage article

Azure Authentication Methods

1. Anonymous public read access
2. Shared Key (Storage Account keys)
3. Shared Access Signature (SAS tokens)
4. Azure Active Directory
5. Active Directory

YADE should support the first three methods of the above list.

Azure Storage Types
The Azure Storage platform is Microsoft's object storage solution for the cloud. The storage services provided by Azure include:

1. Blob Storage
2. Azure Files
3. Azure Queues
4. Azure tables
5. Azure Disks

Blob Storage
Blob Storage is optimized for storing massive amounts of unstructured data. It can be accessed using HTTP or HTTPS. Azure Blobs are accessible via URL, Azure Storage REST API, Azure PowerShell, Azure CLI, and Azure Storage client library. Blob Storage is the storage type that will be supported by YADE.

File Transfer Operation

• Upload a file to Azure blob Storage
  • Uploading a file to an Azure blob using the Azure REST API requires authentication which includes to use a Shared Key or a Shared Access Signature (SAS Token).
    • A Shared Key can be generated by a user that is added with the Authorization header of the REST request.
    • An SAS token can be generated by a user specifically for access to a specific container or blob. This SAS token is passed to the URL as a query parameter.
    • In addition to the Shared Key or SAS token a number of headers are required
      • x-ms-blob-type: BlockBlob
      • x-ms-date : <date of request>
      • x-ms-version : <version of Azure REST API>
  • The content of the file to be uploaded is added with the body of the REST request.
  • Example:

    curl -X PUT \
      '<Account URL>/<container_name>/<file_name><SAS_token>' \
      -H 'x-ms-blob-type: BlockBlob' \
      -d 'This is test content'

• Download a file from Azure blob Storage
  • When creating a YADE profile with the particular URL and Basic Authentication in the HTTPS protocol fragment, then the same Azure authentication methods should be applied as for upload of files. This includes support for Shared Keys and SAS Tokens and respective use of HTTPS headers.

File Transfer Configuration

• YADE XSD Schema
  • The XSD Schema adds a node to specify HTTPS headers and URL query parameters. This node is available at a general level in the XSD Schema and at a per profile level.
  • The Directory setting is mapped to the Blob Container.
  • The FilePath, FileSpecification and FileList settings from a file selection are mapped to the Blob.