We recently had a requirement to save large files from a Linux VM to a separate location, we chose an Azure File Share as a staging area. To achieve this via command line you can use the AzCopy Tool.
To get started you will need an Azure Storage Account – details on how to create one can be found in the quick start article.
- Next you will need a file share:
- From your Azure Storage account Overview page select the File Shares option.
- Select add a file share.
- Name your file share and give it a quota if needed, make a note of the name as this will be needed later.
- We will be using a shared access signature with AzCopy so next step is to generate one.
- Go back to your storage account and choose the “Shared Access Signature” option in the left panel under the settings options.
- Customise the permissions as necessary ensuring read and write is selected, then select the “Generate SAS and connection String”.
- Copy and make a note of the “File service SAS URL” as this will be required later and will not be shown again.
- Now head back to your Linux VM and download AzCopy:
- Download AzCopy
- Expand Archive
- tar -xvf downloadazcopy-v10-linux
- Move AzCopy to the destination you want to store it
- sudo cp ./azcopy_linux_amd64_*/azcopy /usr/bin/
Note you can remove this and previous versions of AzCopy with:
- sudo rm /usr/bin/azcopy
- Now prepare your command, you will need:
- Storage Account Name
- mystorageaccount
- File Share name
- myfileshare
- Local File Name
- .\myTextFile.txt
- SAS URL
- https://mystorageaccount.file.core.windows.net/?sv=2018-03-28&ss=bjqt&srs=sco&sp=rjklhjup&se=2019-05-10T04:37:48Z&st=2019-05-09T20:37:48Z&spr=https&sig=%2FSOVEFfsKDqRry4bk3qz1vAQFwY5DDzp2%2B%2F3Eykf%2FJLs%3D”
- Destination File Name
- myTextFiletxt
- To Upload a File:
azcopy copy “<local-file-path>” https://<storage-account-name>.file.core.windows.net/<file-share-name>/<file-name>?<SAS-token>”
Example:
azcopy copy “.\myTextFile.txt” “https://mystorageaccount.file.core.windows.net/myfileshare/myTextFile.txt?sv=2018-03-28&ss=bjqt&srs=sco&sp=rjklhjup&se=2019-05-10T04:37:48Z&st=2019-05-09T20:37:48Z&spr=https&sig=%2FSOVEFfsKDqRry4bk3qz1vAQFwY5DDzp2%2B%2F3Eykf%2FJLs%3D”
To Download a File:
azcopy copy “https://<storage-account-name>.file.core.windows.net/<file-share-name>/<file-path>?<SAS-token>” “<local-file-path>”
azcopy copy “https://mystorageaccount.file.core.windows.net/myfileshare/myTextFile.txt?sv=2018-03-28&ss=bjqt&srs=sco&sp=rjklhjup&se=2019-05-10T04:37:48Z&st=2019-05-09T20:37:48Z&spr=https&sig=%2FSOVEFfsKDqRry4bk3qz1vAQFwY5DDzp2%2B%2F3Eykf%2FJLs%3D” “.\myTextFile.txt”
Note: you only need the bit after the ? of your SAS URL.
To download/upload files in bulk simply remove the destination file name and use a directory with * wildcard with the –recursive switch.
Use “azcopy -h” for assistance.
Hope this helps