Fetch files with HTTPS
Files can be fetched with HTTP from files.unifaun.se.
Note
You should always use HTTPS.
There are two methods of authentication. The authentication method is specified with the “auth” parameter. If the “auth” parameter is omitted and no “Authorization” HTTP header is sent, the authentication is set to HTTP Digest.
HTTP Digest access authentication
The default authentication mode. The advantage is that the password is not transmitted unmodified to the web server which makes it safer than basic authentication.
Set the parameter “auth” to “DIGEST” or don't send the parameter.
Note
User name should be prefixed with “UO.”, for example, UO.002000111.
Please refer to https://tools.ietf.org/html/rfc2617.
HTTP Basic access authentication
User name and password are sent in the HTTP header.
Caution
This method is only safe if HTTPS is used.
Set the parameter “auth” to “BASIC”. The server will also try to automatically detect this authentication method using the HTTP header “Authorization”.
Please refer to https://www.w3schools.com/tags/ref_urlencode.asp.
Fetching files
List the files in a directory
Use an HTTP GET with the directory path as URL. The parameter view=SIMPLETEXT returns a result in plain text that is easy to interpret. URLs should use UTF-8 character set encoding and standard “URL encoding”. Each row in the result represents a file and is terminated by a line feed (0x0A). The encoding of the result is UTF-8.
GET /files/?view=SIMPLETEXT HTTP/1.1 File1.txt File2.txt File3.txt
Fetch files
Use an HTTP GET with the file path as URL.
GET /files/File1.txt?view=SIMPLETEXT HTTP/1.1
Delete files
Use an HTTP DELETE with the file path as URL.
DELETE /files/File1.txt?view=SIMPLETEXT HTTP/1.1
Another method for deleting the file is to use an HTTP POST with the file path as URL and a parameter “action” with the value “delete”:
POST /files/File1.txt?view=SIMPLETEXT&action=delete HTTP/1.1
Using Wget
GNU Wget is a command line utility that is available both for Microsoft Windows and Unix-like operating systems.
Please refer to http://www.gnu.org/software/wget/ and http://gnuwin32.sourceforge.net/packages/wget.htm. The following script for Unix (bash shell) fetches all files that end with “.txt” from a directory and deletes the files from the source.
#!/bin/bash HOST='user:password@files.unifaun.se' DIRECTORY='out' PARAMS='view=SIMPLETEXT' FILES=$(wget -qO - "https://$HOST/$DIRECTORY/?$PARAMS" | grep -i ".txt$") echo "Fetching files:" $FILES for FILE in $FILES; do wget -qO - "https://$HOST/$DIRECTORY/$FILE" > $FILE && \ wget –qO - "https://$HOST/$DIRECTORY/$FILE" \ --post-data='action=delete' > /dev/null; done
The command wget -nc -nd -r -l1 --no-parent -A "*.txt " \
will fetch all files in the “out” directory that end with “.txt” and haven't already been fetched.
https://user:pwd@files.unifaun.se/out/?view=SIMPLEHTML
Using curl
Curl is a command line utility that is available on several operating systems. Please refer to http://curl.haxx.se/.
The following script for Unix (Bash shell) fetches all files that end with “.txt” from a directory and deletes the files from the source.
#!/bin/bash HOST='user:password@files.unifaun.se' DIRECTORY='out' PARAMS='view=SIMPLETEXT' FILES=$(curl -sS "https://$HOST/$DIRECTORY/?$PARAMS" | grep -i ".txt$") echo "Fetching files:" $FILES for FILE in $FILES; do curl -OfsS "https://$HOST/$DIRECTORY/$FILE" && \ curl -fsS "https://$HOST/$DIRECTORY/$FILE" \ --form-string "action=delete" > /dev/null; done
The following script for Windows (Powershell) fetches all files that end with “.txt” from a directory and deletes the files from the source.
$TARGETHOST="user:password@files.unifaun.se" $DIRECTORY="out" $PARAMS="view=SIMPLETEXT" $FILES = @(curl -sS "https://$TARGETHOST/$DIRECTORY/?$PARAMS" ` | select-string -pattern "\.txt$") if ($FILES.length -gt 0) { write-host "Fetching files:" $FILES foreach ($FILE in $FILES) { curl -OfsS https://$TARGETHOST/$DIRECTORY/$FILE if ($?) { curl -fsS "https://$TARGETHOST/$DIRECTORY/$FILE" ` --form-string "action=delete" > $null } } }