Resumable Upload In Python Flask (Simple Example)

Welcome to a tutorial on how to support resumable file upload in Python Flask. First, I would like to make a quick confession. This frickin guide took me a couple of days, without proper working examples and libraries on the Internet to refer to. Had to fight with a digital dragon, but here it is, my version that is at least working on Python 3. Read on!

 

 

TABLE OF CONTENTS

 

 

EXAMPLE CODE DOWNLOAD

Source code on GitHub Gist

Just click on “download zip” or do a git clone. I have released it under the MIT license, so feel free to build on top of it or use it in your own project.

 

SORRY FOR THE ADS...

But someone has to pay the bills, and sponsors are paying for it. I insist on not turning Code Boxx into a "paid scripts" business, and I don't "block people with Adblock". Every little bit of support helps.

Buy Me A Coffee Code Boxx eBooks

 

 

 

QUICK SETUP

• Create a virtual environment virtualenv venv and activate it – venv/Scripts/activate (Windows) venv/bin/activate (Linux/Mac)
• Install required libraries – pip install flask
• For those who are new, the default Flask folders are –
  • static Public files (JS/CSS/images/videos/audio)
  • templates HTML pages

 

 

PRELUDE) RESUMABLE JS

To support resumable file uploads, we will be using a library called “Resumable JS” (links in the extras section below). How it works is actually pretty simple.

• Instead of uploading a massive file all at once, it is split into multiple smaller chunks.
• Resumable JS will first fire a GET request to check if the specified chunk is already uploaded. The server should respond with HTTP 200 if the chunk is already uploaded, and 404 if not.
• Then, Resumable JS will upload the missing chunks to the server using POST requests.

Maybe a simple example will better explain:

• A large file is split into 100 chunks.
• 30 chunks are uploaded when the browser hangs.
• The user restarts the browser, Resumable JS will fire GET requests to check the upload status.
• Chunks 1 to 30 will return 200 since they are already uploaded.
• Chunk 31 will return 404, and it resumes uploading from chunk 31 onwards.

 

STEP 1) FLASK SERVER

1A) INITIALIZE

# (A) INIT
# (A1) LOAD MODULES
from flask import Flask, render_template, request, abort
import os
 
# (A2) FLASK SETTINGS
app = Flask(__name__)
HOST_NAME = "localhost"
HOST_PORT = 80
# app.debug = True
 
# (A3) UPLOAD FOLDERS & FLAGS
UPLOAD_TEMP = os.path.dirname(os.path.realpath(__file__)) + "/temp"
UPLOAD_TO = os.path.dirname(os.path.realpath(__file__)) + "/upload"
UPLOAD_LOCK = {}
 
# (A4) CREATE UPLOAD FOLDERS IF NOT EXIST
if not os.path.isdir(UPLOAD_TEMP):
  os.mkdir(temp_dir, 0o777)
if not os.path.isdir(UPLOAD_TO):
  os.mkdir(temp_dir, 0o777)
 
# (A5) SUPPORT FUNCTION - GET RESUMABLE CHUNK DATA
def chunker(r):
  return {
    "chunkNumber": int(r.get("resumableChunkNumber")),
    "chunkSize": r.get("resumableChunkSize"),
    "totalChunks": int(r.get("resumableTotalChunks")),
    "identifier": r.get("resumableIdentifier"),
    "path": r.get("resumableRelativePath"),
    "fileName": r.get("resumableFilename"),
    "fileSize": r.get("resumableTotalSize"),
    "tempPath": os.path.join(UPLOAD_TEMP, r.get("resumableIdentifier")),
    "tempChunk": os.path.join(UPLOAD_TEMP, r.get("resumableIdentifier"), r.get("resumableChunkNumber") + ".part")
  }

The first part of the server-side script should be pretty self-explanatory.

• (A1) Load the required modules, Captain Obvious at your service.
• (A2 & A3) Flask settings, upload, and temporary folder path.
• (A4) Create the upload and temporary folders if not found.
• (A5) This is a support function for the upload endpoints, to get information on the current upload chunk. Take note of how the chunks are stored in running numbers in the temporary folder – UPLOAD_TEMP/UNIQUE_IDENTIFIER/N.PART.

 

 

1B) HTML UPLOAD PAGE

# (B) HTML UPLOAD PAGE
@app.route("/")
def index():
  return render_template("S2_upload.html")

Don’t think this needs any explanation. Serve the HTML upload page at the URL root.

 

1C) CHUNK UPLOAD CHECK

# (C) CHECK FILE/CHUNK
@app.route("/uploads", methods=["GET"])
def checker():
  # (C1) GET CURRENT CHUNK INFORMATION
  current = chunker(request.args)
 
  # (C2) RESPOND 200 IF CHUNK UPLOADED, 404 IF NOT
  if os.path.isfile(current["tempChunk"]):
    return ("", 200)
  else:
    return ("", 404)

Remember the part about “using GET request to check upload status”? This part simply responds 200 if a chunk is already uploaded, or 404 if not.

 

 

1D) CHUNK UPLOAD HANDLER

# (D) HANDLE FILE CHUNK UPLOAD
@app.route("/uploads", methods=["POST"])
def uploader():
  # (D1) GET CURRENT CHUNK INFORMATION
  current = chunker(request.args)
 
  # (D2) CREATE TEMP FOLDER IF NOT ALREADY CREATED
  if not os.path.isdir(current["tempPath"]):
  os.mkdir(current["tempPath"], 0o777)
 
  # (D3) SAVE CHUNK
  request.files["file"].save(current["tempChunk"])
 
  # (D4) ALL CHUNKS UPLOADED?
  complete = True
  for i in range(1, current["totalChunks"] + 1):
    if not os.path.exists(os.path.join(current["tempPath"], str(i) + ".part")):
    complete = False
     break

  # (D5) ON COMPLETION OF ALL CHUNKS
  if complete and not current["identifier"] in UPLOAD_LOCK :
    # (D5-1) LOCK
    UPLOAD_LOCK[current["identifier"]] = 1
 
    # (D5-2) LET OUR POWERS COMBINE!
    with open(os.path.join(UPLOAD_TO, current["fileName"]), "ab") as fullfile:
      for i in range(1, current["totalChunks"] + 1):
        chunkName = os.path.join(current["tempPath"], str(i) + ".part")
        chunkFile = open(chunkName, "rb")
        fullfile.write(chunkFile.read())
        chunkFile.close()
        os.unlink(chunkName)
        fullfile.close()
        os.rmdir(current["tempPath"])
 
    # (D5-3) UNLOCK
    del UPLOAD_LOCK[current["identifier"]]
 
  # (D6) RESPONSE
  return ("", 200)

Not going to explain these step-by-step. But essentially, save the uploaded chunk into the temporary folder. On upload complete (all chunks uploaded), we combine them back together into the complete file.

 

1E) START

# (E) START!
if __name__ == "__main__":
  app.run(HOST_NAME, HOST_PORT)

Well, start the Flask server.

 

 

STEP 2) UPLOAD PAGE

2A) THE HTML

<!-- (A) UPLOAD BUTTON & LIST -->
<div id="updrop">
  Drop Files Here. Click To Resume/Pause.
</div>
 
<!-- (B) UPLOAD FILES LIST -->
<div id="uplist"></div>

• <div id="updrop"> As it is. Drop files here to upload, click to resume/pause uploads.
• <div id="uplist"> The list of files.

 

2B) THE JAVASCRIPT

window.addEventListener("load", () => {
  // (A) CREATE RESUMABLE OBJECT
  let r = new Resumable({
    target: "/uploads",
    query: { key: "value" } // optional
  });
 
  // (B) GET HTML ELEMENTS
  let dropzone = document.getElementById("updrop"),
      listzone = document.getElementById("uplist");
 
  // (C) UPLOAD MECHANICS
  // (C1) PAUSE/RESUME UPLOAD
  dropzone.onclick = () => {
    if (r.isUploading()) { r.pause(); }
    else { r.upload(); }
  };
 
  // (C2) FILE ADDED - ADD HTML ROW & START UPLOAD
  r.on("fileAdded", (file, evt) => {
    let row = document.createElement("div");
    row.className = "row";
    row.innerHTML = `${file.fileName} (<span class="status">0%</span>)`;
    row.id = file.uniqueIdentifier;
    listzone.appendChild(row);
    r.upload();
  });
 
  // (C3) UPLOAD PROGRESS
  r.on("fileProgress", (file, evt) => {
    let row = document.getElementById(file.uniqueIdentifier);
    row.getElementsByTagName("span")[0].innerHTML = Math.ceil(file.progress() * 100) + "%";
  });
 
  // (C4) UPLOAD SUCCESFUL
  r.on("fileSuccess", (file, msg) => {
    // DO SOMETHING
  });
 
  // (C5) UPLOAD ERROR
  r.on("fileError", (file, msg) => {
    let row = document.getElementById(file.uniqueIdentifier);
    row.getElementsByTagName("span")[0].innerHTML = msg;
    console.error(file, msg);
  });
 
  // (D) ATTACH
  r.assignDrop(document.getElementById("updrop"));
  // r.assignBrowse(document.getElementById("browseButton"));
});

The Javascript looks like quite a handful, but walk through them section by section.

1. Create a new Resumable() object. Check their website for the full list of options and events.
2. Get the HTML drop zone and files list.
3. HTML interface stuff. Create a new row in the files list when a file is added, and update the upload progress as it goes along.
4. Attach the “upload dropzone” to start.

 

 

 

LINKS & REFERENCES

• Resumable JS
• Python Resumable