3 Ways to Upload Large Files in PHP – Settings, Chunking, Resumable

Welcome to a tutorial on how to upload large files in PHP. Once upon a time in the Stone Age of the Internet, file uploads were manageable without large files. But these days, we have to deal with all kinds of insanely oversized files, and the “traditional” upload mechanism just cannot handle it.

In order to deal with the large file uploads in PHP, there are a few alternatives:

  1. Change the upload_max_filesize limit in php.ini.
  2. Implement file chunk upload. Split the upload into smaller pieces, and assemble them when the upload is complete.
  3. Implement resumable uploads.

But just how do we implement each of the methods? Read on to find out!

ⓘ I have included a zip file with all the example code at the start of this tutorial, so you don’t have to copy-paste everything… Or if you just want to dive straight in.

 

TABLE OF CONTENTS

Download & Notes Tweak PHP Settings Chunking With Plupload
Resumable Upload Useful Bits & Links The End

 

 

DOWNLOAD & NOTES

Firstly, here is the download link to the example code as promised.

 

EXAMPLE CODE DOWNLOAD

Click here to download all the example code in a zip file – I have released it under the MIT License, so feel free to build on top of it if you want to.

 

QUICK NOTES

  • There are 3 versions of “upload” in the folder.
    • 1a-upload.html Basic PHP setting tweak.
    • 2a-chunk.html Chunking upload.
    • 3a-resumable.html Resumable upload.
  • For the resumable upload, install Composer and use it to get the PHP server-side library first – composer require flowjs/flow-php-server.

If you spot a bug, please feel free to comment below. I try to answer questions too, but it is one person versus the entire world… If you need answers urgently, please check out my list of websites to get help with programming.

 

TWEAK PHP SETTINGS

Let us start with the first method, which is an easier way out for you guys who still have rather manageable upload sizes – Just change the maximum allowed file size in PHP.

 

ALTERNATIVE 1) UPDATING THE PHP.INI

php.ini
upload_max_filesize = 150M
post_max_size = 150M
max_input_time = 300
max_execution_time = 300

There are 4 settings to look out for when it comes to allowing large file uploads:

  • upload_max_filesize – The maximum allowed upload file size.
  • post_max_size – The maximum allowed POST data size.
  • max_input_time – Maximum allowed input time.
  • max_execution_time – Maximum allowed time the scripts are allowed to run.

But of course, I will not recommend changing the php.ini file, as it will affect the entire server and all your other projects as well.

 

ALTERNATIVE 2) UPDATING THE HTACCESS FILE

.htaccess
php_value upload_max_filesize 150M
php_value post_max_size 150M
php_value max_input_time 300
php_value max_execution_time 300

If you do not have access to the php.ini file and is running an Apache server, it is still possible to change the settings by creating a .htaccess file in the folder where the upload script is at. This way, the settings will only be applied to the scripts within that folder… Kind of a better solution, but I still don’t recommend doing so.

 

UPLOAD!

1a-upload.html
<form action="1b-upload.php" method="post" enctype="multipart/form-data">
  <input type="file" name="fileup" required/>
  <input type="submit" value="Go"/>
</form>
1b-upload.php
<?php
// MOVE UPLOADED FILE TO DESTINATION
$source = $_FILES["fileup"]["tmp_name"];
$destination = $_FILES["fileup"]["name"];
move_uploaded_file($source, $destination);
echo "OK";

Finally, just upload as usual with the raised the limits, and that’s it!

 

 

INI SET?

For you guys who are thinking of using the ini_set() function to change the upload size – Please take note that according to the official core php.ini directives, upload_max_filesize is only changeable in PHP_INI_PERDIR. Meaning, it can only be tweaked in the php.ini or .htaccess file; ini_set("upload_max_filesize", "150M") will not work.

 

CHUNKING WITH PLUPLOAD

This second alternative is called “chunking” – Splitting a large file and uploading them in smaller chunks. While it may sound difficult, there is thankfully an open-source library called “Plupload” that we can use. No need to reinvent the wheel, and here is how we use it.

 

STEP 1) HTML & JAVASCRIPT

2a-chunk.html
<!-- CHECK https://cdnjs.com/libraries/plupload FOR LATEST VERSION -->
<!-- OFFICIAL WEBSITE: https://www.plupload.com/ -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/plupload/3.1.2/plupload.full.min.js"></script>
<script>
window.addEventListener("load", function () {
  var uploader = new plupload.Uploader({
    runtimes: 'html5,html4',
    browse_button: 'pickfiles',
    url: '2b-chunk.php',
    chunk_size: '10mb',
    /* OPTIONAL
    filters: {
      max_file_size: '150mb',
      mime_types: [{title: "Image files", extensions: "jpg,gif,png"}]
    },
    */
    init: {
      PostInit: function () {
        document.getElementById('filelist').innerHTML = '';
      },
      FilesAdded: function (up, files) {
        plupload.each(files, function (file) {
          document.getElementById('filelist').innerHTML += `<div id="${file.id}">${file.name} (${plupload.formatSize(file.size)}) <strong></strong></div>`;
        });
        uploader.start();
      },
      UploadProgress: function (up, file) {
        document.querySelector(`#${file.id} strong`).innerHTML = `<span>${file.percent}%</span>`;
      },
      Error: function (up, err) {
        console.log(err);
      }
    }
  });
  uploader.init();
});
</script>
 
<!-- UPLOAD FORM -->
<div id="container">
  <span id="pickfiles">[Upload files]</span>
</div>

<!-- UPLOAD FILE LIST -->
<div id="filelist">Your browser doesn't support HTML5 upload.</div>

Yikes! This looks like a handful, but it is pretty much a modified version of the “default Plupload” demo script.

  • There are only 3 HTML elements here.
    • <div id="container"> and <span id="pickfiles"> is where the “upload” button will be generated.
    • <div id="filelist"> is where we show the upload file list and progress.
  • Initiate a var uploader = new plupload.Uploader() object on page load. There are a lot of options, but a few important ones are:
    • runtimes Just use HTML5 and HTML4 if unsure. There is also backward compatibility support by using Flash and Silverlight.
    • browse_button The ID of the HTML browse button.
    • url Server-side script handling the upload.
    • chunk_size Size of each chunk.
  • Next, the various init is what drives the upload interface. Not going to go through line-by-line, but in general:
    • PostInit is fired after the Plupload object is created.
    • FilesAdded whenever a new file is added. We add a new entry to #filelist.
    • UploadProgress on file upload progress. We update the upload progress in #filelist.
    • Error self-explanatory.

That is the gist of it, read the official documentation (links in the extra section below) if you need more customizations.

 

 

STEP 2) SERVER-SIDE UPLOAD HANDLER

2b-chunk.php
<?php
// (A) FUNCTION TO FORMULATE SERVER RESPONSE
function verbose($ok=1,$info=""){
  // THROW A 400 ERROR ON FAILURE
  if ($ok==0) { http_response_code(400); }
  die(json_encode(["ok"=>$ok, "info"=>$info]));
}

// (B) INVALID UPLOAD
if (empty($_FILES) || $_FILES['file']['error']) {
  verbose(0, "Failed to move uploaded file.");
}

// (C) UPLOAD DESTINATION
// ! CHANGE FOLDER IF REQUIRED !
$filePath = __DIR__ . DIRECTORY_SEPARATOR . "uploads";
if (!file_exists($filePath)) { 
  if (!mkdir($filePath, 0777, true)) {
    verbose(0, "Failed to create $filePath");
  }
}
$fileName = isset($_REQUEST["name"]) ? $_REQUEST["name"] : $_FILES["file"]["name"];
$filePath = $filePath . DIRECTORY_SEPARATOR . $fileName;

// (D) DEAL WITH CHUNKS
$chunk = isset($_REQUEST["chunk"]) ? intval($_REQUEST["chunk"]) : 0;
$chunks = isset($_REQUEST["chunks"]) ? intval($_REQUEST["chunks"]) : 0;
$out = @fopen("{$filePath}.part", $chunk == 0 ? "wb" : "ab");
if ($out) {
  $in = @fopen($_FILES['file']['tmp_name'], "rb");
  if ($in) {
    while ($buff = fread($in, 4096)) { fwrite($out, $buff); }
  } else {
    verbose(0, "Failed to open input stream");
  }
  @fclose($in);
  @fclose($out);
  @unlink($_FILES['file']['tmp_name']);
} else {
  verbose(0, "Failed to open output stream");
}

// (E) CHECK IF FILE HAS BEEN UPLOADED
if (!$chunks || $chunk == $chunks - 1) {
  rename("{$filePath}.part", $filePath);
}
verbose(1, "Upload OK");

Yikes! This script looks like another bunch of trouble again but is something really straightforward.

  • It creates an empty .part file on the first chunk.
  • Then simply appends all the chunks to the .part file as they are being uploaded.
  • When all the chunks are assembled, the .part file is renamed back to what it is supposed to be.

Done! You now have a system that is capable of handling large file uploads.

 

 

EXTRA) WAYS TO GET PLUPLOAD

Plupload is available on various channels:

 

RESUMABLE UPLOAD

This last alternative is also another “modern one” – Resumable uploads. Yep, there are open source plugins that we can use, and this one is called FlowJS.

 

STEP 1) THE HTML

Get FlowJS by either using the CDN or download from Github.

3a-resumable.html
<!-- CHECK https://cdnjs.com/libraries/flow.js FOR LATEST VERSION -->
<!-- OFFICIAL WEBSITE: https://github.com/flowjs/flow.js/ -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/flow.js/2.14.1/flow.min.js"></script>
<script>
window.addEventListener("load", function(){
  // (A) NEW FLOW OBJECT
  var flow = new Flow({
    target: '3b-resumable.php',
    chunkSize: 1024*1024, // 1MB
    singleFile: true
  });
 
  if (flow.support) {
    // (B) ASSIGN BROWSE BUTTON
    flow.assignBrowse(document.getElementById('upbrowse'));
    // OR DEFINE DROP ZONE
    // flow.assignDrop(document.getElementById('updrop'));

    // (C) ON FILE ADDED
    flow.on('fileAdded', function(file, event){
      // console.log(file, event);
      let fileslot = document.createElement("div");
      fileslot.id = file.uniqueIdentifier;
      fileslot.innerHTML = `${file.name} (${file.size}) - <strong>0%</strong>`;
      document.getElementById("uplist").appendChild(fileslot);
    });

    // (D) ON FILE SUBMITTED (ADDED TO UPLOAD QUEUE)
    flow.on('filesSubmitted', function(array, event){
      // console.log(array, event);
      flow.upload();
    });

    // (E) ON UPLOAD PROGRESS
    flow.on('fileProgress', function(file, chunk){
      // console.log(file, chunk);
      let progress = (chunk.offset + 1) / file.chunks.length * 100;
      progress = progress.toFixed(2) + "%";

      // QUERYSELECTOR NOT WORKING WITH "-" IN ID...
      // document.querySelector(`${file.uniqueIdentifier} strong`)
      let fileslot = document.getElementById(file.uniqueIdentifier);
      fileslot = fileslot.getElementsByTagName("strong")[0];
      fileslot.innerHTML = progress;
    });

    // (F) ON UPLOAD SUCCESS
    flow.on('fileSuccess', function(file, message, chunk){
      // console.log(file, message, chunk);
      let fileslot = document.getElementById(file.uniqueIdentifier);
      fileslot = fileslot.getElementsByTagName("strong")[0];
      fileslot.innerHTML = "DONE";
    });

    // (G) ON UPLOAD ERROR
    flow.on('fileError', function(file, message){
      // console.log(file, message);
      let fileslot = document.getElementById(file.uniqueIdentifier);
      fileslot = fileslot.getElementsByTagName("strong")[0];
      fileslot.innerHTML = "ERROR";
    });

    // (H) PAUSE/CONTINUE UPLOAD
    document.getElementById("upToggle").addEventListener("click", function(){
      if (flow.isUploading()) { flow.pause(); }
      else { flow.resume(); }
    });
  }
});
</script>

<!-- UPLOAD BUTTON -->
<input type="button" id="upbrowse" value="Browse"/>
<input type="button" id="upToggle" value="Pause OR Continue"/>

<!-- UPLOAD LIST -->
<div id="uplist"></div>

This is yet another handful, but let’s take it apart piece-by-piece:

  • The HTML only has 3 elements.
    • An “upload button” <input type="button" id="upbrowse"/>.
    • A pause or continue upload button <input type="button" id="upToggle"/>.
    • Upload file list <div id="uplist">.
  • On window load, we create a var flow = new Flow() object. A lot of settings again, but importantly:
    • target Server-side script handling the upload.
    • chunkSize As in the previous method, the size per upload chunk.
  • Not going to run through line-by-line, but just like Plupload, we will use the events to build the interface:
    • fileAdded When a new file is added to the list. We add a new entry to #uplist.
    • fileSubmitted When the new file is added to the upload queue. We just immediately start the upload here.
    • fileProgress As the file is uploading. We calculate the upload percentage and update #uplist accordingly.
    • fileSuccess When the upload is complete.
    • fileError Self-explanatory.
  • The difference here is that we can pause and resume uploads with FlowJS – flow.pause() and flow.resume().

 

 

STEP 2) GET FLOW PHP SERVER

Thankfully, someone has made the PHP server-side script as well. Just pull the latest version using Composer:

D:\http>composer require flowjs/flow-php-server
Using version ^1.0 for flowjs/flow-php-server
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 1 install, 0 updates, 0 removals
  - Installing flowjs/flow-php-server (v1.0.3): Loading from cache
Writing lock file
Generating autoload files

 

STEP 3) FLOW PHP SERVER-SIDE SCRIPT

Yep, this is just a small modified version of the demo at the Flow PHP GitHub page:

3b-resumable.php
<?php
// ! GET THE PACKAGE FIRST !
// composer require flowjs/flow-php-server
// SOURCE: https://github.com/flowjs/flow-php-server

// (A) INIT PHP FLOW
require __DIR__ . DIRECTORY_SEPARATOR . "vendor" . DIRECTORY_SEPARATOR . "autoload.php";
$config = new \Flow\Config();
$config->setTempDir(__DIR__ . DIRECTORY_SEPARATOR . "temp");
$request = new \Flow\Request();

// (B) HANDLE UPLOAD
$uploadFolder = __DIR__ . DIRECTORY_SEPARATOR . "uploads" . DIRECTORY_SEPARATOR;
$uploadFileName = uniqid() . "_" . $request->getFileName(); 
$uploadPath = $uploadFolder . $uploadFileName;
if (\Flow\Basic::save($uploadPath, $config, $request)) {
  // File saved successfully
} else {
  // Not final chunk or invalid request. Continue to upload.
}

Probably shouldn’t use is as-it-is, but a good start nonetheless.

 

 

USEFUL BITS & LINKS

That’s it for all the upload methods, and here is a section of small extras and links that may be useful to you.

 

FILE TYPE RESTRICTIONS

All the above examples now accept all kinds of files and extensions. If you want to restrict the file types, I will highly recommend doing a simple server-side check instead.

<?php
$ext = strtoupper(pathinfo($FILE, PATHINFO_EXTENSION));

// Let's say, we allow only XLS, XLSX
if ($ext!="XLS" || $ext!="XLSX") { die("INVALID FILE TYPE"); }

 

YOUTUBE TUTORIAL

 

LINKS & REFERENCES

 

WHAT NEXT?

Thank you for reading, and we have come to the end of this short tutorial. I hope that it has helped to solve your big upload woes. If you have stuff you like to add to this guide, please feel free to comment below. Good luck and happy coding!

7 thoughts on “3 Ways to Upload Large Files in PHP – Settings, Chunking, Resumable”

  1. In the form I created, I choose a folder using input type = “file”. Of course, there are many subfolders and files in the folder. Do you have any idea how I can do this with the 2nd alternative? thank you…

  2. Thank you for this great tutorial its very well written and helped me a lot.
    For file type restrictions. For Method 3 I added the below code into the fileAdded function which stops the file from being uploaded if its not in this case a txt file. Hope this helps someone 🙂

    One thing I noticed in your code, is that uploading a large file that takes minutes to upload, it jumps backwards and forwards on the total uploaded, so for e.g. it will say 100% and then drop back to 96%, its not quite right but I’m not sure what needs to be changed to fix it!

    All the best

    // (C) ON FILE ADDED
    flow.on('fileAdded', function(file, event){
      var check_file = (file.name);
      if (check_file.toLowerCase().endsWith(".txt")) {
        let fileslot = document.createElement("div");
        fileslot.id = file.uniqueIdentifier;
        fileslot.innerHTML = `${file.name} (${file.size}) - 0%`;
        document.getElementById("uplist").appendChild(fileslot);
      } else {
        return false;
      }
    });
    
    1. let progress = (chunk.offset + 1) / file.chunks.length * 100; does the percentage calculation. I have a hunch that parallel upload (multiple chunks upload at a time) is causing the chunk.offset to jump around.

  3. Thank you for this tutorial. But how can I retrieve the output of the responce function (verbose)? In your example the file ‘2b-upload.php’ returns a json object if the upload was successfull. In first step, I just want to show the response via console.log or javascript alert.

Leave a Comment

Your email address will not be published. Required fields are marked *