Multiple ASYNC CURL Requests In PHP (Simple Example)

Welcome to a tutorial on how to do multiple asynchronous CURL requests in PHP. Need to grab data from various websites all at once? Or send data to different servers in parallel? Yes, it is possible – Read on for the example!

ⓘ I have included a zip file with all the source code at the start of this tutorial, so you don’t have to copy-paste everything… Or if you just want to dive straight in.

 

 

TLDR – QUICK SLIDES

Fullscreen Mode – Click Here

 

TABLE OF CONTENTS

 

DOWNLOAD & NOTES

Firstly, here is the download link to the example code as promised.

 

QUICK NOTES

If you spot a bug, feel free to comment below. I try to answer short questions too, but it is one person versus the entire world… If you need answers urgently, please check out my list of websites to get help with programming.

 

EXAMPLE CODE DOWNLOAD

Click here to download all the example source code, I have released it under the MIT license, so feel free to build on top of it or use it in your own project.

 

 

PHP MULTI CURL

All right, let us now get into a detailed example of doing multiple parallel CURL requests in PHP.

 

STEP 1) INITIALIZE & SETTINGS

multi-curl.php
// (A) INIT
// (A1) CURL TARGETS
$list = [
  "http://localhost/dummy.php" => [
    "post" => ["KeyA" => "ValueA", "KeyB" => "ValueB"],
    "callback" => "output"
  ],
  "https://en.wikipedia.org/wiki/Portal:Gardening" => [
    "callback" => "output"
  ]
];
 
// (A2) CALLBACK FUNCTION - OUTPUT RESULTS
function output ($res, $info) { echo $res; print_r($info); }
 
// (A3) PAUSE BETWEEN EACH CURL CALL (MS)
$pause = 1000;

First, let us start with some “settings stuff”.

  • (A1) $list is the list of servers you want to work with.
    • post Optional data that you want to send to the server.
    • callback Kind of optional… The callback function to run when the CURL fetch is complete.
  • (A2) For this example, we keep the callback simple and just output the results.
  • (A3) Used in step 3 – To not crash the server, we introduce a small pause between “each multi CURL check cycle”.

 

 

STEP 2) INITIALIZE MULTI CURL

multi-curl.php
// (B) MULTI CURL INIT
$mh = curl_multi_init();
foreach ($list as $url=>$l) {
  $c = curl_init();
  curl_setopt($c, CURLOPT_URL, $url);
  curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
  if (isset($l["post"])) {
    curl_setopt($c, CURLOPT_POST, true);
    curl_setopt($c, CURLOPT_POSTFIELDS, $l["post"]);
  }
  curl_multi_add_handle($mh, $c);
}

Yep, we are pretty much initiating multiple “normal single CURL” requests:

  • $mh = curl_multi_init() We create a “multiple CURL object”.
  • $c = curl_init() Initialize the individual CURL request.
  • curl_multi_add_handle($mh, $c) Add the individual CURL request into the “multiple CURL object”.

 

 

STEP 3) RUN MULTIPLE CURL

multi-curl.php
// (C) CURL EXEC
// CREDITS: https://gist.github.com/Xeoncross/2362936
do {
  // (C1) GET CURL EXEC STATUS
  $status = curl_multi_exec($mh, $active);
 
  // (C2) RUN CALLBACK WHEN CURL REQUEST IS COMPLETE
  if ($state = curl_multi_info_read($mh)) {
    $info = curl_getinfo($state["handle"]);
    if (isset($list[$info["url"]]["callback"])) {
      $callback = $list[$info["url"]]["callback"];
      $callback(curl_multi_getcontent($state["handle"]), $info);
    }
    curl_multi_remove_handle($mh, $state["handle"]);
  }
 
  // (C3) SHORT PAUSE TO NOT FLOOD SERVER
  usleep($pause);
} while ($status == CURLM_CALL_MULTI_PERFORM || $active);
 
// (C4) CASE CLOSED - ALL DONE
curl_multi_close($mh);

Finally, this should be pretty self-explanatory. We run the multiple CURL requests and loop until all of them have ended; While introducing a small pause between “each cycle”.

 

EXTRA BITS & LINKS

That’s all for the main tutorial, and here is a small section on some extras and links that may be useful to you.

 

THE SUMMARY

Function Description Reference Link
curl_multi_init() Initialize and create a multi CURL handler. Click Here
curl_multi_add_handle() Add a CURL handler to the multi handler. Click Here
curl_multi_exec() Run the multi CURL handler. Click Here
curl_multi_remove_handle() Remove a CURL handler from the multi handler. Click Here
curl_multi_close() Close the multi CURL handler. Click Here
curl_multi_getcontent() Get the result from a CURL handler. Click Here

 

INFOGRAPHIC CHEAT SHEET

PHP Multi-CURL Example (click to enlarge)

 

THE END

Thank you for reading, and we have come to the end. I hope that it has helped you to better understand, and if you want to share anything with this guide, please feel free to comment below. Good luck and happy coding!

Leave a Comment

Your email address will not be published. Required fields are marked *