Multiple ASYNC CURL Requests In PHP (Simple Example)

Welcome to a tutorial on how to do multiple asynchronous CURL requests in PHP. Need to grab data from various websites all at once? Or send data to different servers in parallel? Yes, it is possible – Read on for the example!

ⓘ I have included a zip file with all the source code at the start of this tutorial, so you don’t have to copy-paste everything… Or if you just want to dive straight in.

 

 

TLDR – QUICK SLIDES

Fullscreen Mode – Click Here

 

TABLE OF CONTENTS

 

 

QUICK NOTES

If you spot a bug, feel free to comment below. I try to answer short questions too, but it is one person versus the entire world… If you need answers urgently, please check out my list of websites to get help with programming.

 

EXAMPLE CODE DOWNLOAD

Click here to download all the example source code, I have released it under the MIT license, so feel free to build on top of it or use it in your own project.

 

 

 

STEP 1) INITIALIZE & SETTINGS

// (A) INIT
// (A1) CURL TARGETS
$list = [
  "http://localhost/dummy.php" => [
    "post" => ["KeyA" => "ValueA", "KeyB" => "ValueB"],
    "callback" => "output"
  ],
  "https://en.wikipedia.org/wiki/Portal:Gardening" => [
    "callback" => "output"
  ]
];
 
// (A2) CALLBACK FUNCTION - OUTPUT RESULTS
function output ($res, $info) { echo $res; print_r($info); }
 
// (A3) PAUSE BETWEEN EACH CURL CALL (MS)
$pause = 1000;

First, let us start with some “settings stuff”.

• (A1) $list is the list of servers you want to work with.
  • post Optional data that you want to send to the server.
  • callback Kind of optional… The callback function to run when the CURL fetch is complete.
• (A2) For this example, we keep the callback simple and just output the results.
• (A3) Used in step 3 – To not crash the server, we introduce a small pause between “each multi CURL check cycle”.

 

 

STEP 2) INITIALIZE MULTI CURL

// (B) MULTI CURL INIT
$mh = curl_multi_init();
foreach ($list as $url=>$l) {
  $c = curl_init();
  curl_setopt($c, CURLOPT_URL, $url);
  curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
  if (isset($l["post"])) {
    curl_setopt($c, CURLOPT_POST, true);
    curl_setopt($c, CURLOPT_POSTFIELDS, $l["post"]);
  }
  curl_multi_add_handle($mh, $c);
}

Yep, we are pretty much initiating multiple “normal single CURL” requests:

• $mh = curl_multi_init() We create a “multiple CURL object”.
• $c = curl_init() Initialize the individual CURL request.
• curl_multi_add_handle($mh, $c) Add the individual CURL request into the “multiple CURL object”.

 

 

STEP 3) RUN MULTIPLE CURL

// (C) CURL EXEC
// CREDITS: https://gist.github.com/Xeoncross/2362936
do {
  // (C1) GET CURL EXEC STATUS
  $status = curl_multi_exec($mh, $active);
 
  // (C2) RUN CALLBACK WHEN CURL REQUEST IS COMPLETE
  if ($state = curl_multi_info_read($mh)) {
    $info = curl_getinfo($state["handle"]);
    if (isset($list[$info["url"]]["callback"])) {
      $callback = $list[$info["url"]]["callback"];
      $callback(curl_multi_getcontent($state["handle"]), $info);
    }
    curl_multi_remove_handle($mh, $state["handle"]);
  }
 
  // (C3) SHORT PAUSE TO NOT FLOOD SERVER
  usleep($pause);
} while ($status == CURLM_CALL_MULTI_PERFORM || $active);
 
// (C4) CASE CLOSED - ALL DONE
curl_multi_close($mh);

Finally, this should be pretty self-explanatory. We run the multiple CURL requests and loop until all of them have ended; While introducing a small pause between “each cycle”.

 

 

THE SUMMARY

Function	Description	Reference Link
curl_multi_init()	Initialize and create a multi CURL handler.	Click Here
curl_multi_add_handle()	Add a CURL handler to the multi handler.	Click Here
curl_multi_exec()	Run the multi CURL handler.	Click Here
curl_multi_remove_handle()	Remove a CURL handler from the multi handler.	Click Here
curl_multi_close()	Close the multi CURL handler.	Click Here
curl_multi_getcontent()	Get the result from a CURL handler.	Click Here

 

INFOGRAPHIC CHEAT SHEET