Multiple ASYNC CURL Requests In PHP (Simple Example)

Welcome to a tutorial on how to do multiple asynchronous CURL requests in PHP. Need to grab data from various websites all at once? Or send data to different servers in parallel? Yes, it is possible – Read on for the example!

 

 

TABLE OF CONTENTS

 

 

EXAMPLE CODE DOWNLOAD

Source code on GitHub Gist

Just click on “download zip” or do a git clone. I have released it under the MIT license, so feel free to build on top of it or use it in your own project.

 

SORRY FOR THE ADS...

But someone has to pay the bills, and sponsors are paying for it. I insist on not turning Code Boxx into a "paid scripts" business, and I don't "block people with Adblock". Every little bit of support helps.

Buy Me A Coffee Code Boxx eBooks

 

 

 

STEP 1) INITIALIZE & SETTINGS

// (A) INIT
// (A1) CURL TARGETS
$list = [
  "http://localhost/dummy.php" => [
    "post" => ["KeyA" => "ValueA", "KeyB" => "ValueB"],
    "callback" => "output"
  ],
  "https://en.wikipedia.org/wiki/Portal:Gardening" => [
    "callback" => "output"
  ]
];
 
// (A2) CALLBACK FUNCTION - OUTPUT RESULTS
function output ($res, $info) { echo $res; print_r($info); }
 
// (A3) PAUSE BETWEEN EACH CURL CALL (MS)
$pause = 1000;

First, let us start with some “settings stuff”.

• (A1) $list is the list of servers you want to work with.
  • post Optional data that you want to send to the server.
  • callback Kind of optional… The callback function to run when the CURL fetch is complete.
• (A2) For this example, we keep the callback simple and just output the results.
• (A3) Used in step 3 – To not crash the server, we introduce a small pause between “each multi CURL check cycle”.

 

 

STEP 2) INITIALIZE MULTI CURL

// (B) MULTI CURL INIT
$mh = curl_multi_init();
foreach ($list as $url=>$l) {
  $c = curl_init();
  curl_setopt($c, CURLOPT_URL, $url);
  curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
  if (isset($l["post"])) {
    curl_setopt($c, CURLOPT_POST, true);
    curl_setopt($c, CURLOPT_POSTFIELDS, $l["post"]);
  }
  curl_multi_add_handle($mh, $c);
}

Yep, we are pretty much initiating multiple “normal single CURL” requests:

• $mh = curl_multi_init() We create a “multiple CURL object”.
• $c = curl_init() Initialize the individual CURL request.
• curl_multi_add_handle($mh, $c) Add the individual CURL request into the “multiple CURL object”.

 

 

STEP 3) RUN MULTIPLE CURL

// (C) CURL EXEC
// CREDITS: https://gist.github.com/Xeoncross/2362936
do {
  // (C1) GET CURL EXEC STATUS
  $status = curl_multi_exec($mh, $active);
 
  // (C2) RUN CALLBACK WHEN CURL REQUEST IS COMPLETE
  if ($state = curl_multi_info_read($mh)) {
    $info = curl_getinfo($state["handle"]);
    if (isset($list[$info["url"]]["callback"])) {
      $callback = $list[$info["url"]]["callback"];
      $callback(curl_multi_getcontent($state["handle"]), $info);
    }
    curl_multi_remove_handle($mh, $state["handle"]);
  }
 
  // (C3) SHORT PAUSE TO NOT FLOOD SERVER
  usleep($pause);
} while ($status == CURLM_CALL_MULTI_PERFORM || $active);
 
// (C4) CASE CLOSED - ALL DONE
curl_multi_close($mh);

Finally, this should be pretty self-explanatory. We run the multiple CURL requests and loop until all of them have ended; While introducing a small pause between “each cycle”.

 

 

THE SUMMARY

Function	Description	Reference Link
curl_multi_init()	Initialize and create a multi CURL handler.	Click Here
curl_multi_add_handle()	Add a CURL handler to the multi handler.	Click Here
curl_multi_exec()	Run the multi CURL handler.	Click Here
curl_multi_remove_handle()	Remove a CURL handler from the multi handler.	Click Here
curl_multi_close()	Close the multi CURL handler.	Click Here
curl_multi_getcontent()	Get the result from a CURL handler.	Click Here