In concrete5 version 5.6.2 and up, Jobs can now be written in a way that they process items in batches as opposed to all at once. The benefit of this is especially obvious when writing jobs that process large amounts of data.

To get started on writing a new queuable job, you might want to take a look at the "Index Search Engine - All" job.

https://github.com/concrete5/concrete5/blob/master/web/concrete/core/jobs/index_search_all.php

There is also a file hashing job written by community member Mnkras

https://github.com/Mnkras/file_hasher

Here we can see the 3 main components of a queuable job.

public function start(Zend_Queue $q)

This is where you will do any preprocessing and more importantly where you will load the queue with items to process. For example if we wanted to process a database table full of entries named "PizzaOrders" we might have something like this

public function start(Zend_Queue $q) {
  $db = Loader::db();
  $res = $db->Execute('SELECT id FROM PizzaOrders');
  while ($row = $res->FetchRow()) {
    $q->send($row['id']);
  }
}

This loads our queue. If you are new to message queues you can think of it as a list of tasks another process will process later. This brings us to our next section. Processing the queue.

public function processQueueItem(Zend_Queue_Message $msg)

public function processQueueItem(Zend_Queue_Message $msg) {
  $pizzaOrder = PizzaOrder::getByID($msg->body);
  $pizzaOrder->processOrder();
}

What happens here is that the concrete5 queue engine will process these queue items in batches as opposed to all at once. The benefit of this is that you avoid things like server timeouts and a process exceeding the memory limit.

Finally, we wrap up with a method that allows us to do any clean up, notification etc after the entirety of the queue items are processed.

public function finish(Zend_Queue $q)

public function finish(Zend_Queue $q) {
  $mh = Loader::helper('mail');
  $mh->subject('Pizza Order Processing Complete');
  $mh->body('The pizza orders have completed processing. Time to make some pizza!');
  $mh->to('pizzamaker@example.com');
  $mh->send();
}