Category: RabbitMQ

Are you using RabbitMQ and PHP and trying to consume messages in PHP? You might have encountered some difficulties when trying to daemonize a PHP script (maybe even used supervisord?… yuck). If not, think about it; ever seen successful implementions?… PHP isn’t built for this, it’s against its nature. I’m getting distracted here, but there is an excellent article about this subject; PHP is meant to die.  Maybe it’s wishfull thinking, but I think that PHP is really in need getting more mature about this “nature” and memory management. Anyway, what about a successful implementation with RabbitMQ and  PHP consumers?

In this post I’ll describe some common problems when consuming messages in PHP, which I encountered during development and solved with some 3th party packages and a bit of configuration. I’m assuming you are already familiar with RabbitMQ and PHP, so I’ll leave that out. You might already have used the RabbitMQBundle or the underlying php-amqplib by videlvaro. These packages are not tied to this article, but are worth mentioning when you intend to create messages in PHP and/or setting up the fabric (automatic exchange and queue creation).

 

Daemon or cron run

Periodically checking for available messages is impossible in RabbitMQ without fetching it from the queue, so you don’t know the messages count. In the cron scenario this will create a problem when processing time exceeds period x (next cron overlaps the previous cron run). So we definitely want a daemon, but not in PHP.

 

 

Separation of concerns

Seen from the architectural point of view; there is the message consumption part and a message processing part. In an ideal world these should be separated to be responsible for their own part. This way debugging becomes a lot easier, because it will be clear if the problem should be searched in RabbitMQ consumption or in your application’s proces. Another benefit of separating is that the interface on the processing part becomes re-usable, and not neccesarily dependant on RabbitMQ. From the testing point of view; your processing part is probably more testable when it is seperated from the consumption part.

And above all (for me) is the improvement of the deployment procedure. Because a daemon like this is a simple patch-trough, it never has to be restarted (unless your message consumption configuration needs change). Normally when updating your application only domain logic changes within the processing part. So after deployment messages are automatically processed “the new way” without needing to restart some consumption service.

 

Solution

As I discussed this with ricbra during a PHP Benelux conference, it came to me as a surprise that he already solved this, by addressing these exact same problems and by writing a consumer in Go. This separate consumer  just consumes a message which is passed to a pre-configured processing command. The exit code of your command is used by that same consumer to requeue or ack the message.

 

 

Supervisord is used to ensure that the daemon is running and is started during boot. An example config for supervisord would be something like this: