I read some articles about developers who takes rabbitmq and produce from PHP messages to write to mysql.
they do this to make the page speedup and not wait for mysql to return an answer.
So in my case, since I want to write to mysql every page request in my site, and yet PHP does not have thread pooling, this I assume can make the page render much faster.
I have 2 questions:
- How does that makes mysql more free? does it?
- How can this architecture scale? (in my speciphic use case as described above.)
4
How does that makes mysql more free? does it?
It does not.
Whilst the solution of having asynchronous writes might seem a bit far-stretched, it’s not. Besides performance improvements from the user’s point of view (he’s no longer waiting for the writes to finish, as you have already mentioned), there are other benefits.
What exactly are database writes (database engine ignorant, it does not matter)? They are usually actions which are generated by user input.
The thing is, if the user request passes the input validation (meaning codes like 400 Bad Request or 412 Precondition Failed have not been returned) and the request is actually correct, whose responsibility is it to make sure the request completes even if something temporarily fails on the server? Is it the user’s responsibility to retry after receiving 500 Internal Server Error, or should the programmer of the server handle that?
I believe edge cases like internal server errors are programmer’s and not the user’s fault and as such it’s the programmer’s responsibility to make sure valid requests ending with 500 actually finish.
How do message queues, such as RabbitMQ, come in play?
They make the organization and repetition of failed requests much easier.
Let’s imagine a scenario where your application the users use communicates with a server through a simple REST API. A user’s action generates a critical request to the server which has to be processed and if is not it’s a huge problem.
In a normal synchronous scenario, the action fails and the user is immediately (or maybe after several seconds, depending on the process) notified that the action has failed and he should retry. But what if they don’t, because they probably don’t feel like waiting for another few seconds? You could have potentially just lost some important data about the client not to mention may have lost the client as well because you were constantly bothering him with waiting.
Now imagine that instead of sending the request synchronously the client-side application would send it to RabbitMQ. Because the queue is a very simple service there’s a very little chance it will fail. The user is immediately notified that their request has been accepted and that they will be notified once their request has been processed.
The consumer retrieves the message from the queue and starts processing it. Because you are currently experiencing some issues on the server, processing the critical request has failed several times. After each request the message has been requeued to try again. It has just so happened that on the 5th try the process have finally finished so you notified the user about the situation.
How can this architecture scale? (in my speciphic use case as
described above.)
You can horizontally scale the application by adding more nodes acting as consumers of the RabbitMQ messages, which is very good. On top of that, from user’s point of view the application runs faster, because the requests are, as you have also already mentioned, asynchronous and processed at some time in the future.