I have a multithreaded backend application. It is inputted x, and outputs y, and this makes up a “job”. A job can take up to several seconds to complete. The application is fed input via HTTP requests, and once the job is completed it will return a response to the HTTP (so the response time can be several seconds). The response data can vary in size from a couple of bytes to around half a mb.
The issue is that when a lot of requests are sent to the application, all the threads are used up waiting for the jobs to complete, resulting in long response times for further requests – while I want the application to be focused on just completing the jobs.
I was thinking maybe a better approach is: on request add a job to a queue, and return a message that the job has been received in the request HTTP message. The application will process the results, and then upload them to a database. Then the results can be downloaded from the database via the webserver and sent to a client. However I have some queries about this approach.
-
The response databases will frequently be written to and read from. Is there a class of databases that are designed for frequent accessing in this manor – largely with respect to performance? Also these results will be temporary and need to be erased from the database once they’re read. (I’ve often thought of databases from the more traditional MySQL ones which would require frequent querying in order to achieve what I’ve described).
-
A move away from HTTP responses means that the client will need to be notified in some way that the results have arrived. From a web dev perspective what might be a good way of sending the client’s webpage the response?
-
Is there a name for this database pipeline/architecture so that I can do some more research into it? I know that this process may be loosely described as a microservice architecture were APIs are used to communicate between the backend application.
Any other information/insights into this database communication between frontend and backend would be greatly appreciated.
2
1
A common approach is to separate your write (singular) and read (as many as you want to scale) instances, and set up some sort of automatic replication whereby the write instance is “copied” to the read instances (scheduled, triggered, …).
This ensures that interacting with the write database (when posting updates) is not affected by the amount of reads happening. For example, if millions of people query your database and get varying performance (and this is okay to you) but you have a handful of writers who need high performance in order to do their work.
Note, however, that this inherently introduces the concept of eventual consistency, because the time to replicate a write action to the read instances is non-zero. There will be some timeframe in which an update was written to the database but the read instance is not yet replicated, meaning the change is not visible to an end user who queries your database.
For example, if the replication time is more than a few seconds, this might mean that you send out an email to the user that the job is completed, and if that user follows up immediately (i.e. faster than your replication time), the website might tell them that the job isn’t completed yet, because the read instance has not yet been replicated.
Also these results will be temporary and need to be erased from the database once they’re read.
So you’ll need a cleanup process. This could be a separate scheduled background job, or it could be triggered by the query logic. This depends on your needs. Some options here include:
- A scheduled job which deletes anything at the end of the day older than X
- A scheduled job which deletes anything at the end of the day that has been retrieved in that day (or before)
- A trigger that deletes an entry directly after it has been accessed and sent to the user
- A trigger that deletes an entry only when the user explicitly chooses to delete it
This is business decision territory.
2
This is the age old question of pushing vs pulling. Pushing means that the application takes steps to inform the user of the job completion. Pulling (also called polling) means that the user repeatedly queries the application to be informed of the job completion.
In HTTP, everything tends to work based on the end user making a request, which favors the polling approach whereby the user repeatedly asks the server if their job is done.
However, you could opt for pushing, but this is something that a web application isn’t inherently set up to do. Websockets are what most people think about here (SignalR being a reasonably popular example), but even just sending an email or text message to the user counts as a “push” interaction here. In essence, if your application initiates the communication that the job is finished, it’s a push, not a pull.
How you notify the user is up to you. Generally speaking, polling is much easier to implement, and the performance impact having some additional poll requests for unfinished jobs tends to be negligible.
It’s relevant to note here that push/pull implementations don’t necessarily have to be visibly different to the user. For example, let’s say I start a job, and the webpage opens a little progress window where it informs me of the job progress. While this progress updates very quickly, I have no way of knowing whether this is because the server frequently pushes progress updates, or whether the webpage frequently polls the server. I don’t know, and quite frankly I don’t really care.
As an end user, all I care about is that I am reasonably kept up to date, no matter the underlying implementation. As a developer, I may be inclined to favor push over pull (or vice versa), but I am able to effectively hide my decision from the end user if I so choose.
3
There are several terms here, depending on what specifically you’re looking for.
Though I consider the term to only specifically refer to the non-zero replication time between write and read instances, Wikipedia seems to define eventual consistency as the theoretical concept of separating your read/write instances in its entirety. I disagree, but maybe I’m in the wrong here. As far as I’m convinced, the name for the general concept is better described as a distributed database (though this name also includes different kinds of separations, not just read/write).
From a code perspective, you may want to look into CQRS, which forces you to separate your write actions (“commands”) from your read actions (“queries”).
Note that this doesn’t inherently force you to use separate read/write database instances. But when you do use separate read/write instances, CQRS helps you organize which user action connects to which database instance.
Note also that even though you’re supposed to only query the read instances, there are fringe cases where you could allow limited querying of the write instance when immediate availability of data is paramount. This is generally considered to be restricted to (internal) logic and jobs, and not (public) end-user web requests.