I’m building a system where I need to measure certain algorithms, which are written by the end users. Obviously running external code is a huge security risk, therefore it needs to be isolated. The current solution is to start up docker containers for each submission, run the code inside the container, then terminate it. In terms of scalability, this solution seems rather limited, it already stutters under moderate stress tests. What I had in mind for improvements:

• Container pooling. Having a pool of containers started, kept alive and assigned to a session when it’s needed. I don’t like the idea of sharing containers between users, so once the session is done, the container should be replaced by a new instance in the pool.

Would this solve anything or I’m just increasing the complexity? Should I approach this issue from a completely different angle?