I use network namespaces to isolate my tests of a distributed system.

Basically, every test case has one linux bridge(created with brctl), and many processes, each run in its own network namespace(created with ip netns).

The process’s namespace is connected to bridge with veth pairs(created with ip link): one side of the pair is added to the bridge, and the other added to the process’s namespace.

I find if the number of concurrently running test cases exceeds certain number, my program would produce network timeout errors. I’d like to get help on how to debug/tune my machine so I could run more tests at the same time.

Here are some observations:

• a better CPU could lead to more concurrences
• when running tests, sometimes CPU load > number of cores
• when network timeout happens, I tried to ping and also got timeout
• as the number of ports on one bridge increases, the situation seems worse