I’m new to web dev, but it seems there’s a lot of async operations going on, which can be confusing.

Take this selenium code running on a non-server, local environment:

for url in urls:
    driver=webdriver.Chrome()
    driver.get(url)
    #driver do some web scraping
    driver.quit()

This means for each url, it’s going to spawn a new chrome and chromedriver process, do the web scraping, quit the process, and restart with a new url. If I understand correctly, two chromedriver processes will not exist at the same time.

But if this code is in a server:

@main.route('/dashboard',methods=['POST',"GET"])
def dashboard():
    if request.method=="POST":
        data=request.json

        urls = data['urls']
        for url in urls:
            driver=webdriver.Chrome()
            driver.get(url)
            #driver do some web scraping
            driver.quit()

        return "test"

    return render_template('dashboard.html')

Then it quickly spawns a bunch of chrome and chromedriver processes, and my CPU quickly goes up to 100% usage.

I’m not sure how to restrict this. And I assume flask is running async operations which is causing this.