How to handle google consent page when scrapping article data using google news RSS links?
I have a list of google news links from google RSS feed and I would like to get full text of those articles. I use BeautifulSoup library to scrape the data, however, it seems that google redirects to the cookie consent page first. What is the best way to handle the consent step? Is there a way to bypass the consent page or programmatically accept/reject?