How to deal with Dynamic cookies when web crawling
I am trying to access data(quote value) from an E-commerce website using the ‘requests’ library in python. The problem I have is that the cookies in the website are dynamic. And my code requires a header to get a response.
I can open the website and scrape it but to do that I need to copy the header details from the response header. However I need to automate this process so that I don’t need to manually put the cookie in every time I want to scrape. This the link “https://www.nseindia.com/get-quotes/equity?symbol=RELIANCE”. I am trying to get the ‘Intraday chart’ data so I can store it in a DataFrame and plot it.