I am writing a python script that can automate fetching of content from page. One function is this:
def extract_prices(elements, wait):
parent_element = None
child_element = None
prices = []
for element in elements:
parent_element = element
child_element = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, f"[style='color: rgb(44, 44, 45);']")))
wait.until(EC.element_to_be_clickable(child_element))
child_element = parent_element.find_element(By.CSS_SELECTOR, f"[style='color: rgb(44, 44, 45);']") # line 1
price = child_element.get_attribute("innerHTML")
util.alert(f"{element}, {price}") # line 2
prices.append(price)
return prices
Line 1 will always raise StaleElementReferenceException
if I comment line 2. However, if I leave line 2 as it is now, the function will run perfectly. I’m 100% sure that
- the query (
(By.CSS_SELECTOR, f"[style='color: rgb(44, 44, 45);']")
) is valid, because at some time before this was working perfectly, I can’t recall what did I changed; and - the DOM hasn’t changed because I tried replacing the
wait.until()
withtime.sleep(20)
, which should be more than sufficient for the webpage to stablise, but the issue persist.
I tried quite a lot solutions from ChatGPT but none succeeded, and I don’t really know how to google my issue because I can’t even describe it in a short query.
FYI, the util.alert()
is here:
import tkinter as tk
from tkinter import messagebox
def alert(msg, title="Alert"):
root = tk.Tk()
root.withdraw()
messagebox.showinfo(title, msg)
I also tried replacing line 2 with time.sleep(1)
but the exception is still being raised. I was suspecting that the dialog window provided time for the webpage to stabilise, but it seems like my theory is wrong.
Thank you in advance!
New contributor