Skip to main content
Updated Answer
Source Link
undetected Selenium
  • 194.7k
  • 44
  • 306
  • 387

As you mentioned, you have set WebDriverWait(browser, 30) so effectively your code will be looking like :

browser = webdriver.PhantomJS()
def get_score(url):
    browser.get(url)
    elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))
    description_score = float(elements[0].text)
    service_score = float(elements[1].text)
    logistics_score = float(elements[2].text)   
    

Logically, there is no error in your code block but the the program just wait there because when you invoke get(url) the web client i.e. PhantomJS Browser doesn't returns back document.readyState = "complete" so early. The JavaScript and the AJAX Calls still keeps loading. Hence Page Loading gets elongated.

Once the document.readyState = "complete" is returned by the web client i.e. PhantomJS Browser then only Selenium executes the next line of code:

elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))

Hence the program just wait there for some more time.

Update :

As per your comment you need to look at the options of pageLoadStrategy either setting pageLoadStrategy to eager or none as per this QA Don't wait for a page to load using Selenium in Python

As you mentioned, you have set WebDriverWait(browser, 30) so effectively your code will be looking like :

browser = webdriver.PhantomJS()
def get_score(url):
    browser.get(url)
    elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))
    description_score = float(elements[0].text)
    service_score = float(elements[1].text)
    logistics_score = float(elements[2].text)   
    

Logically, there is no error in your code block but the the program just wait there because when you invoke get(url) the web client i.e. PhantomJS Browser doesn't returns back document.readyState = "complete" so early. The JavaScript and the AJAX Calls still keeps loading. Hence Page Loading gets elongated.

Once the document.readyState = "complete" is returned by the web client i.e. PhantomJS Browser then only Selenium executes the next line of code:

elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))

Hence the program just wait there for some more time.

As you mentioned, you have set WebDriverWait(browser, 30) so effectively your code will be looking like :

browser = webdriver.PhantomJS()
def get_score(url):
    browser.get(url)
    elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))
    description_score = float(elements[0].text)
    service_score = float(elements[1].text)
    logistics_score = float(elements[2].text)   
    

Logically, there is no error in your code block but the the program just wait there because when you invoke get(url) the web client i.e. PhantomJS Browser doesn't returns back document.readyState = "complete" so early. The JavaScript and the AJAX Calls still keeps loading. Hence Page Loading gets elongated.

Once the document.readyState = "complete" is returned by the web client i.e. PhantomJS Browser then only Selenium executes the next line of code:

elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))

Hence the program just wait there for some more time.

Update :

As per your comment you need to look at the options of pageLoadStrategy either setting pageLoadStrategy to eager or none as per this QA Don't wait for a page to load using Selenium in Python

Source Link
undetected Selenium
  • 194.7k
  • 44
  • 306
  • 387

As you mentioned, you have set WebDriverWait(browser, 30) so effectively your code will be looking like :

browser = webdriver.PhantomJS()
def get_score(url):
    browser.get(url)
    elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))
    description_score = float(elements[0].text)
    service_score = float(elements[1].text)
    logistics_score = float(elements[2].text)   
    

Logically, there is no error in your code block but the the program just wait there because when you invoke get(url) the web client i.e. PhantomJS Browser doesn't returns back document.readyState = "complete" so early. The JavaScript and the AJAX Calls still keeps loading. Hence Page Loading gets elongated.

Once the document.readyState = "complete" is returned by the web client i.e. PhantomJS Browser then only Selenium executes the next line of code:

elements = WebDriverWait(browser, 30).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='shopdsr-score-con']")))

Hence the program just wait there for some more time.