scraping javascript content using selenium

Question

I am trying scrape javascript content from a website using selenium and geckodriver but I am getting any data. Following is the javascript code

<div _ngcontent-c2="" class="header-wrapper">
    <div _ngcontent-c2="" class="title">Suda Office</div>
    <div _ngcontent-c2="" class="update">Jul 05 11:07 AM</div>
</div>

<div _ngcontent-c2="">
    <div _ngcontent-c2="" class="item-row title-headers">
        <div _ngcontent-c2="" class="item-col head1">Route</div>
        <div _ngcontent-c2="" class="item-col head2">Destination</div>
        <div _ngcontent-c2="" class="item-col">
            <div _ngcontent-c2="" class="head3 head3-height">ETA</div>
        </div>
    </div>

    <div _ngcontent-c2="">
        <div _ngcontent-c2="" class="alternet-color">
            <div _ngcontent-c2="" class="item-row item-eta-row">
                <div _ngcontent-c2="" class="item-col eta-route">15 T</div>
                <div _ngcontent-c2="" class="item-col eta-destination">
                    <marquee _ngcontent-c2=""> Charbagh</marquee></div>
                <div _ngcontent-c2="" class="item-col eta-col">                
                    <div _ngcontent-c2="" class="eta-display-wrapper">
                        <div _ngcontent-c2="" class="display">
                            <span _ngcontent-c2="" class="space"></span>
                            <span _ngcontent-c2="" class="currentTiming">10 min</span>
                        </div>

                    </div>
                </div>
            </div>
        </div>
    </div>
</div>

I have to get class="item-col eta-route", class="item-col eta-destination" and class="currentTiming" data from the above javascript content. I use following code but it doesn't show anything in output

from selenium import webdriver
driver = webdriver.Firefox()
driver.get(url)
a = driver.find_elements_by_class_name("item-col eta-route")

But a=[] is output. Even d = driver.find_elements_by_class_name("currentTiming") gives following output

[<selenium.webdriver.firefox.webelement.FirefoxWebElement (session="6b1f2344-8e8a-4f48-a29a-54610179d62f", element="38e7ce58-ea66-4461-bee7-f81ac414595b")>]

How can I get proper output from page using selenium?

Answer 1

The problem could be with item-col eta-route class name. There can be hundreds of classes like that in your HTML.

You can try this css selector instead :

div[_ngcontent-c2][class='item-col eta-route'] 

to get the 15 T value.

Introducing webdriver wait will be great idea for stability of your script.

wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[_ngcontent-c2][class='item-col eta-route']")))
print(element.text)  

For Extracting value :

marquee_text = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[_ngcontent-c2][class='item-col eta-destination'] marquee")))
print(marquee_text.text)    

You would need to import these :

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC