All Questions
Tagged with web-scraping python-2.7
864 questions
0
votes
1
answer
75
views
Issues to get a loop using a selenium library
I did this code using Selenium and Pyautogui to clean an sharepoint List, based on a excel list. The first time the program is run normal, but I cant make the loop work, I have no ideia what i do. ...
0
votes
1
answer
611
views
TypeError: __init__() got an unexpected keyword argument 'options' error extracting Google Trends blog post titles using Selenium and BeautifulSoup4
Right now I have this python code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common....
0
votes
4
answers
1k
views
how to extract words from the string in the list in python?
I have a string of type
string = "[A] Assam[B] Meghalaya[C] West Bengal[D] Odisha "
Output = ['Assam', 'Meghalaya','West Bengal','Odhisa']
I tried many ways, but I always end up splitting ...
2
votes
1
answer
339
views
Web Scraping with Beautifulsoup python - cannot scrap all results
i tried to scrap data from this site
https://www.nobroker.in/property/sale/chennai/Adyar?searchParam=...
- 23
0
votes
1
answer
875
views
How to scrape all the page without suddenly being detected as a bot using python
What I am trying to do is to scrape a restaurant using the given URL from the database. The host is https://www.just-eat.co.{tenant}. Then from the response I will get the window.__INITIAL_STATE__ ...
- 583
0
votes
1
answer
118
views
POST Request Python Web Scraping: Get URLs from Tag
Hi i am new to both python and web scraping. From my scripts, I want to POST Request and then get the urls from it. But I got a problem, I cant get the urls from the a tag. When I inspect the website ...
- 3
1
vote
2
answers
2k
views
how to scrape author name and author url from a webpage using python
i am trying to scrape author name and author url from the following webpage.
https://medium.com/javascript-scene/top-javascript-frameworks-and-topics-to-learn-in-2019-b4142f38df20?source=tag_archive
...
- 301
1
vote
0
answers
68
views
How to scrape a webpage that uses JavaScript to build the HTML? [duplicate]
I have a scraper made in python2 using requests and beautifulsoup. We do not want to use selenium. Now the website we are scraping has changed its some page. The page is now building the elements on ...
0
votes
2
answers
118
views
How to Data Scrape from multiple pages
import os
from webdriver_manager.chrome import ChromeDriverManager
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys ...
- 19.2k
-1
votes
1
answer
43
views
Is there a way I can scrape this web page with selenium?
So I want to scrape a list of Confirmed Participate from this site but it's returning an empty list/string
Here is my code:
driver.find_element_by_xpath('//*[@id="main"]/div/section/section[...
0
votes
2
answers
439
views
Beautiful Soup and requests problem it doesn't show any text output
I am using beautiful soup and requests to print full text of the article of this wedsite
https://www.vanityfair.com/style/society/2014/06/monica-lewinsky-humiliation-culture
This is my code:
import ...
- 37
0
votes
1
answer
39
views
how to append in excel sheet and process further
Below program will open the excel file I saved in local, and open the url one by one in delay of 2 seconds.
I need to change with item name instead of URL.
Example: previous input is https://www....
- 35
2
votes
1
answer
597
views
How to select all elements based on alternative attributes? [BeautifulSoup]
I have to select all html-tags that has class attribute if the class name is in list OR has a style attribute with the specified value.
I tried to solve my problem step by step and I got this:
...
- 811
2
votes
1
answer
129
views
Beautiful Soup Scraping
I'm having issues with old working code not functioning correctly anymore.
My python code is scraping a website using beautiful soup and extracting event data (date, event, link).
My code is pulling ...
- 570
0
votes
1
answer
399
views
Python - Web scraping using Scrapy
Just started learning web scraping using scrapy framework. I am trying to scrape reviews of a medicine from a medicinal website using the below code. But if i run "scrapy runspider spiders/...
- 3