All Questions
23 questions
0
votes
1
answer
70
views
Python Requests GET with Proxy - HTTPS scheme returns expected result but HTTP returns header
When setting any URL to use HTTPS as the scheme (i.e., https://), I get my desired response (i.e., page source), but any http url (i.e., http://) fails or I receive a header and I don't understand why ...
- 73
1
vote
1
answer
110
views
ERR_TUNNEL_CONNECTION_FAILED for selenium webdriver HTTPS scraping attempt
I'm currently working on a scraping project and facing some issues with using a proxy from Webshare. This is my first time using a proxy, and I can't seem to access the target webpage.
Every time I ...
0
votes
1
answer
35
views
Taking a list from a webpage and Displaying It?
I am working on a program that would display a wide range of martial arts. I figured that web scraping would be the best way to do this so that as things grow and evolve, the program would, hopefully, ...
- 1
0
votes
1
answer
293
views
ConnectionResetError with Python requests and urllib libraries when accessing specific URL
I'm encountering a ConnectionResetError when attempting to access a specific URL using both Python requests and urllib libraries. Despite providing appropriate headers, the connection is being ...
- 13
0
votes
1
answer
46
views
Item not clickable
So I have a task I have a site and there is a button I need to click and take all the info from the next page that opens. My problem is that I cant access the item(I did some research on what the ...
- 1
1
vote
1
answer
113
views
How to get around anti-adblock html page while scraping using python requests library?
I'm trying to automate a clerical report that I have been assigned and a part of that involves getting the zip code from a street address. I have tried a bunch of stuff like the geopy library and I ...
- 155
1
vote
2
answers
76
views
how to parse data from a specific website
I'm trying to parse data from this page: https://rules.art/card/jmks-season-1-common
You can find hereunder the basic code I'm using to try to retrieve some data (e.g. the card's name "JMK$")...
- 11
0
votes
0
answers
69
views
Solution added to the answer: Web Scraping Python AttributeError: 'NoneType' object has no attribute 'group' Web Scraping Python
There are two python files.
The file titled munilist.py looks as follows (with around 7000+ municipality name-URL pairs):
munilist = ('porto-empedocle', 'https://openbilanci.it/armonizzati/bilanci/...
- 159
0
votes
3
answers
55
views
Cannot do webscraping with Selenium because recognised as bot
I am trying to webscrape with Python "https://www.futbol24.com/" and I am recognised as bot. I tried everything, including the removal of signatures in the javascript of chromedriver.exe, or ...
- 156
0
votes
0
answers
61
views
Unsure of what this http response is (Phone Application Data Scraping)
I am trying to scrape messages off of this phone application (similar to a messaging application) and I was using burp suite to obtain https request information and all the information returned is in ...
- 5
0
votes
1
answer
1k
views
how can i reduce bandwidth usage when making hhtp/https requests to a website
I am running a small personal python service that uses a paid rotating proxy with limited bandwidth to make requests to websites and scrapes data from websites that have no api.
My question is now how ...
- 33
1
vote
2
answers
2k
views
Python urllib request always results in Error 400: Bad Request
Thanks for reading. For a small reserach project, I'm trying to gather some data from KBB (www.kbb.com). However, I'm always getting a "urllib.error.HTTPError: HTTP Error 400: Bad Request" ...
- 301
0
votes
0
answers
44
views
Url requests not working while the flask app is hosted
I have a flask web app running a just-dial scraper code, In my code, I have to request multiple pages of the Justdial site to use it in the bs4 module to extract the data and fill it in the excel ...
1
vote
1
answer
589
views
Extracting data from the ECB Data Warehouse
I would like to extract with Pandas a given value of the ECB Data Warehouse in this page.
I have been going through the ECB help page but could not find the way to make it work.
I have tried the ...
- 37
0
votes
0
answers
83
views
Unable to scrape "https" in python 3.7 using beautifulsoup 4.6.3 (Anaconda)
Failing to scrape odds from sportsbet.com.au as it is throwing an exception.
I cannot read the exception because the terminal closes too quickly. However, I believe the error is due to scraping from ...
