51,515 questions
-3
votes
0
answers
29
views
What is the best source to extract Australian weather data? [closed]
I'm working on a project that requires me to use the average daily temperature. I've explored http://www.bom.gov.au/climate/data, this is a good source, but the issue is the data is categorized per ...
- 17
0
votes
1
answer
36
views
Can't close cookie pop up on website with selenium webdriver
I am trying to use selenium to click the Accept all or Reject all button on a cookie pop up for the the website autotrader.co.uk, but I cannot get it to make the pop up disappear for some reason.
This ...
- 759
-7
votes
0
answers
25
views
Is there a way to bypass cloudflare within dart? [closed]
I've been coding my web-scraping app for 2 weeks now and after designing and everything I found out that the site i want to scrape started using cloudflare, in my previous app i used python so I ...
0
votes
1
answer
28
views
How to scrape tweet/thread and its replies based on conversation_id
I’m currently working on a project that involves scraping a single tweet and all its replies using tweet-harvest with an auth_token. Everything works fine, but I recently ran into an issue where I can ...
2
votes
0
answers
56
views
python-requests-html render inconsistent result
background:
by default the website is only showing few names and there s a "moreBtn" to generate the full list
code idea:
create Html session, render with script clicking the "moreBtn&...
- 31
0
votes
0
answers
31
views
Rotating Authenticate Proxy In Selenium Using Chrome Driver
import zipfile
import json
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
import time
def ...
-1
votes
2
answers
34
views
Why am I getting no data using BeautifulSoup and requests when scraping a news website?
import requests
from bs4 import BeautifulSoup
url = "https://example-news-site.com"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
}
response =...
-1
votes
2
answers
74
views
I wanted to get the number of playoff games of a list of 200 players from Basketball Reference. The code I wrote is giving me 0 value for all players [closed]
I want to get the number of playoff games played by a list of players. To do that I used Selenium and Beautiful Soup. The result is being saved in a csv file but the values for each of the player is ...
-3
votes
0
answers
28
views
Scrape JavaScript based webpage into Excel [closed]
I am trying to create an Excel macro that will use a URL provided in the Excel file to access a JavaScript based webpage, and copy/paste the text from that webpage into the file.
If it could be done ...
1
vote
2
answers
49
views
Importing geographic data with WFS works on Chrome but not on Python
I am trying to pull a geojson file from here.
The JSON appears as expected when I paste that link into Chrome or Safari. However, I get the following error every time when I run the following code on ...
- 121
0
votes
0
answers
52
views
Why can't I click this button in the ChatGPT UI using standard JavaScript? [closed]
I want to script Safari to automate some actions in ChatGPT. To achieve that I need to click plus-button in the input field.
Usually when I try to do something like this I use JavaScript like
var ...
- 1
0
votes
0
answers
21
views
yfinance returning empty dataframes
I have a simple yfinance program that is supposed to get financial statements for any company I pick. I run the code and it just returns an empty dataframe. My yfinance and everything else is up to ...
- 23
-3
votes
0
answers
25
views
I tried to parse the page but always get duplicate texts [closed]
I can get the multiple pages parsed text. However, it will have duplicate paragraphs texts.
For example, the content of the first page, it will be parsed 3 times in total.
I use the python code and I ...
0
votes
0
answers
57
views
Scraper crate can't match <tr> without <table> – why?
I'm trying to write a simple web scraper using the scraper crate to learn Rust, and I encountered a weird (for me) problem. My find_element function can't find <tr> elements unless they're ...
-1
votes
0
answers
67
views
How to scrape the full New York Times article content using Selenium and BeautifulSoup without triggering the "Please enable JavaScript" message?
I'm building a scraper that fetches full article content from the New York Times using both the Article Search API and a hybrid static + Selenium-based HTML scraper. My goal is to extract complete ...
