All Questions
Tagged with web-scraping javascript
3,651 questions
-6
votes
0
answers
35
views
Is there a way to bypass cloudflare within dart? [closed]
I've been coding my web-scraping app for 2 weeks now and after designing and everything I found out that the site i want to scrape started using cloudflare, in my previous app i used python so I ...
- 11
0
votes
0
answers
52
views
Why can't I click this button in the ChatGPT UI using standard JavaScript? [closed]
I want to script Safari to automate some actions in ChatGPT. To achieve that I need to click plus-button in the input field.
Usually when I try to do something like this I use JavaScript like
var ...
- 1
-1
votes
0
answers
70
views
How to scrape the full New York Times article content using Selenium and BeautifulSoup without triggering the "Please enable JavaScript" message?
I'm building a scraper that fetches full article content from the New York Times using both the Article Search API and a hybrid static + Selenium-based HTML scraper. My goal is to extract complete ...
1
vote
2
answers
60
views
Puppeteer is not scraping the last page
I'm scraping a news article for my project using Puppeteer. I'm unable to scrape the last page. There are 10 pages and each page has 10 links (total 100 data). However, I’m noticing that sometimes it ...
- 23
1
vote
1
answer
37
views
How to scrape html and JS without WebView in background when the application is not running in android
I want to scrape the data from the webpages and run a script java also using background mechanism like service in Android App. I can do that using WebView but then it needs to run on main thread.
I ...
- 291
1
vote
0
answers
49
views
How can I extract the complete content from a CodeMirror editor using the browser console?
I'm trying to extract the full code from a CodeMirror editor on chatgpt.com using the browser console. My current approach only retrieves the visible portion of the code, likely because CodeMirror is ...
- 474
1
vote
2
answers
97
views
Can't find button using playwright (or puppeteer) for web scraping
There are many similar questions (like this: Scraping Websites With Playwright), yet I did not find and solution to this:
I have this url:
https://www.derstandard.at/search?n=&fd=2025-02-17&td=...
- 1,499
0
votes
0
answers
69
views
How to Extract Code Blocks from Different Tabs in a Code Documentation Using Crawl4AI (or any other tool)?
I'm trying to scrape code blocks from multiple tabs in a documentation page using Crawl4AI. While I'm able to extract Markdown content, the code blocks inside tabbed sections are not being captured.
...
- 23
0
votes
0
answers
35
views
How to fill an input and select an option from the dropdown using Puppeteer?
I am working on a project with JavaScript and chose Puppeteer to perform web scraping on various websites. One of the websites I need to scrape is this one Mapas SII, from which I plan to obtain ...
2
votes
1
answer
95
views
Puppeteer Scraping: See XHR response data before request completes for real time data
I am using puppeteer to scrape a website for real time data in nodejs. Instead of scraping the page, I am watching the backend requests and capturing the JSON/TEXT responses so I have more structured ...
- 21
0
votes
1
answer
98
views
Why can't I extract listings information
I am trying to extract the EPC rating from each listings. you can only get the EPC rating when you click on the listing. each time i run my script it
it's keeps timing out, what could be the issue ? ...
0
votes
1
answer
142
views
Can't use captcha solver api services due to missing the element "data-sitekey"
I'm tryingto to use a captcha solving service but all of them request a code that's attached to an element called "data-sitekey" that should be in the HTML of a page with recaptcha on it. ...
- 1
1
vote
1
answer
172
views
Automation Script to Connect people on LinkedIn
https://github.com/Gurukishore-G/Linkedin-Automation
Here is the url to the automation script that I use in browser console to automate the sending of connection requests in Linkedin. The script sends ...
0
votes
0
answers
39
views
Mark telegram message as read using js in console
I have a request from client to read messages from telegram web page (scraper) and send them to the other telegram chat. I have created the parser and all other parts and even added the scrolling. But ...
0
votes
1
answer
72
views
Puppeteer "Target Close" crash at random points
I have this little puppeteer script that should scrape the specific website, get some info from it, navigate through pagination and then print the data. But for some reason, it keeps crashing at a ...
