All Questions
5 questions
3
votes
1
answer
83
views
Multi-Page Web Scraping Code Using Selenium with Multithreading
I have written a web scraping script using Selenium to crawl blog content from multiple URLs. The script processes URLs in batches of 1000 and uses multithreading with the ThreadPoolExecutor to ...
- 31
1
vote
2
answers
157
views
Web scraping spider
I'm currently working on my first web scraping project and I need to scrape a lot of websites. With my current code it takes more than a day but for my project I need to scan the same websites every 5 ...
- 27
0
votes
1
answer
459
views
Instagram Scraper running to slow
Can someone help me how to speed up this scraping process getting all the usernames from the # 'cats' on instagram. My goal is to make this as fast as possible because currently the process is kinda ...
- 21
2
votes
1
answer
176
views
Massive IP2host scanner
I'm writing a "IP2host" resolver. Basically all it does is a NSLookup and output's all domain's on the IP. I'm trying to run this on a world scan that's 40M+ lines of IP's.
I have used basic ...
- 21
1
vote
1
answer
5k
views
Crawl multiple pages at once
This an update to my last question.
I want to process multiple pages at once pulling URLs from tier_list in the crawl_web ...
- 111