新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Apache Nutch is an extensible and scalable web crawler

A scalable, mature and versatile web crawler based on Apache Storm

ACHE is a web crawler for domain-specific search.

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

News crawling with StormCrawler - stores content as WARC

A set of reusable Java components that implement functionality common to any web crawler

Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or filesystem to various data repositories such as search engines.

Continuous scalable web crawler built on top of Flink and crawler-commons

Common Crawl fork of Apache Nutch

Open Source Web Crawler for Java - A fork of yasserg/crawler4j

Java based web-crawler program which makes use of pool based multi-threading, simple UI with Swing and jsoup to nested web crawling

Apache Nutch is an extensible and scalable web crawler

web crawler allowing full page render crawl using HtmlUnit

🕷 a flexible web crawler framework

A Library for web crawling websites harvesting URLs of embedded links and images

这是一个基于Spring Boot框架开发的链家房源数据爬虫系统。本项目致力于为用户提供一个便捷、高效的房源数据采集解决方案。通过自动化爬取链家网站的房源信息，系统能够实时获取各个城市的房源详情，包括房屋价格、位置、面积、户���等关键信息。

Java Web Crawler Program to get all links or images download from websites and use Google or Bing search options .

This is pretty basic example of web page crawling in java and is not fully production ready crawler and is done for test purposes only