Norconex Crawler V4 is here — in beta. It's a ground-up rewrite of our open-source web and file system crawler, and it's ready for real-world testing.
A few highlights:
- A free visual configurator — build and validate a crawl right in your browser, no installation required.
- Official Docker images —
docker runand you're crawling. - Built-in clustering — distribute a single crawl across multiple nodes.
- Documentation the community can edit directly — it lives right in the source repository, open to pull requests, not a separate wiki nobody keeps current.
It's feature-complete, and we're inviting the community to try it and tell us what breaks before it goes final.
For the full technical write-up — architecture, migration notes, everything — head over to our detailed release post.
And if you're running V3 in production, it just got a refresh too: V3.2.0 synchronizes all thirteen modules under one version for the first time, published to Maven Central with updated documentation.
