| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2016-08-30 | tell why crawler stops | Christian Weiske | |
| 2016-08-30 | Add crawlBlacklist configuration option | Christian Weiske | |
| Resolves: #7 | |||
| 2016-08-30 | Fix notice | Christian Weiske | |
| 2016-08-30 | Option to disable linked URL indexing | Christian Weiske | |
| Resolves: #2 | |||
| 2016-08-29 | Send If-Modified-Since header on crawling and indexing | Christian Weiske | |
| 2016-02-11 | debug option for crawler | Christian Weiske | |
| 2016-02-10 | remove debug statement | Christian Weiske | |
| 2016-02-10 | crawler supports "nofollow" now | Christian Weiske | |
| 2016-02-10 | send accept header during crawl | Christian Weiske | |
| 2016-02-10 | rework crawler; add atom link extraction | Christian Weiske | |
