2016-02-11 | Christian Weiske | support base href | blob | commitdiff | raw |
2016-02-11 | Christian Weiske | sanitize title better | blob | commitdiff | raw | diff to current |
2016-02-11 | Christian Weiske | use correct meta robots attribute | blob | commitdiff | raw | diff to current |
2016-02-10 | Christian Weiske | crawler supports "nofollow" now | blob | commitdiff | raw | diff to current |
2016-02-10 | Christian Weiske | rework crawler; add atom link extraction | blob | commitdiff | raw | diff to current |