rework crawler; add atom link extraction