#decemberAdventure day 9: back to #gopher A number of years ago I setup my own search tool for the gopherspace. I've not re-done a full crawl since getting it working, and due to some design issues, it eventually stopped working reliably enough to keep running. Due to RSI issues, I have prioritized other projects in the intervening years, but have slowly made a list of notes and plans for fixing it. Today's adventure is the first part in implementing my plan and getting it back to a useable state. I've rewritten the crawler. The new one is a lot less buggy than the original, and has a number of improvement including a correctly working filter (supporting robots.txt and a defined list of servers not to index), better discovery of servers from the gopher maps, tracking when servers were last scanned, request rate limiting, and facilities for avoiding recording duplicate entries.My initial tests have been on my main gopher server (forthworks.com:70) and a number of my private ones. This totals 32k selectors across 3 servers. I'm going to start a broader scan of the public gopherspace soon, so will update once I get through the initial scan of a few servers.My full logs are at https://charles.childe.rs/DA2025