#decemberAdventure day 9: back to #gopher
Uncategorized
1
Posts
1
Posters
1
Views
-
#decemberAdventure day 9: back to #gopher
A number of years ago I setup my own search tool for the gopherspace. I've not re-done a full crawl since getting it working, and due to some design issues, it eventually stopped working reliably enough to keep running. Due to RSI issues, I have prioritized other projects in the intervening years, but have slowly made a list of notes and plans for fixing it. Today's adventure is the first part in implementing my plan and getting it back to a useable state.
I've rewritten the crawler. The new one is a lot less buggy than the original, and has a number of improvement including a correctly working filter (supporting robots.txt and a defined list of servers not to index), better discovery of servers from the gopher maps, tracking when servers were last scanned, request rate limiting, and facilities for avoiding recording duplicate entries.
My initial tests have been on my main gopher server (forthworks.com:70) and a number of my private ones. This totals 32k selectors across 3 servers. I'm going to start a broader scan of the public gopherspace soon, so will update once I get through the initial scan of a few servers.
My full logs are at https://charles.childe.rs/DA2025
-
undefined stefano@mastodon.bsd.cafe shared this topic on
Gli ultimi otto messaggi ricevuti dalla Federazione
-
@cybersecurity sicuramente l'internazionale nera si compra e distrugge la stampa di opposizione...
-
"A quiet mind is able to hear intuition over fear."
-
You and your smarter friend. | More comics: https://rooniecomics.com/comics/in-some-ways
-
Giornata mondiale del #tango. E questa è stata la sua ultima reincarnazione. #GotanProject 3.0 #Live (2011) - Invidious https://iv.duti.dev/watch?v=vTnBMsrsBI8
-
@Ali_per_Viaggiare ricordate quando la Lancia faceva le macchine "lussuose" (la moquette della y10 era uno spettacolo) l'Alfa quelle sportive, la Maserati quelle super sportive lussuose, la Fiat la panda 4x4 steyr puch., la seat la Marbella per noi poveri... @Otttoz
-
@kigen sempre e comunque #postmeridiem
-
meet me down in the marketplace of ideas and i'll spit right in your mouth before we are both smothered by four cement trucks driven by the last people with a US dollar on earth, who pour a transparent rapid permineralization concentrate over our bodies so a fleeting moment of emotion can be cast to the fossil record and sold as a palatial ornament
-
@marty75 se siamo ancora in tempo...arriviamo siamo in tre.
