#decemberAdventure day 9: back to #gopher
Uncategorized
3
Posts
1
Posters
4
Views
-
#decemberAdventure day 9: back to #gopher
A number of years ago I setup my own search tool for the gopherspace. I've not re-done a full crawl since getting it working, and due to some design issues, it eventually stopped working reliably enough to keep running. Due to RSI issues, I have prioritized other projects in the intervening years, but have slowly made a list of notes and plans for fixing it. Today's adventure is the first part in implementing my plan and getting it back to a useable state.
I've rewritten the crawler. The new one is a lot less buggy than the original, and has a number of improvement including a correctly working filter (supporting robots.txt and a defined list of servers not to index), better discovery of servers from the gopher maps, tracking when servers were last scanned, request rate limiting, and facilities for avoiding recording duplicate entries.
My initial tests have been on my main gopher server (forthworks.com:70) and a number of my private ones. This totals 32k selectors across 3 servers. I'm going to start a broader scan of the public gopherspace soon, so will update once I get through the initial scan of a few servers.
My full logs are at https://charles.childe.rs/DA2025
-
#decemberAdventure day 9: back to #gopher
A number of years ago I setup my own search tool for the gopherspace. I've not re-done a full crawl since getting it working, and due to some design issues, it eventually stopped working reliably enough to keep running. Due to RSI issues, I have prioritized other projects in the intervening years, but have slowly made a list of notes and plans for fixing it. Today's adventure is the first part in implementing my plan and getting it back to a useable state.
I've rewritten the crawler. The new one is a lot less buggy than the original, and has a number of improvement including a correctly working filter (supporting robots.txt and a defined list of servers not to index), better discovery of servers from the gopher maps, tracking when servers were last scanned, request rate limiting, and facilities for avoiding recording duplicate entries.
My initial tests have been on my main gopher server (forthworks.com:70) and a number of my private ones. This totals 32k selectors across 3 servers. I'm going to start a broader scan of the public gopherspace soon, so will update once I get through the initial scan of a few servers.
My full logs are at https://charles.childe.rs/DA2025
Update on the initial scan: 91 servers (of 488 found in the indexes) scanned, 397 pending, 11 unreachable. 579,936 selectors with 2,193,285 descriptions. Data set is 569MiB in size.
I'm stopping my scans for today, will resume them tomorrow.
-
undefined stefano@mastodon.bsd.cafe shared this topic on
-
Update on the initial scan: 91 servers (of 488 found in the indexes) scanned, 397 pending, 11 unreachable. 579,936 selectors with 2,193,285 descriptions. Data set is 569MiB in size.
I'm stopping my scans for today, will resume them tomorrow.
Update on my scan of gopherspace.
725 servers identified
84 unreachable
6 restricted due to robots.txt or manual exclusion requests
325 scanned completely
1 in progress
393 pending1,269,190 unique selectors, 5,699,901 descriptions.
The scan will continue (slowly). I'm going to start writing the new front end for doing searches of the collected data next week.
Gli ultimi otto messaggi ricevuti dalla Federazione
-
@andycarolan £100 a month for these three?!?! OMGGGGGGGGG 😱
This is precisely why I switched to Pixelmator Pro for graphics and DaVinci Resolve for video editing.
These prices are extortionate!!!
-
@bbacc che versione di office?
-
@Iloo thanks! Most people on the Fediverse will already be familiar with privacy and FOSS tools we'll recommend. But we think the podcast could be great for Fedi people's spouses, parents, siblings, cousins and friends who think that adopting these tools is "too hard" or "too time consuming".
Oh and the cherry on top is that we plan to use @Castopod for it 🤗
-
… E tutto questo aggratis!!!
L'ultima versione di Translate supporta già 70 lingue e non richiede dispositivi speciali
Google lancia la traduzione simultanea universale: bastano qualsiasi telefono Android e auricolari - Zeus News
https://www.zeusnews.it/n.php?c=31619
-
@_elena I have looked into it on a few occasions, and was close to moving to Affinity. But then Canva bought them.
I remember you saying about AE. It's definitely most people's go-to app for that kind of thing.
Even between just Adobe, Webflow and Quickbooks, I'm paying over £100 a month 😩
-
@_elena Would definetely listen to a podcast like that 🤩
-
@andycarolan every once in a while some creator does a show & tell of tools they adopted as alternatives to Adobe. I know it's super hard to quit it... in my domain, I argue there is no sophisticated alternative to AfterEffects, especially when it comes to the wealth of ready-made templates. I'm lucky I'm basically on a filmmaking hiatus so I don't have to deal with that...
-
My customer wanted a feature that involves AI integration. The contractor responsible for it handed this task to a couple of interns back in October, and now they made a merge request for this feature that includes 400+ commits and 250+ changed files.
