Announcing the Data.gov Archive (lil.law.harvard.edu)
from misk@sopuli.xyz to technology@lemmy.world on 09 Feb 2025 16:22
https://sopuli.xyz/post/22430187

#technology

threaded - newest

tastysaganaki@reddthat.com on 09 Feb 2025 16:44 next collapse

🙏

just_another_person@lemmy.world on 09 Feb 2025 17:03 next collapse

At least private institutions are stepping up. Where’s the rest of the universities?

NaibofTabr@infosec.pub on 09 Feb 2025 18:43 next collapse

Dependent on federal money.

corsicanguppy@lemmy.ca on 09 Feb 2025 23:25 collapse

… Thus, not saying a thing if they do actually cache it.

rbos@lemmy.ca on 10 Feb 2025 03:05 collapse

I work for a Canadian university library and I assure you that data archival efforts are proceeding apace. I can’t really say more without possibly endangering American colleagues.

Bakkoda@sh.itjust.works on 09 Feb 2025 18:12 next collapse

I’ll officially be seeding more of this type of stuff (annas, cdc, etc) than all of my “Linux isos”. Feels good.

Tikiporch@lemmy.world on 10 Feb 2025 04:53 collapse

Did they create a torrent for this, or just straight downloads?

finicky_foyer@sh.itjust.works on 10 Feb 2025 05:10 collapse

It appears to be hosted via S3 compatible object store. Their docs provide examples to browse/download via awscli or boto3 (as well as rclone).

source.coop/repositories/…/description source.coop/repositories/harvard-lil/…/access

tonytins@pawb.social on 09 Feb 2025 18:17 next collapse

Thank goodness. I was hoping for something like this.

errer@lemmy.world on 09 Feb 2025 23:01 next collapse

16TB is within the capacity of individuals. Given I don’t trust even Harvard to be safe with the impending onslaught, anyone with the means of downloading a copy for themselves should do so.

Xanza@lemm.ee on 10 Feb 2025 12:18 collapse

Cold storage is $2 per TB/mo. Could easily be crowdfunded. $410/yr to keep it around.

NudeNewt@lemm.ee on 10 Feb 2025 05:08 collapse

Remember folks, to avoid Linux ISO honeypots you must enable Super-Seeding.

Xanza@lemm.ee on 10 Feb 2025 12:20 collapse

Is there a list of super-seeding complaint clients? I can’t seem to find one. Hell, I can’t even check to see if my client supports it.

NudeNewt@lemm.ee on 10 Feb 2025 15:15 collapse

Most p2p clients support it, you’ll need to refer to their documentation to enable.

For the most part all you need to do to enable it is right click a completed download then select “super seeding” mode. It’s at least that straightforward for QBT.

Xanza@lemm.ee on 10 Feb 2025 15:51 collapse

I never knew. My version of Deluge is 1.3.5, and super seeding requires 2.0+. So that looks like the issue. lol

NudeNewt@lemm.ee on 10 Feb 2025 18:42 collapse

Hmm, in that case I recommend you read into “Bandwidth Tweaking”:

deluge-torrent.org/userguide/bandwidthtweaking/

Xanza@lemm.ee on 10 Feb 2025 18:53 collapse

I’m gonna look into upgrading, but I’ve implemented this as well. Thanks.