![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}
Helpful Archiving Tools (DRAFT)
Oct. 27th, 2020 08:23 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
morgandawnCreating an Permanent Snapshot Of A Website
Very few tools allow you to create a permanent, online copy of an entire website (see below for download tools)
-Wayback Machine - it crawls websites and also allows you to submit a single page "archive on demand" (must be public, not under an adult/age statement splash screen). Is an excellent long term solution, cannot unfold LJ comments. Has a Python tool available to submit bulk URLs. Will archive embedded art or you can run the image URL separately through the on demand tool. You can also archive embedded PDFs or documents, possibly downloadable audio and video files if you run the download link through the on demand tool. Cannot archive streaming video or audio
Neat trick: if you archive a website and then open links inside the archived page, you can then archive those pages without having to manually submit the additional URLs. Still slow (archiving requires 2 clicks)
-Archive.is -- on demand archiving (must be public, not under adult/age statement splash screen). Best for pages the Wayback Machine cannot handle because it is owned by a single person, long term stability is uncertain. No bulk submission, can unfold LJ comments. Will create a screenshot of image including embedded art, but cannot archive audio, video, downloadable PDFs or other docs. Does a slightly better job of archiving Twitter
-Manual screencaps - download to your computer, not online
Possible uses: documenting facts and history on the Fanlore wiki, preserving your favorite fic and art, [others?]
Downloading An Entire Website - TBA
As always, downloading an entire website carries with it both practical and social considerations. From a technical standpoint downloading the website can put a strain on the website owner's web-host causing the site to be shut down or permanently taken offline. Finally, many of these tools are complicated and require customized settings so you don't inadvertently download every website linked from your target website. It can get messy very quickly and I personally do not recommend that the average fan attempt to use them unless they know what they are doing.
Very few tools allow you to create a permanent, online copy of an entire website (see below for download tools)
-Wayback Machine - it crawls websites and also allows you to submit a single page "archive on demand" (must be public, not under an adult/age statement splash screen). Is an excellent long term solution, cannot unfold LJ comments. Has a Python tool available to submit bulk URLs. Will archive embedded art or you can run the image URL separately through the on demand tool. You can also archive embedded PDFs or documents, possibly downloadable audio and video files if you run the download link through the on demand tool. Cannot archive streaming video or audio
Neat trick: if you archive a website and then open links inside the archived page, you can then archive those pages without having to manually submit the additional URLs. Still slow (archiving requires 2 clicks)
-Archive.is -- on demand archiving (must be public, not under adult/age statement splash screen). Best for pages the Wayback Machine cannot handle because it is owned by a single person, long term stability is uncertain. No bulk submission, can unfold LJ comments. Will create a screenshot of image including embedded art, but cannot archive audio, video, downloadable PDFs or other docs. Does a slightly better job of archiving Twitter
-Manual screencaps - download to your computer, not online
Possible uses: documenting facts and history on the Fanlore wiki, preserving your favorite fic and art, [others?]
Downloading An Entire Website - TBA
As always, downloading an entire website carries with it both practical and social considerations. From a technical standpoint downloading the website can put a strain on the website owner's web-host causing the site to be shut down or permanently taken offline. Finally, many of these tools are complicated and require customized settings so you don't inadvertently download every website linked from your target website. It can get messy very quickly and I personally do not recommend that the average fan attempt to use them unless they know what they are doing.
