Stolenhistory.net Archives for Download

dakotamoon

Member
Joined
Dec 5, 2020
Messages
24
Reaction score
94
Much appreciated, I'm trying to install the linux app, but may have to go back to my one windows pc to complete the install. Thanks again.
 

KeeperOfTheKnowledge

Active member
Trusted Member
Joined
Aug 26, 2020
Messages
20
Reaction score
135
Location
Minnesota
@KeeperOfTheKnowledge amazing work my friend! What are you using to scrape it? We have httrack working pretty well for other sites.
I use only basic Linux shell scripting with wget and aria2c. I write scripts to scrape forums for user posted links, then archive all the links as well as the forums. I intend to write an indepth post of what and how I archive for those who are interested in archiving their own data.
 

luddite

Administrator
Staff member
Trusted Member
Joined
Aug 26, 2020
Messages
135
Reaction score
445
I scrape the cookie or embed credentials. Both methods work.
 

Seven823One

Member
Trusted Member
Joined
Sep 16, 2020
Messages
19
Reaction score
36
Location
Virginia, USA
Site:
twistatweet.com
I am using Windows version WinHTtrack 3.49-2
The only mention of cookies in its config is on "Spider" tab, where there is a checkbox to accept cookies... The login password could be entered in "Insert URL" popup window. I was afraid there is a cookie timeout in play, since it took 2-3 days for my connection to download 3.83Gb (78K files) of data from the forum.
 

luddite

Administrator
Staff member
Trusted Member
Joined
Aug 26, 2020
Messages
135
Reaction score
445
I am using Windows version WinHTtrack 3.49-2
The only mention of cookies in its config is on "Spider" tab, where there is a checkbox to accept cookies... The login password could be entered in "Insert URL" popup window. I was afraid there is a cookie timeout in play, since it took 2-3 days for my connection to download 3.83Gb (78K files) of data from the forum.
I will PM you about this.
 

Similar threads

Users who are viewing this thread

Top