[Userscript] Booru Sync - Syncing favorites and upvotes across sites

Marker
Boot badge - It's Bootiful
Fried Chicken - Attended an April Fools event
Liberty Belle - Sings the song of the unchained

Misanthrope
Booru Sync
 
Requires Violentmonkey or compatible userscript manager.  
Click here to install
 
Instructions  
The button for opening the UI could be found next to the upload button.  
Before you start, make sure you’re logged in to the sites you want to sync to, and have all the API keys filled in.
 
The script trys to identify identical images on other boorus by the image’s hash value. Normally it is done by simply requesting the data using the site API. However there an issue with Derpibooru that makes it returns only the hash value for pre-optimized images on many older uploads.
 
The “Enable fallback” setting tries to mitigate this by employing two additional methods to find matching images when the normal method failed to yield result:
 
  • Client-side hash calculation: Downloads the full image/video and calculates the hash value on the browser.  
  • Reverse image search: Send the file url to the target booru for reverse image search, and potential matches are listed in the final report for manual review. (Not availiable for Twibooru)
     
    Bewarned that these methods are relatively bandwidth intensive, depending on the number of images the script needed to check. Use it sparingly.
     
    The “Filter by tags” field allows you to sync only selected images from the source booru. For example, you can use created_at.gt:7 days ago to sync only images uploaded in the past week. Or you can use -holding hooves to keep all your hoof holding images contained to one site (you weirdo).
     
    full
     
    Known/Unsolvable issue  
  • If you are syncing to Twibooru, you must run the script on the site itself.  
  • When downloading files for client-side hashing, the connection will frequently timeout on large videos. When this happens, a list of failed downloads will be included in the report generated at the end.
Anonymous #B990
I now have a reason to make accounts on all the altboorus. The question now is whether I should keep a consistent name.
Anonymous #02DC
Changelog for 1.1.0:  
  • Clickable links in the log outputs!  
  • Fixed an error with client-side hash search when there are multiple sync destinations.
Anonymous #B990
Where is the UI element to get the booru sync to appear? I installed it (as well as Violent Monkey) and don’t see anything obvious. It’s Firefox ESR under Debian if that makes any difference.
Anonymous #B990
@Marker  
I think either my browser or my Violent Monkey install is broken, as it does not appear on any site even though Violent Monkey says it is enabled on this site.
Marker
Boot badge - It's Bootiful
Fried Chicken - Attended an April Fools event
Liberty Belle - Sings the song of the unchained

Misanthrope
@Anonymous #B990  
Yes, this is odd. I tested on a fresh install of Firefox ESR portable edition (Windows) and the button showed up just fine.  
Are there any error messages in the dev console? Do you have other scripts installed?
Marker
Boot badge - It's Bootiful
Fried Chicken - Attended an April Fools event
Liberty Belle - Sings the song of the unchained

Misanthrope
Changelog for 1.2.0:  
Client-side hash calculation are now cached. This should drastically cut down the number of images that needs to be checked on subsequent syncs.
Anonymous #CA88
@Anonymous #B990  
My problem was that I was on an ancient ESR edition of Firefox (blame Debian). Once a new ESR showed up in apt, the sync finally showed up. For whatever reason, syncing to Twibooru didn’t want to work at all (lots and lots of 403 errors). The other two had great success.
Anonymous #5320
Is it possible to sync images that were unfaved/unupvoted? Perhaps through comparing target images against source images, rather than source against target?
 
Secondly, could a way to export the log to a file be implemented? When using the fallback it can cause the tab to become quite sluggish when interacting with it. Not to mention being easy to lose your place.
Marker
Boot badge - It's Bootiful
Fried Chicken - Attended an April Fools event
Liberty Belle - Sings the song of the unchained

Misanthrope
@Anonymous #5320  
Log exports I can implement. But undoing faves and votes sounds very difficult to tackle. Consider how the script should handle the situation where an image is faved on the target booru, but doesn’t exist on source.
 
Simply unfaving every single image that could not be matched means limiting your faves to images that exists on both sites. In addition, image matching is not 100% reliable. It’s possible for the sites to have images that are visually identical, but possess different file signatures. Or sometimes a match can only be found through the fallback search methods, meaning it has to be force enabled or running the risk of losing faves.
 
So what if the script only unfaves images that could be found on the source booru, but isn’t faved/upvoted? Well, there is still the risk of data loss if a faved image on the target booru is later uploaded to the source. The script will see the image on the source booru is unfaved, and thus unfaving it on the target booru.
 
So unfortunately, I don’t intend to implement this feature. It’s just too much effort and far too error prone. If you only need to unfave/unvote images on an occasional basis, I would recommend my Booru Switcher script as a workaround. Using it on an image page will redirect you to the same image on the other sites, where you can manually unfave the image.
Marker
Boot badge - It's Bootiful
Fried Chicken - Attended an April Fools event
Liberty Belle - Sings the song of the unchained

Misanthrope
A small update with three changes:
 
  • A button to export log to file as plaintext  
  • Cap the number of log entries to be displayed to 1000, older entries get hidden as more lines are added. (Don’t worry, it doesn’t affect log export.)  
  • Cut down on the number of network requests when syncing small number of images: Now the script skips fetching vote interactions from the destination boorus if it determines the number of fetch needed (pages) > the number of images to be synced.
     
     
    @Anonymous #5320  
    Sorry it’s taken me this long to implement log export. Once I lose momentum it takes a ridiculous amount of effort to get me started again.
Anonymous #3CE1
After reading, it looks like downvotes aren’t imported too. Is there a chance that could also be implemented? I have less than 4k IDs downvoted because I don’t do it that much and I use the fucking filters. Or would it be due to those who just carpet-bomb tags that you haven’t added that feature at the moment?
 
A separate question, because I know it’s a thing that happens. Does it also keep faves and upvotes separate? Because there’s times where someone faves something, but removes the upvote or downvotes it. I figure it’s more of an edge case, but I figure I should ask.
Marker
Boot badge - It's Bootiful
Fried Chicken - Attended an April Fools event
Liberty Belle - Sings the song of the unchained

Misanthrope
@Anonymous #3CE1  
Hey there.  
I didn’t implement syncing downvotes because I personally treat the faves and upvotes more akin to bookmarks, so having the downvotes synced just isn’t very useful to me, not to mention adding another piece of complexity to my spaghetti code that I have to worry about.
 
>Does it also keep faves and upvotes separate?  
I’m afraid it doesn’t. As you said, it’s kind of an edge case, one I didn’t (and still don’t) want to bother with.
Anonymous #5320
Links for twibooru images try to send you to /images/ID, but /images/ doesn’t work for twibooru.
Anonymous #5320
Twibooru syncing appears to be having issues again. Earlier it was receiving errors for something to the effect of “This API will soon be discontinued, please use the new API”, and at the moment it syncs one image out of however many, and then stops while searching for the second.
Syntax quick reference: **bold** *italic* ||hide text|| `code` __underline__ ~~strike~~ ^sup^ %sub%

Detailed syntax guide