/hydrus/ - Hydrus Network

win32
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v151/Hydrus.Network.151.-.Win32.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v151/Hydrus.Network.151.-.Win32.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v151/Hydrus.Network.151.-.OS.X.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v151/Hydrus.Network.151.-.OS.X.-.Extract.only.tar.gz
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v151/Hydrus.Network.151.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v151.tar.gz

I had a good week. I cleaned and wrote a lot of code. Most of my time went into boring things that are good but don't change the look or feel of the program, but I did also fix a couple of important bugs, and I made tag archives more powerful.

no booru api integration

I wanted to extend my booru parsers to interface with The Booru API™. After some research, I decided not to do it yet because:

1) Although the different boorus that have APIs are obviously trying to follow a few distinct standards, in practise some of them use different calling urls and parameters, and they serve their data in different markup languages and in different layouts. In order to support any decent number, I would have to write a very generalised API that would take a whole load of work.

2) The one thing they all did do was serve tags without namespaces! Most boorus, of course, do not explicitly label their namespaces, and instead colour them on the webpage. I currently harvest this information by looking through the HTML for the CSS class and prepending the contained text with 'series:' or whatever. So, absent some overly complicated scheme like parsing a booru's master taglist and swapping tag aliases like hydrus does siblings, in order to get namespaces from booru API information, I would have to parse the html page the image was on, which is what I do already. This means any booru interface I wrote would only save a bit of time on the initial gallery page parsing, and that's it.

Ultimately, I decided my time was best spent on other things. If the boorus get a bit more standardised, and if they include namespace information in their APIs, I'll revisit this topic.

bugs

If you have been getting full program crashes after navigating the media viewer with the manage tag dialog's new previous/next buttons, I am fairly confident I have fixed it.

Also, I introduced an odd bug a week or two ago when I collapsed some constants together that meant opening the import files dialog would bork a lot of future gui layout and sizing. I had planned to look at this problem next week, but then I encountered it while making this build, and seeing it was so easy to replicate, I made a quick fix. This means the import files dialog looks a bit ugly right now, but it won't break anything else. I will make it look neater next week.

Import folders will now ignore zero-length files (which are sometimes set up by backup/sync software), so if you have these, you shouldn't get any more error spam from them.

tag archives

Under services->review services->tag service->perform a service-wide operation, there is a new button to one-time add or delete an external tag archive's tags, by namespace. The same dialog already lets you export to tag archives, so I hope this makes one-shot mass-tagging a bit easier for everyone!

This is also useful if you want to quickly copy your local tags from one client to another.

You can only delete tag archive tags if the service is 'local tags' or it is a tag repository and you are a janitor (i.e. you have 'resolve petitions' permission). This is because doing a normal user's mass delete would need to create petitions for every tag->files mapping, and the current admin workflow is not set up to quickly deal with 300,000 petitions!

I have added a couple of useful functions to the hydrus tag archive class and also optimised the hydrus client database's code that interacts with them; doing a large operation will now take a few more seconds to calculate, but the actual sync should be a lot faster, and it will also include non-local files, which were not previously being checked.

full list

- added a possible solution to the manage tags dialog next/previous buttons crash
- completely overhauled temporary file management throughout client and server
- removed old temp folder
- copy files to clipboard will now copy the database's exact file paths (rather than copying to a temp folder beforehand), making it a lot quicker
- the import folders daemon will no longer attempt to import (and hence spam errors about) zero-length files
- added one-time tag archive sync button to the 'perform a service-wide operation' dialog in review services. this lets you add or delete the archive's tags by namespace
- revised the tag archive sync code a little so it syncs to non-local files as well, if the hashes can be lined up
- added hash cross-referencing to archive sync code before maina processing step, often saving a huge amount of time
- fixed a parsing error with advanced content updates at the gui-end
- added GetName and IterateHashes to hydrus tag archives
- added default 'similar to' max hamming distance to file system predicate defaults in client options
- fixed up the default text display of the file service system predicate
added some text to the sort/collect options panel to better explain what the big listbox is for
- fixed a popup numerical typo when uploading files to a repository
- fixed an utterly bizarre bug caused by the import files dialog that was screwing up subsequent dialogs' layout and size, apparently by wx voodoo
- refactored all the functions and classes of hydrusconstants to better locations
- misc other refactorisation
- misc cleaning
- renamed hydrusdownloading to clientdownloading
- changed all default { } and [ ] parameters to None. I don't think it mattered, but just in case for the future
- moved some ugly global variable stuff to better places, hoping to do more in future
- cleaned up many unused imports

next week

I really want to rework my media viewer so the background tags and text float above the current media as you move the mouse. I'd also love for interactable elements like the tags dialog to be more fluid as well, so a simple mouse gesture lets you start managing tags. This is likely too complicated for one week's work, but I think I can get the floating done.

Also, I have decided to move from YAML to JSON for my general storage and transmission protocol. I did some 'normal usage scenario' tests this week, and JSON was sometimes 200 times faster. JSON isn't as flexible as YAML, but it means tag update processing will spend all that much less time on the 'parsing update' step. I want to make a plan for moving over, and also for improving how the whole system works, making it easier to do db updates when something I am storing changes between versions.