/hydrus/ - Hydrus Network

>>2162

Drag-and-drop some files onto the main client window to bring up the import files dialog. This has two green-texted buttons, import now and add tags based on filename. Hit the 'add tags' one, which gives you the more complicated dialog that lets you add filename-regex and other complicated tags. The new checkbox is on that dialog, called 'try to load tags from neighbouring .txt files'.

It tries to parse the .txt files' tags live, btw, so they should be listed in the 'tags' column as soon as you hit the checkbox. Let me know if the control doesn't layout or work correctly for you.

>>2165

>>2169

we doin it now.

https://github.com/JetBoom/boorutagparser-server

>>2169

I will try to add .txt parsing to import folders in the next week or two. Regex is a more complicated thing to store, so that'll have to be put off until the next big rewrite of that code.

>>2128

>These redirect all gateway links to your local daemon when it's on

lmao, don't even run the gateway on my local machine

hacked this up though, tedious but whatever

>>2177

Thank you for this report–it looks like tumblr changed their API just slightly. I have fixed the parser for next week. Let me know if it doesn't work for you.

>>2179

Shortcuts are a bit of a mess right now. There is an ugly options panel at file->options->shortcuts. You can change fullscreen_switch, and previous/next, but not zoom stuff yet.

I would like to rewrite and clean up all the shortcuts in the program, making every action customisable and clean to edit, but it'll be a big job. I will have it on the list of features to vote for, then next time that comes around, so please vote for that if you care about it most of all.

>>2180

That's a good idea, thank you. I'll add a 'here are some add-ons that help you use multihashes in browsers' bit.

>>2182

Hmm, my (Win 10) Windows Defender doesn't. I have opencv_ccalib310.dll at 252,928 bytes. It was last modified 26th Jan 2016 at 3:45pm Central Time (I think that modified stamp propagates through the extract/installer). Do you have the same?

>>2195

I am thinking of listing the known urls in the media viewer, top-right, below where ratings and remote file services go. I can have them work like regular hyperlinks on the hover window. I hadn't thought of editing them–maybe I could do right-click->edit urls for that control or something?

>>2201

>I am thinking of listing the known urls in the media viewer, top-right, below where ratings and remote file services go. I can have them work like regular hyperlinks on the hover window.

Sounds like a good idea.

I also had an idea of putting it in the sidebar under the tags for the selected image, like this image I mocked up.

That might take up a bit of room for people who don't want it though, maybe it could be hidden with an option.

I'd be more than happy with either idea though.

>I hadn't thought of editing them–maybe I could do right-click->edit urls for that control or something?

That would work nicely.

The reason I'd like to manually edit that is because I already have my image collection downloaded, so it'd be a bit easier to import my collection as is, and then add the URLs.

>>2071

I do remember you reporting this previously, and I am sorry this is still causing you a problem.

I just had another look at the code, but nothing pops out at me as to why this is happening for you. The client will only delete from an export folder if it is set to 'synchronise', which I think I remember you had not set. Even then, it is only supposed to delete files that do not match the expected search results. This might occur if you had a search set that might sometimes give 0 results, although I don't know what that would be–maybe something with multiple system:age parameters. I presume you have a simple 'wallpapers' search for your export folder or similar.

I think we should see if it really is the export folder code that is clearing these folders out. Please go services->pause->export folders (so they won't regularly run) and then close the client. Copy some random files into your export folder and check back after the period you would normally expect the files to disappear in. If they do not disappear, then it is likely the client is doing it, so then open the client and wait that period again. If they do not disappear, then it is likely the export folder code is doing it.

If it is the export folder code doing it, and you don't have a 'synchronise' export folder, then I really don't understand what is going on. I seem to remember writing some debug code or something for your situation, but I can't remember the outcome. We could try that again, if you like.

>>2105

That's interesting. If the files are definitely disappearing when export is paused, then I am confident that the export folder code isn't doing it–it is only ever called in one place, and can only fire when it is unpaused. It even abandons its work if the pause is set while it is running.

So, unless something very odd is going on, it is either something else in hydrus or an external program doing it.

Was the file you randomly added–that wasn't deleted–a jpg or something else that hydrus can import, or was it a .txt or something? If you add random jpgs, do they disappear? Although it sounds silly, please double-check you don't have an import folder set to that same location, especially if you run more than one client–this repeated clearing out could certainly be caused by a 'delete' import folder. You can even try hitting 'pause import folder sync', just in case there is some hidden import folder or something.

And how about your external storage locations? Are they all as you expect in file->options->file storage locations?

Then, can the delete occur when the client is not running? If so, then it is almost certainly an external program doing this. It could be some other wallpaper harvesting program or something like that that is only taking files younger than x hours, or only the aforesaid jpgs.

Just as an aside, I'll add some log stuff to import and export folders for v195–maybe that'll tell us more.

>>2151

Thank you for the update. I think this was ultimately a user-interface problem. Both of those dialogs could do with some better info text, and I will add a test for and warning about overlapping import and export folders.

Although I have plans to make it easier, there aren't a lot of ways to mass import tags right now.

Also, when I add 'this file looks like this file–should they have the same tags?' to the program, your specific problem may be fixed in general.

Until then, I think you will have to use a script of some sort, and unless you want to get very complicated, you'll want to use python, so you can create a 'hydrus tag archive' using my code, which would let you import all the tags for the new files. I would do:

Create empty HTA

Export all files with tags in .txt files.

For each ( file, text_file ):

convert file to webm/mp4

figure out sha256 hash of new file

submit ( hash -> tags ) to HTA

Then manually import all those new files, and manually import the HTA to whatever tag service you want.

If you have experience with python or other general programming, I can walk you through it. It wouldn't be more than ten or twenty lines of code. But if you have no experience with programming at all, it might be more difficult.

In fact, now I think of it, I'll look into adding a checkbox to the import workflow that says 'import tags from a [filename].txt if one exists'. Since I added it for the export, I might as well allow the opposite. I might not have time for it this week, but it won't be too difficult.

>>2062

>2) Name all files with their respective tags

don't do this

Even though we implemented the txt file here

>1041

I'm still yet to get it working through batch/powershell

however, if the tags aren't enough to overflow the filename boundary, this scipt we created, initally just for exif tags, can easily be repurposed into an ffmpeg script

>>2067

Right, I've done this for tomorrow. You'll have to mass-rename your txt files from [hash].gif.txt to [hash].webm.txt or whatever, but otherwise, the 'add tags based on path' dialog now has a checkbox that'll try to parse tags from these neighbouring txt files. Let me know if it doesn't work for you.

If you don't have a mass renamer, I like Batch Renamer:

https://sourceforge.net/projects/batchrenamer/

>>2063

1. Lossless WebMs are huge. APNGs, while equally as lossless, are not. webm related

2. WebM is a video format, like MP4. Not an image format.

3. If anything were to replace APNG, it would be something like WebP or FLIF

Unfortunately, it seems FFMPEG doesn't give 'Duration' for apngs, and I don't see another reasonable way of getting that information (or num_frames, which is what I really want), so I won't add apngs yet. Again, if you discover a python library that can figure this stuff out, or if FFMPEG learns to parse them better, then I'll try again.

Sorry, I don't have much experience putting out source for other people to run, and I'm not really naturally professional anyway, so I never bothered to research it. I'd never even heard of a depends.py before, so that's why there isn't one!

If you haven't seen it yet, I put together a help page on my experiences getting the source to run on different platforms here:

http://hydrusnetwork.github.io/hydrus/help/running_from_source.html

Specifically, this seems to cover most of the bases:

(sudo) pip install beautifulsoup4 hsaudiotag lxml lz4 mp3play nose numpy pafy Pillow psutil pycrypto PyPDF2 PySocks python-potr PyYAML requests Send2Trash twisted

Although there can be more complicated stuff to do for things like wx–see the page.

Is depends.py something that'll auto-install this stuff? Is there some built-in library that'll check my code and automatically generate that file or otherwise figure out what the local python installation needs? I don't know what documentation I should be looking at, for this stuff, so I would be interested in what you can recommend. I prefer simple and multi-platform above everything else.

Although hydrus has always been content-agnostic, and my art collection is only about 0.25% of all my stuff, the general experience of fine art is probably the thing I care about most in the world. When I look at this stuff, or I listen to Wagner or Bach or Mozart or any of the other masters, all of my hope and enthusiasm returns.

Beyond /hydrus/, I have always been mostly a lurker, but when I do post, I usually attach art or quotes from stoic leaders like Aurelius. I imagine it only stands out here because I am not completely Anon.

Other than art-dump threads, I got a lot of my art from torrents, by the way. There are a bunch of gallery digitisation projects floating around with a handful of seeds. Most of them have the usual filename problems, so I am very slowly sorting what I have and batch-adding at least artist name and 'fine art' to the PTR. I have about 20,000 files still to import, so I won't run out any time soon!

>>2019

>There are a bunch of gallery digitisation projects floating around with a handful of seeds

I didn't know this was a thing. What's the terminology to look up? Are they on the standard public trackers?

>>2041

Yeah, the word to use is 'paintings', as most variations of 'art' will give you 'erotic art' and music-'artist' stuff. There's a load here:

https://kat.cr/usearch/paintings/

But quality varies wildly, and you can easily end up with two thousand more works of Dutch Still Life than you had originally planned, just because the seeds are sparse. Adding 'Monet' or 'European' also helps, depending on what you like.

I am sorry to hear this. This looks like a dll error, similar to this other recent report: >>1950 . Do you have a file called 'lxml.etree.pyd' in your install directory? It might have size 835KB.

The v192 I just put out has some new dlls because of the new development computer I built. Please try updating to that, as your problem might be fixed by accident.

Otherwise, try this debug build I put together:

http://www.mediafire.com/download/dsva1ykwegcr2g7/Hydrus_Network_192_debug_-_Windows_-_Extract_only.zip

Just extract it to your desktop or somewhere and run it from console. To do this, shift-right-click on its folder and select open command window here. Then type 'client' and hit enter. The debug build will spam a lot of additional information to the screen and may tell us more about the problem you have. Take a screenshot and post it here or email it to me.

Yes, I have that file with that exact file size. I was about to try updating then realized I was already version 192 and I just wasn't paying attention when I wrote this, oops. And I tried the debug version, it worked and ran Hydrus. Here's the screenshot anyway. Maybe I could just export all of my files and tags to a new Hydrus install?

Thanks for your help so far.

>>1982

Sorry for the late reply here. Unless I am misunderstanding you, I am pretty certain you were indeed running v191, as I had only put v192 out a few minutes before I made >>1977 . If the debug version works, then I should think v192 will as well. Here's the release post:

>>1976

Please give that a go, and let me know how it goes. If updating doesn't work, does a simple extract-to-desktop v192 work?

>>2024

exiftool actually supports folders as targets as well as recursing into sub-directories with the `-r` flag.

So something like this works fine.

@echo off
exiftool -r -Subject= -LastKeywordXMP= -XPKeywords= -Rating= -RatingPercent= -About= -ExifByteOrder= -Padding= %1
pause

You can save that as a .bat and drag any folder onto the batch file and it will scrub through all files within including sub-directories. exiftool also renames the original files to *.*_original so backing up isn't necessary but still advised.

>>2018

Like >>2024 said your script throws an invalid syntax error, all due to that first parenthesis being on the wrong line…

@echo off
FOR %%I IN (*.png *.jpg *.jpeg) DO (
echo %%I
exiftool -Subject= -LastKeywordXMP= -XPKeywords= -Rating= -RatingPercent= -About= -ExifByteOrder= -Padding= %%I
)
pause

Batch is such a "WONDERFUL" thing. This might be an OS difference Windows 10 or 8 may have made Batchs' syntax more bearable.

Feel free to use either solution!

>>2028

Oh, boy.

#!/usr/bin/env python2
import os
import subprocess
import sys
import json
import random
import sets
import logging
import traceback

TAGS = ['XPKeywords', 'Keywords', 'Subject', 'LastKeywordIPTC', 'LastKeywordXMP']

def main():
	print("Script start in \"%s\"." % os.getcwd())
	os.chdir(os.path.dirname(os.path.realpath(__file__)))
	print("Change working directory to \"%s\"." % os.getcwd())

	logging.basicConfig(filename="rename.log", level=logging.DEBUG)
	logger = logging.getLogger('rename')
	ch = logging.StreamHandler()
	ch.setFormatter(logging.Formatter('%(asctime)s:%(levelname)s:%(message)s'))
	logger.addHandler(ch)

	logger.info("Starting...")

	if len(sys.argv) < 2:
		raise Exception("Need a valid argument.")

	raw = subprocess.check_output(['exiftool', '-j', '-r', sys.argv[1]])
	if not raw or len(raw) <= 0:
		raise Exception("exiftool didn't return anything?")

	data = json.loads(raw)
	for img in data:
		source = img['SourceFile']

		name = "%s;%%s.%s" % (os.path.splitext(os.path.basename(source))[0], img['FileTypeExtension'])

		file_tags = sets.Set()
		for tag in TAGS:
			if tag in img:
				if isinstance(img[tag], basestring):
					file_tags.update(img[tag].split(';'))
				else:
					file_tags.update(img[tag])

		name = name % ';'.join(file_tags)
		logger.info("Renaming \"%s\" to \"%s\"" % (img['FileName'], name))

		os.rename(source, img['Directory'] + os.sep + name)

	logger.info("Ended.")

if __name__ == "__main__":
	try:
		main()

Try dragging and dropping the json file onto the window. I think I set that dialog up to catch drop events.

This is code I threw together, so it might not work totally right. I think it overwrites an existing sub with the same name, for instance. Let me know if it gives you any trouble.

Have fun, and stay safe.