[ home / board list / faq / random / create / bans / search / manage / irc ] [ ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog

Name
Email
Subject
Comment *
File
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options
Password (For file and post deletion.)

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 8 MB.
Max image dimensions are 10000 x 10000.
You may upload 5 per post.


New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Currently prioritising: simple IPFS plugin


File: 1454562400616.png (26.54 KB, 186x208, 93:104, 1454042071830.png)

 No.1983

Hydrus newbie here, I've scanned the help section and haven't found an answer so I'll go ahead and ask here. My apologies if there's a general help thread; I'm somehow not seeing it in catalog view.

I've got a number of images that I've tagged via windows (.jpgs), or that I've written tags or other important details into the filenames (.pngs, .gifs, .webms, some .jpgs). My question is, is there a way to bring those tags into hydrus as I import? The ones that are in the filenames are easy enough, though time consuming, via manually typing them but the .jpgs with tags in their data I'm unable to see when importing.

Secondly, is there a way I could somehow force the filename to become a tag as I import? Not just for the files I already have named or tagged via filename correctly, but I like that I can look back over files I've picked up from the chans and know when I got them. Or does hydrus have some sort of "date created" tag that could read those in, instead of just the date when I imported them?

 No.1984

In the import files section, there is an option to add tags based on filenames, which you can use regex in order to define what parts of the filenames become tags in the db. There's a link in there to a regex tutorial, but alas, I have not personally used this function. You may have more luck, however.

As for your other question, I'm afraid I don't know. The help documents may have more information.


 No.1985

>>1984

Thanks! This worked out great; technically preserves filenames, and since most of them are just numbers or hashes, I can find the ones with actual tags easily and manualy tag them properly.

Awesome, so now the only issue would be pulling tags on .jpgs that I've done via windows.


 No.1988

Couple of other questions…

1. Sometimes when importing a folder, I'm told a certain number of files had "unsupported mimes." Judging from the file count, they don't import; is there a way to determine which files those are if I opt to manually delete the originals via windows? None of the filetypes jump out as being unsupported so the only safe option would be to use hydrus to delete successfully imported files…

2. Is there a way to see the most commonly used tags without starting a search? When I first open, I see system: everything, number of tags, etc. when I would rather see the most tags that are used. Especially since I've used the gelbooru db to mass tag files, I don't know what a number of the tags are so I would be blindly guessing without some sort of master tag list for local tags.


 No.1989

>>1988

1. I believe the supported mime types are:

audio/mp3

audio/ogg

audio/flac

audio/x-ms-wma

application/x-shockwave-flash

application/pdf

image/jpg

image/png

image/gif

video/x-flv

video/mp4

video/x-matroska

video/x-ms-wmv

If it's not one of those, you're probably going to run into issues.

2. You will always need to start a search, but you can click system:everything to load a default search, then select incidence (desc) in the selection tags dialog to view tags by number. To make this easier, you can also set incidence as the default tag view option by going to Options -> Tags -> Default tag sort.


 No.1990

>>1989

Oh, and video/webm. Forgot that one. That's also supported.


 No.1992

>>1989

Think I figured it out; the "unsupported mimes" were subfolders. Didn't realize folders counted towards file total.

Set incidence (desc) as well. thanks for the help senpai


 No.1996

File: 1454717534121-0.png (339.84 KB, 580x720, 29:36, 1399185495486.png)

File: 1454717534363-1.jpg (34.56 KB, 362x346, 181:173, 1416070789240.jpg)

File: 1454717534363-2.png (362.43 KB, 700x700, 1:1, 1437650550536.png)

Can you share those images with me?


 No.1997

>>1996

I don't have the images unfortunately, but I'm starting to collect. I think a thread on /a/ is what resulted in the op image, which I've since seen used on /c/ /r9k/ and /pol/

Best bet would be to search an archive for "interrogation marks" because everyone who sees the image always parrots the phrase

>interrogation marks

since you'd usually say question marks

Anywho…still >interrogating about a method to import tags done via windows, pretty much the only thing keeping me from switching to hydrus now


 No.2003

File: 1454806078452-0.jpg (37.54 KB, 362x346, 181:173, 1454594341668.jpg)

File: 1454806078452-1.gif (149.25 KB, 600x338, 300:169, 1453270438792.gif)


 No.2009

File: 1454997641950-0.png (93.16 KB, 295x221, 295:221, dae63c0a3b111301feb774a8e9….png)

File: 1454997641951-1.png (854.96 KB, 726x860, 363:430, add9f7b629aafb3f25ce5de98d….png)

File: 1454997641951-2.png (602.03 KB, 963x720, 107:80, 274229acf594e5b562b593dce3….png)

there's somewhat of a problem when using jpg/exif tags on images that are imported

they have a different hash to an exact same image, which means that they don't sync up well on other repositories

while this has no affect if using it by yourself, but it's something to note

I usually use them when exporting files from a database, so that the files in the client remain tagless


 No.2010

>>2009

Sure, but I have ones that I've tagged before I found hydrus, and I was hoping there was a way to import those tags onto those files as I import them.

Otherwise I have a LOT of work to redo, and I wont even be able to see them without using explorer's details view and trying to match them to image


 No.2012

>>2010

I'm a bit confused. Are you saying your tags are in the image's metadata? If so, I think I could make a script to strip the tags from the file(thus restoring the original hash), then put those tags in the filename, then you could use regex with hydrus to mass import them.

Can you post a few of your tagged images, so I know what pattern you were using?


 No.2015

>>2012

exiftool.exe is what's been tossed around here a few times

otherwise tags in filenames is nothing new and isn't hard

unless you hit that character limit

that's when we moved tags to text file


 No.2017

File: 1455078080604-0.jpg (642.74 KB, 595x842, 595:842, 0257ca477fea9e799f9c181a5d….jpg)

File: 1455078080604-1.jpg (6.31 MB, 6013x4081, 859:583, 471745.jpg)

File: 1455078080605-2.jpg (145.49 KB, 718x840, 359:420, 1442509355537.jpg)

File: 1455078080605-3.jpeg (441.14 KB, 2531x1600, 2531:1600, b4957c1f07ed832e28605182f….jpeg)

>>2012

Yeah, pretty much. Or I'm assuming that's how windows explorer stores them

>few examples i guess, though it should be straight forward.

>>2015

People had enough tags to hit character limit?


 No.2018

File: 1455081312589.png (164.67 KB, 1394x835, 1394:835, kon.png)

>>2015

Yea I was going to use exiftool.


@echo off
for %%I in (*.png *.jpg *.jpeg) do
(
echo %%I
exiftool -Subject= -LastKeywordXMP= -XPKeywords= -Rating= -RatingPercent= -About= -ExifByteOrder= -Padding= %%I
)
pause

would remove tags and ratings from all image files in a directory.

>otherwise tags in filenames is nothing new and isn't hard

of course not, but I may not need to do that at all depending on their answer

>>2017

I used exiftool to remove the tags from those images, and it does indeed match back to the original hash from gelbooru.

Since these came from gelbooru and such, why not just sync with one of the tag archives? They'll get tagged automatically, once you remove the file tag data.


 No.2020

>>2018

The anime pics will tag sync, sure, but I've got various other chan pics and such that wouldn't. They're the same tagging style though so I thought anime pics would be a suitable example.

So how exactly do I combine this script with exiftool?


 No.2021

File: 1455090871440.png (176.15 KB, 1366x821, 1366:821, 16-02-10_18-52-32-Environm….png)

>>2020

put exiftool in your path

the path is basically a bunch of directories that windows can use without referring to their folder

it's like when you run 'cmd.exe' in Run and it just works


 No.2022

>>2021

Ok I got that set; how do I use the script? Batch file? Am I supposed to swap some line for the directory/file path?

My apologies but I'm quite retarded when it comes to this stuff.


 No.2023

>>2022

from teh looks of it

make a backup of your images, and you have two options

put the batch in the path, open command prompt to the folder where your images are and execute the batch by typing it into the console

or

save it as a batch in the folder with your images and run it

really do a backup

don't lose your tags incase the script is faulty


 No.2024

>>2023

>>2018

I must've botched something here; script seems to execute but does nothing. It looks like it says "the syntax of the command is incorrect" but it closes so fast I can't be certain.


 No.2025

File: 1455192044415.jpg (72.1 KB, 601x861, 601:861, b8a59bf7e2e9e01351278746fb….jpg)

>>2024

exiftool actually supports folders as targets as well as recursing into sub-directories with the `-r` flag.

So something like this works fine.


@echo off
exiftool -r -Subject= -LastKeywordXMP= -XPKeywords= -Rating= -RatingPercent= -About= -ExifByteOrder= -Padding= %1
pause

You can save that as a .bat and drag any folder onto the batch file and it will scrub through all files within including sub-directories. exiftool also renames the original files to *.*_original so backing up isn't necessary but still advised.

>>2018

Like >>2024 said your script throws an invalid syntax error, all due to that first parenthesis being on the wrong line…


@echo off
FOR %%I IN (*.png *.jpg *.jpeg) DO (
echo %%I
exiftool -Subject= -LastKeywordXMP= -XPKeywords= -Rating= -RatingPercent= -About= -ExifByteOrder= -Padding= %%I
)
pause

Batch is such a "WONDERFUL" thing. This might be an OS difference Windows 10 or 8 may have made Batchs' syntax more bearable.

Feel free to use either solution!


 No.2028

>>2025

Awesome, thanks!

Question though, was there a way to remove the tags from the metadata and append them to the file name? This does work for scrubbing, but ideally I was hoping to move the tags to filename so I see them when mass importing into Hydrus.


 No.2034

File: 1455404440474.jpg (129.7 KB, 827x1169, 827:1169, a59d6a46de75be5add56888088….jpg)

>>2028

Oh, boy.


#!/usr/bin/env python2
import os
import subprocess
import sys
import json
import random
import sets
import logging
import traceback

TAGS = ['XPKeywords', 'Keywords', 'Subject', 'LastKeywordIPTC', 'LastKeywordXMP']

def main():
print("Script start in \"%s\"." % os.getcwd())
os.chdir(os.path.dirname(os.path.realpath(__file__)))
print("Change working directory to \"%s\"." % os.getcwd())

logging.basicConfig(filename="rename.log", level=logging.DEBUG)
logger = logging.getLogger('rename')
ch = logging.StreamHandler()
ch.setFormatter(logging.Formatter('%(asctime)s:%(levelname)s:%(message)s'))
logger.addHandler(ch)

logger.info("Starting...")

if len(sys.argv) < 2:
raise Exception("Need a valid argument.")

raw = subprocess.check_output(['exiftool', '-j', '-r', sys.argv[1]])
if not raw or len(raw) <= 0:
raise Exception("exiftool didn't return anything?")

data = json.loads(raw)
for img in data:
source = img['SourceFile']

name = "%s;%%s.%s" % (os.path.splitext(os.path.basename(source))[0], img['FileTypeExtension'])

file_tags = sets.Set()
for tag in TAGS:
if tag in img:
if isinstance(img[tag], basestring):
file_tags.update(img[tag].split(';'))
else:
file_tags.update(img[tag])

name = name % ';'.join(file_tags)
logger.info("Renaming \"%s\" to \"%s\"" % (img['FileName'], name))

os.rename(source, img['Directory'] + os.sep + name)

logger.info("Ended.")

if __name__ == "__main__":
try:
main()
except:
print(traceback.format_exc())
raw_input()

I hope you have python2 installed, and I would be careful with this as I wrote it fairly fast with not a lot of sanity checks. If you do have python2 installed it should work in a similar way as the batch scripts, as long as it has exiftool in the same directory or in windows PATH with it you can drag an image or folder onto it and it *should* rename every file to `<filename>;<tag;>.<ext>`. Save it as `whatever.py` or whatever.

This doesn't scrub any tags so you'll just have to run the scrubbing scripts afterwards.

I would post this as a "compiled" exe but that'd be shady as fuck and kinda dumb as it relies on exiftool.

I didn't realize how handy exiftool is until after writing this it provides a `-j` flag for json output which is way nice and even reads file formats other than jpeg/jfif.

Good luck!~


 No.2036

>>2034

Thanks friend! Works fantastic.




[Return][Go to top][Catalog][Post a Reply]
Delete Post [ ]
[]
[ home / board list / faq / random / create / bans / search / manage / irc ] [ ]