[ home / board list / faq / random / create / bans / search / manage / irc ] [ ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog

Name
Email
Subject
Comment *
File
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options
Password (For file and post deletion.)

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 8 MB.
Max image dimensions are 10000 x 10000.
You may upload 5 per post.


New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Currently prioritising: simple IPFS plugin


File: 1457561016502.jpg (32.87 KB, 433x380, 433:380, 1395023535367.jpg)

 No.2231

I feel like we need a single thread to contain all the single-line answer questions that are asked around here. Would provide a nice archive of "noob questions", and would prevent the board from being clogged up by simple, single-purpose Q&A threads.

 No.2232

Let's start with my question: is the regex character "\K" supported by hydrus?

If not, does anyone know how to match files that have trailing zeroes without including the zeroes? So that

F:\Example Doujin\001.jpg
would get matched in to
page:1

If \K works, I would just use

\\0*\K\d{1,3}
, but that doesn't seem to be working…


 No.2233

>>2232

Just to expand on this, I know that

[1-9]+\d*(?=.{4}$)
is given as a shortcut, but when your pages are formatted like
001_001.jpg
where the first group is the chapter and the second is the page, that particular regex doesn't work.


 No.2234

Is there a way to set the default zoom for images?


 No.2235

What's the difference between an Inboxed and Archived file?


 No.2237

File: 1457569219619.jpg (176.54 KB, 375x850, 15:34, 692f9faf33e49b50417255ff08….jpg)

>>2232

>>2233

hydrus is written in python and uses the stock re library which doesn't support a few things, including \K, that some other regex engines do. For a overview of python regex see here: https://docs.python.org/2/library/re.html

hydrus_dev it might worth looking at the pypi regex library that aims to replace re it supports some nice features like \K and capturing groups in lookaheads/lookbehinds which would be nice if you ever add support for capturing groups in resulting tags.

https://pypi.python.org/pypi/regex

I'd provide you with a working regex, but i'm not very good at crafting them :/.

>>2234

Closest option to this is file->options->media->Zoom smaller media to fit media canvas. Other than that not that I know of.

>>2235

This boils down to hydrus's inbox concept, essentially the inbox is a sort of staging area for files. It's used for files that have just been imported and have no tags, that way you can sort what you have and have not tagged. The little mail icon is just to indicate whether a file is in the inbox or not. A file being archived meaning it's NOT in the inbox.

This is not the only way to use this flag, but I believe it's the intended use.

Hopefully this helps a little bit.


 No.2238

>>2232

>>2233

This regex will only select a string of numbers that don't start with "0" if they are the last characters before a "." and a series of non "."s followed by the end of the string. In other words, this will select all the meaningful digits in an integer before the file type.

[1-9]\d*(?=\.[^\.]*?$)

http://regexr.com/3cvmf

If you're looking to get to know regex a little better, I'd recommend playing around on this site; having the matched text highlight as you type it makes understanding the different parts of an expression easy.


 No.2258

So, I ended up adding my whole Image folder to Hydrus.

The issue is that Hydrus then deleted my original files, moving them all to its own datastore. This would be fine, but I have one issue.

How do I go about uploading images through my browser?

It's unfeasible to go through my datastore and find the hash for my individual file.

Is there an easier way, is what I want to know.


 No.2260

File: 1457883817726.png (172.22 KB, 530x579, 530:579, 63b2441362de3f27b3d18b2d2c….png)

>>2258

Drag and drop the image thumbnail in hydrus onto a file select in your browser (or anywhere actually).

You're also able to export images from hydrus through the share option in the context menu.


 No.2262

File: 1457894878526.jpg (309.07 KB, 1280x1009, 1280:1009, 812ab70d5785ba9416fd3893fc….jpg)

>>2234

As >>2237 says, the fit/don't fit checkbox is the only zoom option right now, but I would like to add per-mime zoom settings, and more options (so you can say things like 'fit a png unless its fit size is less than 110%, in which case just show it as 100%').

>>2237

That regex library looks good, thank you. I would like to overhaul the path tagging dialog, not only to add groups and other advanced regex stuff but also adding better gui or maybe even a wizard to make it easier for people unfamiliar with regexes to use it. A big limitation though is fitting those advanced controls into an already cluttered dialog. I may just have to have multiple pages.

When I focus on that, I will probably update the regex library as well.

>>2238

That regex site is great, and your example is as well. I will add it into the program and the help.

>>2258

When you import files, make sure the 'delete files after successful import' checkbox on the first import path selection dialog is set how you want. If you would like to keep your original files as well, that is a completely fine way of running the program.

Otherwise, as >>2260 says, you can just drag and drop thumbnails out of the program to do a quick export. This works very well for upload boxes like here on 8chan. Due to technical limitations, they retain their ugly hash filenames. If you would like to export en-masse to the filesystem with nicer filenames (e.g. 'creator - title.jpg') you can do that through the more complicated file export dialog. Let me know if either do not work for you!


 No.2280

Is there a way to hide some files from "system:everything", but make then visible when searched? Or another way to see all images without seeing then.

Blacklist only hide the tag, not the file.


 No.2282

If I synced with a downloaded db, gelbooru for example, after the sync do I still need to keep that db file or all the mappings now inside my client db?

Also, do most of you leave your client open and idling a majority of the time, or only open it whenever you're posting an image?


 No.2283

Question: Is there an established place where we can discuss the "rules" for tagging in the public tag repo?

I'm thinking about things like how the medium: namespace should be used. Should medium:animated only be for animated stuff, while medium:video be used for real footage, etc.

I feel it would be nice to have a wiki for this.


 No.2286

File: 1458418267175.jpg (729.92 KB, 2617x2047, 2617:2047, 88257c3723975090e57c55b2bf….jpg)

>>2280

Not really. You could try giving all your files a local 'nsfw'-like tag and then including '-nsfw' in your searches, but that's a bit awkward and prone to mistakes. For now, if you want to divide your files into multiple file domains, then I recommend you run multiple clients, one for each purpose. I may extend the client to support multiple 'local files' services in the future, but I am still thinking about the best way to do it.

>>2282

Most of the time, you probably want to stay synced with a tag archive. Most use md5 as their file hash, while hydrus uses sha256, so only mappings for local files (where both sha256 and md5 is known) can be cross-referenced and imported to the db. A few tag archives use sha256, so all data is imported in one go, but this is rare.

Either way, staying synced to a tag archive only adds very small lag, usually a handful of milliseconds per file import, so you don't have to worry about performance loss.

I usually leave my main IRL client on my laptop open all the time with about eight tabs for different ongoing purposes. I have it set to run maintenance jobs on idle, so it generally stays synced to the PTR all the time. It stays open through computer sleep/hibernation without problems.

I also have a file-empty admin client that I use to process PTR petitions, that I only have open for a few minutes a day, just to get that job done. I have that set to sync on shutdown, which usually takes ten or fifteen minutes.

Both scenarios work ok for me, but it took some tweaking on file->options->maintenance and processing to get what I wanted.

>>2283

Here is probably best for now, but perhaps there could be a more permanent place. I assume you have read my rough schema at:

http://hydrusnetwork.github.io/hydrus/help/tagging_schema.html

In general, I am easy-going about this stuff. If you start tagging files with good and appropriate 'medium:' namespaces, I figure other people will start using them as well. If your tags are less helpful, they might be revoked or altered with siblings. Remember that most files in hydrus have metadata that allows for 'system:mime is video' and 'system:duration>0'-type searches, so adding things like 'animation' may not be as important as you expect.

Anyway, I don't have a specific place to discuss this, but feel free to create a wiki.


 No.2288

Would having Hydrus on the same SSD (but not partition) as my OS be the cause of how slow it loads search results?

Each 256 group takes like 3 minutes each.


 No.2290

>>2286

>>2282

I was just curious if the db file i imported/synced from is still worth keeping or not; if all the tagging info from it is now in my client db then it's effectively just a copy that's taking up space.


 No.2293

File: 1458495948079.jpg (275.7 KB, 1608x1673, 1608:1673, 1d3bf15479adb71efe214e84a6….jpg)

>>2288

That seems exceptionally slow, especially from an SSD. Having it on your system drive's disk might slow it slightly, but I would expect that to be slight and occasional.

I have written a general help page on this (although a lot won't apply to you, because most slowdown is usually due to fragged HDDs) here:

http://hydrusnetwork.github.io/hydrus/help/reducing_lag.html

At the bottom of that page are instructions on how to produce a profile. If you like, please generate one or more profiles of your slow file results loading (and anything else that might be slow) and send me what you get.

Also, some general questions:

Are you synced with my public tag repository?

How large is your install_dir/db/client.db file?

What OS and CPU do you have?

If you are running windows, and you open task manager (Ctrl+Shift+Escape), what sort of idle CPU and HDD usage is your computer using?

If you create a search that always produces the same small set of files, and you press F5 to refresh it, does it load faster on the second or third try?

>>2290

I think you should stay synced, because most of the regular boorus are md5 tag archives, so you likely won't have imported all the info.

Since you have mentioned this, I will make a job to include a little check when you sync with an archive that will tell you if the hash type makes an ongoing sync redundant, to clear up future confusion about this.


 No.2294

I downloaded around 800 images from pixiv without realizing I had to check the tag buttons for hydrus to download the tags. Is there anyway to get the tags for the 800 images? I checked the box 'get tags even if file is already in db' but it still won't download.


 No.2296

>>2293

I'll look in to getting some profiles, but in the meantime:

- Yes, I am synced. This seems to be a main factor in my searches being slow.

- client.db is 4.7GB

- Windows 10, i7 2700k

- idle CPU is ~10%, idle Disk is 0%

Also some stats:

- search for a tag with 3114 files: 11m50s

- search for a tag with 76 files: 56s

- f5 on the same tag: 50s


 No.2299

>>2294

You unfortunately need to redownload all 800 images. You can do this by going into the subscription, and clearing the cache. If it still won't redownload, delete the subscription, and create a new with the exact same information.


 No.2302

>>2296

Hello, it seems like you might be experiencing the same problem I've had on two different machines/os's now.

I made a thread about it (ignore the stuff about DPI scaling, that's a different issue): https://8ch.net/hydrus/res/2277.html

Sounds about the same, right? I've also linked to two log extracts if you want to compare.


 No.2307

File: 1458670492360.jpg (215.24 KB, 1539x1925, 1539:1925, dff013ab7a15af4afa65ab423e….jpg)

>>2296

As I say at >>2306 , I may have fixed this problem for tomorrow's release. Please let me know if you still have lag after you update.


 No.2311

>>2307

Will do. I've generated pre-update profile, will do the same search post-update and post both.

By the way, do profiles have any identifying information in them? Should I email you directly, or can I just upload and post a link on the board?


 No.2313

>>2311

I don't think they usually do since the default install directory is c://hydrus network it avoids your name and such.

If you're paranoid about it then either read through it yourself or just email it to hydrus dev


 No.2314

>>2307

I've actually just gotten a huge speed increase without touching anything.

Searches now take a matter of seconds for thousands of files instead of a matter of minutes.

Did you do anything to the PTR that would cause this change?


 No.2317

Can a boorus tag system be downloaded through hydrus and then systematically go through each image adding the necessary tags or Alternative option?

Sorry my english not very good.


 No.2319

File: 1458844958909.png (8.55 KB, 414x248, 207:124, 4c94cb02dc37b71207f0981f0f….png)

How exactly do I instantly synchronize tag databases, apart from closing hydrus (which will interrupt the synchro after 30 minutes)?


 No.2320

>>2319

Options > Maintenance and Processing

will let you tell Hydrus to either perform 3 hours of maintenance on shutdown

OR

let you set "Assume the system is idle if…" to like 0-1 minutes. Hydrus will only do maintenance (which includes tag repo synchronization) if it considers your system to be idle.


 No.2321

File: 1458845670312.png (69.12 KB, 800x466, 400:233, 3ef0182a0a0c1ac98551be8db6….png)

>>2320

Thanks!


 No.2327

Gonna ask another question, is there a problem if I'm downloading the public tag repository tags, and it's stuck on "content row 0/100,164: writing"?

It hasn't moved in a while…


 No.2329

>>2327

Specifically, it seems hydrus locks up when doing this (according to the db logs):

2016/03/27 14:08:34: Profiling write content_update_package


 No.2331

File: 1459205090473.jpg (7.83 KB, 225x166, 225:166, 149512a6d5b46c1040f4453b83….jpg)

Is there a way to delete tags that aren't in use by any image, or was that one of the "coming soon" features? I recently fixed a spelling error for a character name in all my files, but I'm worried I'll mess up again since the misspelling still comes up in the tag suggestions.




[Return][Go to top][Catalog][Post a Reply]
Delete Post [ ]
[]
[ home / board list / faq / random / create / bans / search / manage / irc ] [ ]