[ / / / / / / / / / / / / / ] [ dir / 1piece / dcaco / firechan / imas / lovelive / marx / mde / sunny ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog   Archive

Welcome to 8chan, the Darkest Reaches of the Internet.
Name
Email
Subject
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 16 MB.
Max image dimensions are 15000 x 15000.
You may upload 5 per post.


New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Current to-do list has: 1,588 items

Current big job: last parts of downloader overhaul, then login manager


YouTube embed. Click thumbnail to play.

aea724 No.9608[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v317.tar.gz

I had a great week. I polished some ongoing download stuff and caught up on a bunch of smaller jobs.

tag filter and tag import options

A while ago, I wrote a new 'tag filter' that applies blacklist and whitelist rules to a list of tags. It is more powerful than some of the old systems I was using at the time, but the ui was pretty ugly, so I didn't roll it out everywhere it could be. This week, itPost too long. Click here to view the full text.

40 posts and 9 image replies omitted. Click reply to view.

3d3ac1 No.9676

>>9651

>1000 of the largest thumbnails in t00 only come to 53mb

Those are compressed jpgs, most likely thumbnails are stored uncompressed in memory.




File: 3f8a8d22364b1de⋯.jpg (111.13 KB, 400x400, 1:1, 1394958218797.jpg)

d5679d No.9327[Reply]

Here you can ask questions so that the board is not clogged with small threads.

>>6021 has reached its bump limit, so I made a new thread.

72 posts and 6 image replies omitted. Click reply to view.

3de2ff No.9669

Is there any way to open the File Import Status window on a multiple watcher tab? It kind of sucks to see that a bunch of files were ignored without being able to see which files were ignored or check why they were ignored or even to do literally anything about it at all without just giving up on multiwatchers and just re-adding the thread on a normal watcher tab.


73416d No.9672

>Otherwise, you can get the md5 and do a md5: on gelbooru

I have lots of images from a few boorus that aren't auto tagged by the PTR. Is there a built in way or plugin that does an md5 search of all my files for me? Doing it manually would take forever.


73416d No.9673

>>9672

Meant for >>9662


6884fa No.9674

I use these addons to one-click download from pixiv:

https://addons.mozilla.org/en-US/firefox/addon/ank-pixiv-tool/

https://addons.mozilla.org/en-US/firefox/addon/px-downloader/

and they (after configuring) produce filenames of:

AnkPixiv/?site-name?/(?member-id?) ?memoized-name?/(?illust-id?)?illust-year?-?illust-month?-?illust-day?[?illust-hour??illust-minute?] ?title?

ie:

PxDownloader\椅子(20663)\(18182022) 2011-04-15[2044] ももこ

Tbh the above formatting is a clusterfuck, just ignore it (the first addon stops working a lot, so I use the second one, which randomly fails to fetch images and never tells you so it sucks, but the first is still not working so I had to use+mention it). Can Hydrus match ripping this information? I made the pixiv illustration id the first thing in the filename, so basically if Hydrus can lift that and tag them as such I wouldn't really need the information following, though I guess it'd be nice.


6884fa No.9675

>>9667

>>9619

For what it's worth I once found a random site that shows me NSFW-flagged tumblrs without logging in, and just now I searched it in my history to simply reload the same page that worked for me just a few months ago, but now it doesn't (it returns "Hrmm, couldn't find tumblr user 'Hecchidechu'. Showing twitter search."). You don't want to visit that tumblr btw since it's a western artist who loves armpit hair.

Tumblr definitely broke something, probably on purpose. I wouldn't be surprised if tumblr-hosted images no longer showed up in reverse image searches. Considering they blocked NSFW tumblrs from being searched for or viewed without an account, they're fuckers and I don't have faith they'll lift this, so hoping it can be circumvented.




File: ab23431121b8563⋯.jpg (96.69 KB, 492x650, 246:325, ab23431121b8563a7bc3b7ac57….jpg)

0675b5 No.5115[Reply]

Gonna start another thread because >>173 is becoming too big for most people.

In the tag manager, whenever multiple tags are selected for multiple files with different tags, if you press the del key, the program will still ask you if you want to delete them all or put them all in the selected files. This does not make sense, if I'm pressing the del key it's obvious that I want them gone.

454 posts and 136 image replies omitted. Click reply to view.

35b3df No.9632

>>9578

I'm afraid the thread watcher cannot handle these legacy links. Please try the '8ch thread (all linked files)' entry under the 'simple downloader', which will parse the html for file links directly, for these rare very old threads.


35b3df No.9633

>>9584

Thanks. Yeah, it looks like more JSON. Maybe they will switch all manga over to it at some point. I will fold a fix into the pixiv parser when I find some time.


35b3df No.9634

>>9595

I am afraid you may be out of luck. Although I have plans to ruggedise them and add dated backups, sessions are fragile atm. If the session save got borked due to the dismount or the recovery after the crash otherwise failed, I think that data is likely gone. If it is not listed in the 'pages' menu as a session, there will not be a backup. If you have a backup of your client, I can help you export and import the session from that backup to your active db.

>>9618

He just means to get the files that were in those tabs back.


35b3df No.9635

>>9604

Thank you, I will check this out.


2d5e38 No.9671

On the topic of parser bugs: Sankaku Chan gets at most 500 urls then stops parsing for more urls.

So it can get the initial 500 urls and any new content later, but anything older than the original 500 urls is impossible to scrape at the moment.




File: ea31ab670c8dee7⋯.png (12.55 KB, 340x175, 68:35, mariadb-usa-inc.png)

928501 No.9068[Reply]

ITT: create proposals for making Hydrus more optimized.

Proposal: Why can't Hydrus switch to MariaDB?

If it is faster, then it should be better. The only trouble is having the need to rewrite the queries, which from an SQL standpoint should be a non-issue, right?

List of Databases with Open Source License and Open Source APIs:

SQLite - Currently used in Hydrus, has minimal features

MySQL - A more well-rounded SQL Database with user management

PostgreSQL - An SQL with complex features with less performance

MariaDB - SQL/NoSQL database with heavy optimizations

ElasticSearch - A literal search engine instead of a normal Database

Teradata - IDK

https://www.digitalocean.com/community/tutorials/sqlite-vs-mysql-vs-postgresql-a-comparison-of-relational-database-management-systems

https://www.infoworld.com/article/2611812/mysql/mysql-face-off--mysql-or-mariadb-.html

21 posts and 1 image reply omitted. Click reply to view.

c022ca No.9658

As mentioned by >>9094 the bottleneck is mostly how the I/O and CPU is handled by hydrus. Imports are done sequentially when they can be sped up a lot by using multiprocessing. I'm sure other actions are still done sequentially too. A transition to a graph database like ArangoDB could be better in the long run, but that's never going to happen.

Looking at the client.master.db database, I'm not sure why he added an index to the md5, sha1 and sha512 columns but not to the subtag or namespace columns. Doesn't make sense to me (and is the sha512 index really necessary?). Also it boggles my mind that foreign keys aren't being used at all.


35ccf5 No.9659

>>9658

I am also expecting multi-threading could be a place where we can optimise the code (since most computers now run on 4/8 cores).

Perhaps SQLite, MD5/SHA hashing and de-duplication are not made for multi-core and/or GPU computers.


c022ca No.9660

>>9659

>multi-threading

Python threads are all executed on the same core. That's why I said multiprocessing. It spreads out each subprocess across each core. Based on your post you don't know much about software, so think of a subprocess in python like a normal thread.

>are not made for multi-core and/or GPU computers

Everything you've mentioned can be easily sped up with multiple cores. Using a GPU would be even faster but there's no point in using that here. I'm actually pretty surprised he hasn't implemented multiprocessing functions in bottleneck situations like importing. It's very easy to split up the work once you've scanned all the files. You just divide them up by the number of cores and have each subprocess do that portion of the work. If you have 4 cores you have each core do 1/4 of the files you want to import.


35ccf5 No.9661

>>9660

>Python threads are all executed on the same core. That's why I said multiprocessing

Well due to people call 4 core Intel CPUs having "hyperthreads" making it 8 virtual cores, I would say that is easy to have those things mixed up. If I have to use a proper term Parallel Programming (as in Concurrency) would be more fitting.

>Everything you've mentioned can be easily sped up with multiple cores

I meant that it has not been implemented yet by the dev since (s/are not/has not been/)

>I'm actually pretty surprised he hasn't implemented multiprocessing functions


e699ad No.9670




YouTube embed. Click thumbnail to play.

9ba0bc No.9533[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v316.tar.gz

I had an excellent week. I was able to cleanly 'do' the big gallery overhaul, resulting in many changes. This is an important release for anyone who does a lot of downloading.

gallery update

So, the meat of the gallery overhaul is complete! Where available, subscriptions, the gallery downloader, and the url downloader will now use the new parsing system to parse 'thumbnail pages'. The new pipelinePost too long. Click here to view the full text.

34 posts and 6 image replies omitted. Click reply to view.

000000 No.9589

>>9586

gg To go top would be nice.


9ba0bc No.9636

File: d3c8f84bb7c568a⋯.png (3.04 KB, 512x85, 512:85, gelbooru 0.2.5 file page p….png)

>>9575

Thanks. If I do the exact same URL, I get the tags ok. If you are on linux (and maybe running from source?), it might be that you have odd parsing libraries and the parsing tree is getting cut off. Please check help->about–what does it say about 'html5lib' and 'lxml'? Are they both present?

If html5lib is missing, you should be able to add it with:

pip install html5lib

From the terminal. Then reboot the client and try again.

It could also be that you somehow have a wrong/bad version of the gelbooru parser. You might like to try importing the attached png through the same 'manage parsers' dialog and testing again. If it works correctly, then delete the old one and fix up the gelbooru 0.2.5 link under manage url class links in the same networking submenu.


9ba0bc No.9637

>>9576

Thanks. Yeah, I think I am just going to write a simple alternative for popups for non-windows people. Floating windows are just the biggest pain to get working multi-platform. I apologise for the inconvenience.


9ba0bc No.9638

>>9579

>>9580

>>9581

>>9585

>>9586

>>9589

Thank you for this information. The new shortcuts system I am slowly migrating over to does not support double-shortcuts like ZZ, but I hope to eventually move pretty much all events over to it, so remapping arrow keys for movement should be doable in the future. I am drowning in work atm, but as the downloader overhaul comes to a close I should be able to revisit some of these longer term rewrite/cleanup jobs. Let me know how it works for you as I roll it out.


9ba0bc No.9639

>>9587

I would like to rewrite session loading to make is less resource intensive and do delayed loading like you suggest here.

But yeah also moving to multi- pages should massively reduce the memory burden for downloaders that would otherwise have like 3000 thumbnails attached. Now the db just has to load the file import status list, which is basically just a URL and a couple of numbers per item, rather than all the tags and so on that go with it.

I have some more ideas here to relieve gigantico-clients, but it will have to wait for the next pass on the downloader overhaul.




File: 1426721772716.png (100.78 KB, 1624x1081, 1624:1081, 1327614072601.png)

7f2c0e No.471[Reply]


Drag and drop windows with tag rules. Show two windows side by side and one window can be programmed with the rule "ADD tag foo" and the other one has the rule "REMOVE tag foo, ADD tag bar" and you can drag and drop files to them.

Deriving tags from regex of other tags/namespace tags. A file has the tag "filename:big_ugly_name" and we could regex that namespace for another tag.

Tag sets with hotkeys: save a set of tags under a hotkey so it's quick to add them to a file while filtering

Opaque window behind tag list in the corner so it doesn't get hidden by picture background

Option to default certain mime types to be excluded from slideshow and only open externally, will help with videos with odd codecs that don't preview in the slideshow correctly

Option to specify hamming distance in "find similar images", you can't change the option once it's in the filter window and you have to enter the hash manually in the "system:similar to" option
567 posts and 187 image replies omitted. Click reply to view.

048c6e No.9445

One feature I'd really like to see improved is collections.

You should be able to control the collection namespace order. The current order (alphabetical) doesn't make much sense (chapter before creator?)

Files that do not match the namespace (loose files) should not be collected, or at least make this an option.

It seems unintuitive that double-clicking a collection opens the media browser with all files in the page, rather than only the files in the collection. The only way to view the collection currently seems to be an archive/delete filter or "open selection in a new page".

Another idea that would require more extensive reworking: Normal pages could use the first collection level (creator). You could then "dive" into the collection, which would open a new page with that collection's files collected by second level (title). And so on (volume, then chapter).


dba546 No.9538

Retroshare support like >>>/hypno/41015


860e00 No.9582

Vi-Vim shortcuts.


9260a0 No.9598

>>471

Hydrus script annotations


9260a0 No.9599




File: 059b0c500891970⋯.gif (158.61 KB, 500x556, 125:139, 059b0c500891970314a6fa0187….gif)

127435 No.9593[Reply]

I had a great week. I have completely revamped some important tag filtering ui, polished the new multi-gallery and -watcher, caught up on a bunch of older issues, and added parsers for imgur and derpibooru.

This release should be as normal tomorrow.

360454 No.9596

fantastic, Just get hydrus down to a reasonable image size and then you add derpi… hello 600k+ images again.

cant wait


103416 No.9597

>>9593

Getting Fiber in next few weeks… looks like I should get a server/NAS for Hydrus!




HookTube embed. Click on thumbnail to play.

8dff80 No.9429[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v315/Hydrus.Network.315.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v315/Hydrus.Network.315.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v315/Hydrus.Network.315.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v315/Hydrus.Network.315.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v315/Hydrus.Network.315.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v315.tar.gz

I had a mixed week. The gallery rewrite proved more complicated than I expected, so I put it off for a week.

all misc stuff this week

I updated to the new version of wxPython. It has a bunch of bug fixes that I think will help some menu regeneration problems some users have had, and I hope improve stability for the platforms/flavours where crashing has been a persistent issue. Please let me know iPost too long. Click here to view the full text.

50 posts and 6 image replies omitted. Click reply to view.

8dff80 No.9561

>>9516

>>9515

>>9517

>>9518

>>9520

>>9521

>>9526

>>9535

Thanks for all this. I am not working on dupe stuff at the moment, but I have written these points down and will keep them in mind. I am overall happy with the first version of dupe stuff but agree that it could really do with a full pass.


8dff80 No.9562

>>9519

I don't know much about vim–what sort of shortcuts are you looking for?


8dff80 No.9563

>>9531

I was talking to someone else who had this issue a couple of weeks ago and couldn't reproduce it. When I do 4/5/6 on my test machine, I get more each time. The fundamental object that is being tested here is a 8x8 matrix representing some simplified vertical/horizontal waveforms in the image (it is a DCT), so I wonder perhaps if certain classes of image tend to be 'mirrored' in some way on the two axes represented in the matrix, but I am not sure.

I'd be interested if you discover anything new about this. The max potential number here would be 63 or 64 but most of the action is in the 0-8 range, with anything above, say, 12, being completely useless, so I may end up re-doing the shape-based dupe detection to be a 16x16 matrix (i.e. 256 bits) to give us more resolution here, although this would slow search significantly, so it might be better just to put my time into better workflows. The big bottleneck now seems to be human attention (it takes ages to go through pairs by hand), so I wonder if I should attack that first.


8dff80 No.9564

>>9532

Yeah, this is highly requested, so I'll roll it in before this whole overhaul is done. There's some stuff here for it already:

https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/NEW%20Download%20System

but I haven't tested it myself and it isn't easy mode yet. You might just want to wait for me to add it naturally.


8dff80 No.9565

>>9532

>>9564

Shit, sorry, I read that wrong. Yeah, I'll add a gallery parser and 'searcher' for it when I have that system ready, so it'll just pop into the gallery selector button soon enough!




File: afce1fb7c68c55e⋯.png (1.52 MB, 1280x1732, 320:433, afce1fb7c68c55e7d5ade0f1e7….png)

File: 5f505978378ccd3⋯.png (7.83 MB, 3400x4600, 17:23, 5f505978378ccd323ed5e04eec….png)

File: 1125a234c31ad6d⋯.png (227.42 KB, 1939x557, 1939:557, client_2018-06-01_18-03-28.png)

9b85e6 No.9010[Reply]

I think that suggestions make the program better, improvements here and there also go a long way, but really there are hard aspects of the program that cause people to stop everything they are doing till they are improved/reworked.

let me give an example. I have an image db that is currently 2 million images large.

There was no way in fuck that I was keeping tabs on this shit on its own, so the program is very good in that regard, at least my shit is sorted and can be brought up if I tag/rate something.

now, here comes the stop gap.

I have 2 million files, and before my db corrupted, I had something around the area of 200k potential matches for duplicates

so I need to go through the duplicates first

my database broke through so I have only 827k images in the dup finder with 73k images.

Now thinking ok, while things get sorted out, why not do them? dup finder only works on 2 images at a time, regardless of if the rest of my db in or not there is work that can be done

well here is a random pair

im on a 4k monitor, a 55 inch 4k, so when an image is small and one is big, you can generally see one is better easily, however the smaller one is not worse by a lot, note the bigger ones embers, on the yellow shit at the bottom right, zooming into 100% on it makes it clear its not just an upscale, and that it actually is a higher resolution. but if you put these in the program, and find these in dup detector, note that due to the slightly different resolution the program won't keep zoom or where its zoomed in between the two images.

also note that trying to move where its zoomed into fast with the mouse results in making a selection rather then moving the image.

I also noted at least with ratings a while back making a 'this one is better/prefer this one' selection did not transfePost too long. Click here to view the full text.

41 posts and 8 image replies omitted. Click reply to view.

3e139f No.9493

>>9481

1: Define parent/child tag relationships and/or use namespaced tags. Auto-apply appropriate parent tags as you apply their children. Search for the intersection of tag:subtag and subtag:subtag2. Problem solved.

2: It seems you grew attached to a particular implementation of dumbed down regex, this is probably a good thing for dev to look into implementing, but for now, why not just use regex?

3: Use import timestamp to order them by, there's nothing to "respect". I'm not sure what you think this is, were you importing under 1k files total or something?

5: This is indeed a failure, but seems like user error to me.

6: Tag siblings is bad in Hydrus no matter how many people shill it, use parent/child relationships on tags instead wherever possible, only don't fuck it up

7: Custom namespaces are piss easy are you high my man

8: The lowercase autism is a carryover from Linux filename/tag autism I think, Linux doctrine holds that caps don't exist on a system level (sometimes) because it leads to confusion (because a lot of the guys who work on Linux are hacks, BSD is superior in every way btw)


99a911 No.9499

>>9493

Wait, why are tag siblings bad? I need some enlightenment

I recently started making use of them since there often are discrepancies between tags among the boorus, and i can't say i'd rather use parents on artists names (would look like an image has multple artists when it's just one), or character names and copyright names


c32131 No.9514

>>9481

3. I hate this so much, lots of times i find myself importing a bunch of images where among them there are some sequences meant to go in order 1-2-3-4 or even just pairs

And despite my best efforts to have them in order while importing (so that at least the import time is going to reflect and respect the correct image order), it always finds a way to fuck it up by putting random images in-between those sequences

My only options then are to import them separately or to tag them "sequence:##, order:#", but i find both of these to be horrible solutions

First one fucks up the "true" date in which i have got them, since because i have to upload them separately from the rest they get put at the end or at the start of what could be a month worth of images

Second method is first a pain having to create a tag and order for every sequence of 1-2+ images you have, and second you're then forced to use the tags to the sorting fucking it up, for example if you search for creator-sequence-order (with secondary as newest), all the sequenced images are gonna be sorted after all the non-sequenced, despite the fact that they could be the newest ones

I don't see why it has such an hard time to remember that they were already uploaded in the correct order, there weren't any pictures in between so why does hydrus thinks otherwise?


9c71fd No.9534

>>9493

Just got it why 5) (no creator upload order) is bad.

Imagine pulling one picture from 2 boorus. That means 2 creator times. Which one is to be selected? Because selecting one will break ordering of the second one.


e85bdf No.9555

DOWNLOAD BY TWITTER ACCOUNT

Way too many jap artists either post exclusively on twatter or post shit there that they don't on pixiv (whether uncensored or just not there). It's annoying to copy each tweet URL into Hydrus.

Also, a twitter login thing would probably be necessary because of private accounts.




File: 394b5cb77be6dce⋯.gif (249.76 KB, 500x375, 4:3, 394b5cb77be6dce092d06ef952….gif)

a67a54 No.9525[Reply]

I had an excellent week. A good first version of the new gallery parser is done for gallery pages, subscriptions, and simple drag-and-drop. Furthermore, all gallery pages are now 'multiple' (much like the thread watcher, which is now also multiple-only) and support multiple queries for different sites running on the same page at the same time!

This will be a big release. I still want to do some more testing, but I think I will have it out at the normal time. I recommend heavy gallery users wait for the release post to review the changes and make sure you update well.

2310aa No.9527

Thanks, looking forward to it!




File: f5c97ffef11977e⋯.gif (263.73 KB, 711x797, 711:797, f5c97ffef11977e610309257be….gif)

69deb6 No.9475[Reply]

This week went well, with plenty of work going into the gallery update, but I am afraid I felt more and more ill throughout the week and no longer feel confident I can put out a good release. I could tie off the work I have done and rush something out tomorrow, but I am not thinking completely straight and this is a big change, so I am putting v316 off for a week so it can all be tested properly and isn't in buggy limbo.

I'll just restart next week as usual schedule, so v316 should now be on the 1st August. Thanks everyone, and sorry for the delay!

3fbada No.9479

Im curious, these don't need to happen soon, just wondering if they are on the docket of things that will happen

is gallery log for multi thread watchers a thing, I ran into a thread where 14 images were deleted, and because I dont delete things currently, I wondered what the hell happened, so I had to load up that thread, ultimately they were images in trash not deleted and they were there for a reason, but opening the thread and trying to find what images did not download was tedious.

and on multi thread watchers, i have one of my multi pages that has 170k images, but 70k are already in db, would there be a way to add a 'highlight new' to the highlight options? I don't think the current file import options works with highlighting, either that or all the threads of 150+ images I just checked with only show new checked all had only new images, which would be shocking.

Like I said, neither of these need priority as my use for them is going to come several months down the road in the case of highlight, or just a nice resource to have in the case of the log, just wondering if they are on the docket of things to do, already in in a way im not seeing, or if they just got added to the ideas pile.


bf96de No.9480

File: abb0bdd7547e603⋯.jpg (86.74 KB, 1006x921, 1006:921, 1332267360830.jpg)

take a vacation nigga. you deserve it.


6d5ca3 No.9487

>>9475

Take care of yourself man, get some rest.


69deb6 No.9510

File: 0bb8c518b34a080⋯.jpg (285.95 KB, 1242x1583, 1242:1583, 0bb8c518b34a080f8a2b927966….jpg)

>>9479

Yeah, I have a panel rolling out for v316 that will show the highlighted watcher in detail, and you'll see the gallery log there.

I am not sure I totally follow this. The thumbs that show when you highlight should be those set to present in the watcher's file import options, so if you have it only to set 'new', it should not show 'already in db' ones. You'll be able to check this better with the new highlight panel, so please let me know what you figure out there!

>>9480

>>9487

Thanks lads. I played vidya all Wed, now I am back to it. 6/10 health, 10/10 enthusiasm, so we'll see how it goes.




HookTube embed. Click on thumbnail to play.

9767ae No.9368[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v314/Hydrus.Network.314.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v314/Hydrus.Network.314.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v314/Hydrus.Network.314.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v314/Hydrus.Network.314.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v314/Hydrus.Network.314.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v314.tar.gz

I had a great week. Tag import options are now easy to mass-manage, and I have fleshed out the new downloader system's file page parsers.

default tag import options

Almost done with tag import options, I promise!

This is an important but slightly complicated update. Nothing huge will change, and you'll see what is going Post too long. Click here to view the full text.

13 posts and 3 image replies omitted. Click reply to view.

9767ae No.9425

File: ee47c64520d991e⋯.png (3.62 KB, 512x86, 256:43, tumblr api post page parse….png)

>>9406

Thank you for this report. I've had a devil of a time getting xdg-whatever to work on some different flavours. What happens if you set an explicit launch path under options->files and trash? Does that work, or do you get a different error?

>>9408

Thanks. I have split the job up into like five pieces that I can slip in a bit every week. I'll basically make notes into multi-notes and then auto-attach a 'deleted' note in several new ways. Let me know how it works for you when I get to rolling out the fun bit.

>>9410

wx has a bunch of ways you can do it, I think, including like some popular 'ui builder' app where you are drag-and-dropping elements into a test window and it writes all the code for you, but I prefer to hand-code it all. There's a 'wx demo' that you can get alongside the base code, here:

https://extras.wxpython.org/wxPython4/extras/4.0.3/

https://extras.wxpython.org/wxPython4/extras/4.0.3/wxPython-demo-4.0.3.tar.gz

Which is the absolute shit and lets you see how different widgets go. It works well for me in doing multiplat with native controls, but it has instability issues if you do things in the wrong way and I know a lot of people who don't like it for a variety of reasons.

I've got a ton of experience with it now, so let me know if you end up liking it but want to know the right way to do sizers and layout flags or whatever. Feel free to copy any of my code or classes if they are useful.

Post too long. Click here to view the full text.

5ae410 No.9426

some feedback

i looked at the program for half an hour or so, tried to do some things and failed. if you have 150 IQ and are in software design or programming its probably easy to use but for just avg IQ person like myself i found the program to be not very intuitive :(


33a07c No.9427

>>9426

Did you use the help pages? Hydrus has a bit of an initial learning curb, but it functions like any booru so you should be able to do some basics.


5ae410 No.9428

>>9405

>>9427

ok, ill study it some more

i dot know what a booru is, i just manually downloaded about 500 memes from 4chan-pol over the last year or so and then i got the program ChanThreadWatch to download some image threads. Was hoping that i could just put about 100 images in a folder and then point Hydrus to that folder and let it go online and find some matches. Looks like it will take a few hours of reading though


9e54fd No.9436

>>9425

re: db migration

With the new version I'm not getting the issue anymore, thanks for fixing it.




4034a4 No.9407[Reply]

After SQlite run out of disk space on after exit maitenance, with following errors:

DBException: OperationalError: database or disk is full

and then

OperationalError: no such savepoint: hydrus_savepoint

(log https://pastebin.com/8JWzsrwj)

Hydrus client does not offer to do maitenance anymore after exit and does none while idle. Encountered on 310, after updating to 313 Hydrus asked to do maitenance once, but no maitenance was actually done, and it wasn't asking anymore again.

679dca No.9424

>>9407

Thank you for this report. The big maintenance routines need to make some large temporary files when they work, so they sometimes need up to, say, 5GB free space. There is a check for this free space for Vacuum jobs but not for repository processing. I will add one.

How big are the four client*.db files in your install_dir/db folder? How much free space is on that drive? How much free space does your system partition have, if that is different (some of the db files are written to your system temp location)?

It may be not doing more maintenance now because it may have successfully 'paused' the repository on the previous failure.

Best solution to your problem is probably to make sure your hydrus drive (and any drive with active files on tbh) is at the bare minimum 5%/10GB free, whichever is larger. Ideally, system drives should be 20% free, but I appreciate this is not always doable. Then check services->review services to see if the repository needs unpausing or whatever to continue work. If it has a more complicated error in its status reports there, please let me know and we'll work on it.


4034a4 No.9437

Actually, that "lack of space" is somewhat my fault, I temporarily moved database files from hdd to a ram-disk for them to be processed.

Total size of db files is about 4.5 gb, the disk was 9gb.

Also, public tag repository was indeed paused. After reenabling it, after-exit maitenance was again offered. So, solved ))




File: 4b5b39992727430⋯.gif (177.96 KB, 765x619, 765:619, 4b5b39992727430553b78f81b6….gif)

d8f651 No.9421[Reply]

I had a mixed week. The gallery work proved a little heavier than I expected, so I put off pulling the trigger until v316 so I can do it properly and instead caught up on smaller bug reports, library updates, and other little issues.

The release should be as normal tomorrow. I will try to catch up on my messages in the morning.



056949 No.9367[Reply]

Hello!

I'm not sure if this isn't a user error or something like that, but I've tried setting up a hydrus server on my (Linux-based) NAS and so far it seems to run well (I can upload and download images from different machines), but I've run into the issue that it doesn't seem to be possible to upload mp4 videos.

Whenever I try to upload one, I get the following error (after a while, so it seems like it actually uploads it to the server and only afterwards shows the error):

ForbiddenException

Filetype is not permitted!

Traceback (most recent call last):

File "include\HydrusThreading.py", line 283, in run

callable( *args, **kwargs )

File "include\ClientGUI.py", line 3460, in _THREADUploadPending

service.Request( HC.POST, 'file', { 'file' : file } )

File "include\ClientServices.py", line 818, in Request

network_job.WaitUntilDone()

File "include\ClientNetworkingJobs.py", line 963, in WaitUntilDone

raise self._error_exception

ForbiddenException: Filetype is not permitted!

I'm using the latest server version from last week.

It doesn't seem to matter which Client I'm trying to upload from (I've tried on Windows 10 and Ubuntu) and the file also doesn't seem to make a difference – I've tried different mp4 videos, large (500mb+) and small (<5mb), same error every time. The videos itself also work fine in the client.

I'm trying to upload with my admin access key, so it shouldn't be a permission issue either.

Is this intentional? Doesn't the server support mp4 videos or videos in general? Or is it an error?

If videos aren't supported on the server, do you think that's something you could add easily? If that would require adPost too long. Click here to view the full text.

6 posts and 1 image reply omitted. Click reply to view.

f6a389 No.9379

>>9374

This isn't even the thread I was replying in.

I'm positive I replied to the v314 thread.

I have no idea how this happened, sorry.


056949 No.9402

>>9370

And a final follow-up:

Seems like the issue is with my server somewhere, because running it locally via Docker (on a Debian-based image) and uploading mp4 files works fine.

If you're curious, this is the Dockerfile I've written: https://paste.ee/p/0EsL7

If you haven't worked with Docker before, assuming you are on Windows:

1. Install Docker: https://docs.docker.com/docker-for-windows/install/

2. Create a new directory, call it whatever you want

3. Create a new file in that directory called "Dockerfile"

4. Open a shell, go to that directory and execute "docker build -t hydrus-server ."

You can then run "docker run -p 45870:45870 -p 45871:45871 -p 45872:45872 -d hydrus-server" to create a new container from the image and run the server.

To reach the server, you'll have to use 0.0.0.0:45870 etc. in hydrus client.

Running "docker ps" will give you a list of running containers. To stop the server, note its container id listed there and just run "docker stop <container id>" (actually, you don't even need the whole id, just enough so Docker can identify it, the first 2-3 letters should be enough).

I'll probably switch over to running via Docker anyway, so you can consider this issue closed. :D


355af5 No.9404

>>9402

Thank you for these follow-ups. I will chase up how the 'unable to grab ffmpeg' error is being propagated up to something so non-useful. Please let me know if you run into any further trouble.

>>9373

>>9379

np m8, thank you for this report. I get the same thing–it looks like pixiv are rolling out a new dynamic update for their manga now as well. I will try to fix these this week.


056949 No.9409

>>9404

I've tried some more stuff (updating ffmpeg, running the server without a manager (I use something called supervisor to manage running lots of stuff)) and it still didn't work on my NAS.

But downloading a static build and putting it into the "bin" directoy of hydrus DID work.

I'm not sure why hydrus server wouldn't be able to access my global ffmpeg though - I've tried with several users and it shouldn't be a permissions issue, ffmpeg is definitely in the PATH as well (running "which ffmpeg" returns a valid path).

Anyway, as I said, I'm probably gonna use Docker anyway, but you might want to look into that error message generation. Because I can remember getting an error about missing ffmpeg before when I hadn't installed it yet, so this must be somehow different. :D


355af5 No.9423

>>9409

Thanks. The errors are a bit better this week, and I'll keep investigating this.




Delete Post [ ]
[]
Previous [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
| Catalog | Nerve Center | Cancer
[ / / / / / / / / / / / / / ] [ dir / 1piece / dcaco / firechan / imas / lovelive / marx / mde / sunny ]