No.541
A number of things about hydrus are a bit unintuitive, and the documentation is not helping. I think the basic local functions are also presented in a way that artificially raises the learning curve, but I'm used to it now and can't really remember what I was having problems with. I'll tell you one thing though- having a bunch of tabs hidden behind the "pick new page" under View, when probably most of those items should really be under Download (which is sparsely populated and begging for more items), is an organization nightmare.
I set up a hydrus server on my desktop, and configured my laptop as a client. The client can only see things it uploaded- a picture uploaded from the desktop isn't seen on my laptop, pictures uploaded from my laptop aren't seen on the desktop client. Apparently it's going to sync in 28 hours……………… Did my right click > upload, then pending > upload actions not upload the file? If not, what was the popup that told me "upload finished"? If so, why does the client need to wait 28 hours to see a change in the server? Why is there no "sync now" button?! If the file is there why must I wait 28 hours to have it show up in the client?!
The client db and server db are separate, even on the same computer. This means that on the PC running the server, I have to import pictures to the hydrus client, upload them to the local hydrus server, delete them from the hydrus client lest the library be twice as big as it needs to…. then if I ever want to view a picture on the computer where the pictures are in fact stored, I need to "download", eg roundabout copy from hydrus\db\server_files to hydrus\db\client_files? To add insult to injury, trying to view a picture with the internal viewer causes it to "download" but not open- you need to retry opening it after the download. Am I doing something wrong here?
There are two different pages named "manage services" that contain completely different settings. I now understand that one of them is for server.exe, but you would do well to rename one of them to something less conflicting.
Tags and the tag server- I have absolutely no idea what is going on here. If I upload a picture to a server, does that server inherit the picture's tags? I don't think it does (I'll find out in 28 hours, probably), but this means I have to copy all tags from local and paste into the tag server, for every picture. I'm willing to bet this is what "perform a service wide operation" is for, but nothing in the documentation fully explains what "mappings" are and what "copy all mappings to [local] tags" will do. F3 seems to always bring up the local tag editor, too, so I can't easily tag things on the remote end- I have to switch over to remote tags as opposed to just F3 > type tag > enter > move on with life. Or tag locally and periodically export the local tag database to the server.
Tag parents, once created, apparently cannot be deleted. I accidentally added one I did not mean to, and although it is not affecting my library, it's sitting in the parent tag management box with an X next to it. Why won't it go away?
I hope this doesn't come off as hostile, I'm just feeling slightly exasperated and overwhelmed.
No.545
File: 1427575729053.jpg (59.63 KB, 720x479, 720:479, 170aadeec25c2e3f06a8a633d5….jpg)
Hey, I'm sorry this has been frustrating.
For the most part, I have developed the serverside of things either for my own use, or in cooperation with people who have been using the software for a long time and understand it very well. It can be difficult to guess what newer users will trip up on.
I could explain everything that is going on here, but I think it would be more productive if I first focused on what got you to this point. If you don't mind answering:
About how long have you been using hydrus?
Did you ever connect to my public tag repository?
What do you think the hydrus server is meant for? What do you want to do with it?
Did you start reading the help?
If you did, what was good in the help? What was bad? What confused you?
Did you quit reading the help? If so, when was that?
Have you looked around file->options? If so, was it obvious what the tabs did, or confusing?
Thanks!
No.547
This is probably my third day playing with hydrus.
I'm looking for a way to organize and navigate my picture + webm collection, which the client.exe does very nicely. My desire to use the server is summarized in help/server.html
>Would you—
> like to share your image collection between your computers, creating a simultaneous backup?
Yes! I do realize that hydrus seems to be designed around *massive* collections meant to handle thousands of contributions without wasting resource, and that the server database is likely going to be used in hub / spoke scenarios so the redundant client db makes sense in that light… but I only came to this realization after poking around all the settings, and that line I quoted above was a bit misleading. It's just kind of baffling to me that I uploaded one single file and there's no option to manually trigger an update. There are plenty of other database actions in the client that warn me left and right that what I'm about to do could take a lot of resources and time. Why must the server be locked to a schedule?
>Did you ever connect to my public tag repository?
Briefly, to verify I was using the server right. It started downloading a lot of data and I stopped it shortly after.
>Did you start reading the help?
>If you did, what was good in the help? What was bad? What confused you?
The help is fine, its scope is just limited. It doesn't quite explain the server functions built into the client beyond the act of connecting to a server.
>Did you quit reading the help? If so, when was that?
help/db_diagrams.html was a bit too involved for me.
>Have you looked around file->options? If so, was it obvious what the tabs did, or confusing?
Everything in the options is either self explanatory or easily understood. I think "default advanced tag options" was the only thing that was not instantly understood, since it's just a blank box.
No.598
File: 1428172120616.jpg (219.46 KB, 1976x996, 494:249, d07b9688b282401564a79fe480….jpg)
>>547Thank you for the response here, and sorry I only got around to it now.
I should put more work into making the server more flexible and user-friendly. I've discussed adding a manual sync, but it is deep into my to-do list. I've also semi-planned a true 'sync' service that would automatically push and delete between multiple clients. Not many people use the server, so it tends to get put on the back burner.
I am glad you found the help and the client generally easy to understand. I have made a note to put some time into improving the server help and better explaining what it does.
>>541When I designed the server, my chief intention was to offload CPU effort to clients. I don't like that modern servers do so much unneeded work, nor the greedy motivations that are moving data management and calculation away from users. So, my services are designed to only share a few kilobytes per user per day. They don't do any searching or other complex on-demand query, instead acting as synchronisation hubs for clients. A client submits things to a server, and after a time, every other client will sync with that cache. I had to choose a check-in period, and chose 100k seconds, which is a bit over a day.
100k seconds works great for my tag service, where it doesn't matter if someone sees a few tags at that delay, but it is proving a problem for home users, who want insta-sync. I think, ultimately, that I will write low-client-count bandwidth-heavy versions/modes of my services for LAN use. Perhaps I should state this explicitly in my help.
I have also considered initialising new clients in a 'simple' mode that'll cut out advanced buttons like 'perform a service wide operation' that are best clicked only by people with experience in exactly what mappings and namespaces and archives are. Or perhaps launching a wizard to take the new user through some initial steps, or even just putting a glossary further up front in the help, possibly with pictures to better illustrate the concepts. I am not sure. Any further suggestions you have on this would be welcome.
No.599
I'll delete those db diagrams as well. They are way out of date, and not as useful as just booting sqlitestudio.
No.618
>I think, ultimately, that I will write low-client-count bandwidth-heavy versions/modes of my services for LAN use. Perhaps I should state this explicitly in my help.
I would appreciate that very much.
Unrelated question: I batch imported 500 pictures from a pixiv artist, and used regex to tag the page numbers, titles, and added the creator: tag for all the pictures. These tags go to my local server.
I didn't commit/upload the pending tags. I select all the pictures and run through the archive/delete filter. The tags for all the pictures I deleted are still pending to the server, and I can't find a good way to eliminate them. I tried to export the tags for the remaining pictures, thinking I would forget/cancel the upload and reimport the tags, but the tags haven't been uploaded so the export yaml was useless.
Is there any good way to go about this? Or will I just have to upload the tags, then view all the pictures on the server, export all tags for all pictures, delete all tags, import the yaml?
No.634
File: 1429037356170.jpg (1.23 MB, 1311x2000, 1311:2000, fbc8e49bbc9556e65d283e2d1b….jpg)
>>618I am sorry for the late reply here, I hope I can still help.
You should be able to see deleted files by changing the 'local files' button on the autocomplete dropdown to 'all known files'. This way, you can do something like:
system:file service:is not currently in local files
system:number of tags > 0
And then make sure you are only including pending tags. You can set the specific tag service as well (instead of 'all known tags'), to limit the tag count calculation just to your private tag service.
You can then select all the files and remove the pending tags manually through the manage tags dialog. There isn't a way to automatically say 'delete all pending tags from all deleted files' yet. Maybe I should add a warning on deleting a file that says 'this file has pending tags, do you want to forget them?' On a filter, it could trigger at the end.
No.638
thanks for the tip
unrelated, but I don't know where else to put this:
I discovered running hydrus from a samba share works well. As in, I run //server/share/hydrus/client.exe , no issues yet
further unrelated: the viewer draws a grey backdrop before drawing the picture. if you hold right arrow, you'll see nothing but grey. maybe this is a stupid request, but perhaps get rid of the intermediate grey? If I have a sequence of images irfanview can display a poor man's gif by just holding right arrow and cycling through. Pixiv has their stupid proprietary animated picture thing which scraper programs collect as a series of jpgs. I have yet to find a way to make a lossless webm out of a series of images (every method I've tried re-compresses)
No.643
>further unrelated: the viewer draws a grey backdrop before drawing the picture. if you hold right arrow, you'll see nothing but grey. maybe this is a stupid request, but perhaps get rid of the intermediate grey?
actually, yes definitely get rid of this. It restricts the speed at which I can sift through pictures
No.644
>>643
This is both a consequence of wxpython (the interface library), and limitations of storage retrieval. It might be possible recolour the grey placeholder to the normal background, but the images would still need to be loaded from the hard-drive/ssd before being shown.
Maybe start loading the image while showing the image just before, and first display the new one when its fully loaded. It would reduce response times, and might not be possible due to wxpython, but it would look much better when flipping though.
No.646
File: 1429480730510.jpg (121.56 KB, 831x690, 277:230, 9af4058feb501077ac3cd146ab….jpg)
>>638
I'm surprised about the samba thing, but if it works, that's great! I've added some database profiling tools in v154, and I'd be really interested in seeing your search speed over that connection.
>>638
>>643
>>644
I hate the grey background too, but never got round to fixing it up. It'll just take a bit of thought and work to reuse my media window or something so I am not generating new ones with every new image, or if it is an event issue, I need to reorder my events a bit. I've put this on my priority list to fix. If I have time on Tuesday, I'll do it, otherwise next week for v155.
No.649
File: 1429503323479.png (182.8 KB, 344x393, 344:393, 7d0407b5023bad50f0e6c65013….png)
>>644
>limitations of storage retrieval
irfanview and honeyview have no problem sifting through pictures at roughly 15-20fps, on an HDD without caching. But I should probably just make an apng. Still though, no harm in improving hydrus' load times.
>>646
It takes 20-30 seconds to actually start client.exe over samba, over 300mbps wireless N wifi, but once booted up it is effectively as fast as using the client locally- no noticeable slowness in thumbnail generation or even general image viewing. I am incredibly satisfied with using hydrus in this way- now I can access my remote libraries with all the power and ease of client.exe, while completely cutting out server.exe. Not having to run software on the host computer is a plus as well. We previously discussed allowing client.exe to store it's database on samba shares- but under the current model the PC hosting that samba share wouldn't have access to said database. Just running client.exe from samba neatly bypasses that problem.
The internal viewer doesn't seem to "stream" webms/gifs/videos though, so files >15MB will have a load time of 3+ seconds while you stare at the grey background. While this in of itself isn't a noteworthy problem for me, I think addressing this will improve hydrus not only for my samba scenario, but also for those with slower HDDs (as far as gif/video content is concerned, at least). For what it's worth I know that mpc-hc re-worked how they read files from a local drive in 2014 (per my request :). The change allows you to watch files that are incomplete/currently downloading (eg, "streaming"). If you can get python to access files in that way…
For the record, now that I've found the samba solution, and having been using hydrus for a while now, my frustrations have evaporated. I'm starting to find the lack of file names liberating rather than restricting. Still not sure how I'm going to batch remove duplicates from the entire library, though. I can't just scan the database and delete things with my third party program: I'd want to merge the tags of the duplicates before deleting the inferior copy. Which is something that would have to be done one by one…
No.671
>>649
Curious, does it go any faster if you map the share to a drive letter and then run it? Assuming the db is running on purely relative paths, every file access is called with the UNC path (double-backslash notation), which requires a bunch of netbios overhead to look up the server name and browse to it. Using a mapped drive lets Windows avoid a lot of those network lookups.
No.674
>>638
>I have yet to find a way to make a lossless webm out of a series of images (every method I've tried re-compresses)
I don't know if this helps, but the huffyuv video codec is lossless
No.675
Of course, you'll need to encode as mkv or something like that because webm only supports VP8/VP9
No.676
>>671
My LAN is fast enough that I could not notice a difference, if there even was one. It didn't help the startup time, which is still 40s.
That trick did wonders for FreeCommander XE though, thanks! It use a single thread and locks up for 10s of seconds at a time when browsing net shares, even LAN ones. But it's fast and fluid with a mounted drive.
Given the drastic improvement in FreeCommanderXE and no improvement in hydrus, I think it's safe to say Hydrus doesn't suffer from the lack of mounting.
>>674
I've actually thoroughly compared all of the lossless codecs I could find (I recently bought an HDMI capture card and want to get the best possible quality) and found that x264 lossless is generally the best. Last time I tried to use aPNG I ended up with an 80MB file so perhaps just making an mkv would be best. I've seen webm slideshows with quality so high I thought they were lossless, but I'm not breaking ground in this area…
Although the hydrus dev has expressed interest in creating a container to treat multiple images as one file. Maybe I should just wait for that and use hydrus' slideshow, instead of trying to create a video.
No.682
>>676
mkv rules. Now if only Hydrus could have smarter video support…
I don't think Hydrus should be trying to play the video anyway. Make a few thumbnails and play them in sequence in the viewer, maybe? But leave the actual playback to the dedicated programs. I've suggested a few times that we need a way to specify files that will open externally by default. Video is complicated enough that when making a viewer you really need to go big or go home.
Creating a container? Frankly I'd be inclined to just fake a mimetype for CBR/CBZ files and use that. They're nothing but renamed zip files with a little metadata.
The tricky part would be changing the zip contents, but Hydrus needs a method of replacing/editing files and keeping the tags anyway.
No.684
File: 1430259853305.jpg (676.3 KB, 1878x2804, 939:1402, de0079c7dc027ae7f2ce00e3e0….jpg)
>>682
I really like the idea of grabbing five or ten frames from mkvs or whatever and just paging through those like some non-youtube video sites do, and having external launch be a one-click. I'm swamped with work atm, so I'll think about it a bit and then do a prototype.