[ / / / / / / / / / / / / / ] [ dir / bbg / choroy / dempart / doomer / flutter / vfur / vietnam / vril ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog   Archive

Winner of the 75nd Attention-Hungry Games
/caco/ - Azarath Metrion Zinthos

March 2019 - 8chan Transparency Report
Name
Email
Subject
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 16 MB.
Max image dimensions are 15000 x 15000.
You may upload 5 per post.


New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Experienced user with a bit of cash who wants to help out? ---> Patreon

Current to-do list has: 1,626 items

The program is now on Python 3! Check v335 release post if you need to update from before then!

Current big job: None--check the pinned poll thread


File: 3711d0607c5673e⋯.mp4 (3.38 MB, 1024x576, 16:9, 3711d0607c5673ea2d3f398a9a….mp4)

ff909c  No.12152[Reply]

Ok lads, as I am now finishing up OR search, I am soon going to be free to work on a new 'big job'. I am pleased that I was able to make simple Client API and OR search in much faster iterations than previously. I hope to continue like this, keeping the next big job 8-12 weeks at the most before running a new poll.

The current list is:

Just catch up on small work for a couple of months

Reduce crashes and ui jitter and hanging by improving ui-db async code

Clean up code and add unit tests

Improve tag siblings/parents and tag 'censorship'

Add ways to display files in ways other than thumbnails (like 'details' view in file explorers)

Add text and html support

Add Ugoira support (including optional mp4/webm conversion)

Add CBZ/CBR support (including framework for multi-page format)

Add import any file support (giving it 'unknown' mime but preserving file extension)

Improve 'known urls' searching and management

Explore a prototype for neural net auto-tagging

Add support for playing audio for audio and video files

Add ui for waifu2x and other file converters/processors

Write some ui to allow selecting thumbnails with a dragged bounding box

Add popular/favourite tag cloud controls for better 'browsing' search

Improve the client's local booru (this likely now means a backend migration to the Client API)

Improve duplicate db storage and filter workflow (need this first before alternate files support)

Improve shortcut customisation, including mouse shortcuts

Add ratings import/export, and add 'rating import options' to auto-rate imports

Add more commands to the undo system

Improve display of very large/zoomed files in the media viewer

Set thumbnail border colours on user-editable rating and namespace conditions

Post too long. Click here to view the full text.
98 posts and 8 image replies omitted. Click reply to view.
Post last edited at

ff909c  No.12327

>>12293

Thank you. There are multiple jobs for improving notes that I have not been able to get to. I am adding "Expand file notes system" to the list to cover a general push in this direction. This would include multiple notes and likely connecting the notes system to the downloader as well.




YouTube embed. Click thumbnail to play.

71afbb  No.12289[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v348/Hydrus.Network.348.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v348/Hydrus.Network.348.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v348/Hydrus.Network.348.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v348/Hydrus.Network.348.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v348.tar.gz

I had a good week. It is mostly small updates and fixes.

all misc this week

After the recent weeks' thumbnail work, I became frustrated with how laggy the client's file system could get under heavy load. This week I have written a new file access locking system that has less latency under heavy simultaneous use and is also safer for certain edge cases. Multiple import queues and thumbnail fading and regular media browsing will all interact with less lag now.

The Client API has several improvements: the /add_tags/add_tags call has its 'hashes' parameter fixed and now does sibling-collapse and parent-expansion on its 'add' tags (which caPost too long. Click here to view the full text.

9 posts and 3 image replies omitted. Click reply to view.

00534b  No.12309

>>12305

>How do I make it so hydrus changes this?

A small script and the client API.




File: 3f8a8d22364b1de⋯.jpg (111.13 KB, 400x400, 1:1, 1394958218797.jpg)

d5679d  No.9327[Reply]

Here you can ask questions so that the board is not clogged with small threads.

>>6021 has reached its bump limit, so I made a new thread.

683 posts and 126 image replies omitted. Click reply to view.

07a0db  No.12270

>>12264

Unwieldy how? I'm curious because calculating ratings by pairwise comparison sounds exactly like a feature I thought I wanted, and I didn't know hydrus used to have it built in.

Is it that it takes too many comparisons to get an accurate score?


8b816f  No.12280

>>12257

I love you HyDev, thank you.


9cb648  No.12294

How do you deal with variant sets? Is there any way to mark as alternatives images in current selection without going through duplicate search?


034b1d  No.12308

>>12294

Yes in right click menu (advanced mode on), or you can bind it to a keyboard shortcut


4cd362  No.12311

>>12270

Main problem was setting up a workflow that compared 'appropriate' files against each other. I forget the exact old implementation, but I think it just presented random pairs. Asking whether a picture of a cute elf is 'better' than a screenshot of a thread proved completely unhelpful. Also to 'accurately' rate a file on a ten-star scale, it was like 4-6 decisions, which on 100,000 files means way too much work, especially when the comparisons were so bizarre.

I think if I were ever to bring a system like that back, I would have to use a search or something, like how I am going to add to dupe filter this week, so you'd say 'ok, let's rate some elf pictures against each other'.

The whole idea is on the back-burner for now. Just clicking your estimate of whether something is 4- or 5-star is quicker and gives more control for subjective adjustment than the last attempt at 'objective' rating did. I could bring it back, but I am still waiting for a better solution. It is still vapourware, but maybe in five years we'll be able to teach a neural network to recognise ratings, or more likely rating estimates (like 'I know you like tag x, y, z, so I'd estimate this file that has those also is prob 4-5 stars), to massively reduce the workload here.




File: 7a56ef4350b0bb9⋯.jpg (64.84 KB, 445x488, 445:488, 7a56ef4350b0bb96c5b6a4eb80….jpg)

ff2c17  No.11542[Reply]

BUGS THREAD

92 posts and 41 image replies omitted. Click reply to view.

ff2c17  No.12253

>>12147

>>12231

I am sorry you had a hard time with this, but I am glad you are putting things back together.

I know it sucks to hear right now, but this sort of problem can be a great driver to get a good backup routine going. It happened to me about ten years ago, and has happened to more than a few hydrus users since I started doing this–you lose a load of shit and previous effort and decide to spend a bit of cash to avoid that pain again. I highly recommend a cheap external usb drive, either from a store or second-hand from a friend. Even just having a copy of the .db files–rather than including all your client_files as well–can make a real difference.

If you are interested and not sure how to do it:

https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#backing_up


ff2c17  No.12254

>>12149

Thank you for this report. Have you had any crashes or power outages recently? The file identifier error may have occured due to a recent-ish hard shutdown of the client. As a precaution, I recommend you shut down the client and check the 'help my db is broke.txt' document under install_dir/db/ as background reading. You might like to just check the integrity of your db and maybe run a chkdsk to be sure your drive is ok.

Can you also check install_dir/db/client - 2019-4.log? If you scroll way down to the timestamp where this was happening (ctrl+f -> 'file identifier' will probably get you to it too), are there any error tracebacks related to the pause/unpause or missing/broken file imports? I would be interested in seeing that information, as it may better explain why subs were acting wrong here.

Subs may take a few seconds to register a 'pause' event, and will typically not stop working until the current gallery page or file being worked on is done, but they shouldn't flip behaviour entirely.

If this was just a blip due to a power cut or whatever, and you are confident your drive is healthy and database is not corrupt, you I think have two paths: if you keep using your client and it seems totally fine, especially if those subscription files are reimported and are fine, I think it may have fixed itself, but if you keep getting it, particularly in relation to the same set of files that are tagged on the PTR, your solution is to go services->review services->remote->tags->PTR and hit reset processing cache.

Let me know how you get on!

Post last edited at

ff2c17  No.12255

>>12189

Yeah, it looks like they started putting secondary post attachments to a new subfolder in the URL structure, and it was being recognised and incorrectly truncated hydrus-side. I am rolling out a new url class for 348 that seems to fix it. Please let me know if you still have trouble here.

It wasn't a block, but the accidental hydrus truncation was doing something like this:

https://yiff.party/patreon_data/9078764/24083586/3164951/coffee-girl--SFW.jpg

->

https://yiff.party/patreon_data/9078764/24083586/3164951


1f987d  No.12271

I've noticed that the url import isn't attaching tags to gelbooru and pixiv links anymore. Is there a setting I'm missing? I am on version 347 now, I was on 343 before.


ff2c17  No.12310

>>12271

It seems ok to me in a drag-and-drop test on a gelbooru URL I just did now, and I do not think I have changed anything in the past four weeks–can you say any more about your situation? How are you importing this URL: through a search or a drag and drop? Have you imported any new parsers for those domains? And what are your 'default tag import options' set up as under network->downloaders->manage default tag import options? Do you have one TIO for all domains to rely on, or specific ones for gelb and pixiv? Do they all look correct?




File: ea31ab670c8dee7⋯.png (12.55 KB, 340x175, 68:35, mariadb-usa-inc.png)

928501  No.9068[Reply]

ITT: create proposals for making Hydrus more optimized.

Proposal: Why can't Hydrus switch to MariaDB?

If it is faster, then it should be better. The only trouble is having the need to rewrite the queries, which from an SQL standpoint should be a non-issue, right?

List of Databases with Open Source License and Open Source APIs:

SQLite - Currently used in Hydrus, has minimal features

MySQL - A more well-rounded SQL Database with user management

PostgreSQL - An SQL with complex features with less performance

MariaDB - SQL/NoSQL database with heavy optimizations

ElasticSearch - A literal search engine instead of a normal Database

Teradata - IDK

https://www.digitalocean.com/community/tutorials/sqlite-vs-mysql-vs-postgresql-a-comparison-of-relational-database-management-systems

https://www.infoworld.com/article/2611812/mysql/mysql-face-off--mysql-or-mariadb-.html

49 posts and 3 image replies omitted. Click reply to view.

138bc8  No.11586


55366b  No.11927


b2bc38  No.12295


f049d5  No.12302

>>12295

Why don't you actually develop something on your own instead of endlessly shitting out github links


b2bc38  No.12307

>>12302

Nah that is for >>12277




File: b2474c1d0a864f1⋯.jpg (193.25 KB, 1280x853, 1280:853, b2474c1d0a864f1d89b2622611….jpg)

9923a1  No.12279[Reply]

I had a good week, mostly working on small jobs and fixes. File and thumbnail access should be less laggy, some drawing code should be more resource efficient and stable, and the client api has a bit of work as well.

The release should be as normal tomorrow.



File: d7e685360192f2a⋯.png (20.52 KB, 722x449, 722:449, api-image-for-blog.png)

b45f6a  No.11626[Reply]

ITT: We propose new features that can be solved by using the API, and recommend new API commends for it

1 post omitted. Click reply to view.

59ad1a  No.11761

Here are some features that I can think of that are useful

1. Image tag and metadata "hash"

Image data hash, so if you were to download metadata from the same file twice, and the metadata has not changed, the hash would be the same, telling the user to not download it again and waste precious bandwidth.

2. IPFS H2H API

IPFS operational API, so that you can create folder or file hash of any search results, and be able to pin or unpin them at will. This will be useful in the long term when people use it for Hydrus-2-Hydrus (H2H) P2P file share.

3. De-duplication API

internal APIs for file de-duplication software like Pippy360 for images and Chromaprint for music, and then allow external APIs be called for IQDB-like file search and hash download capabilities. (requires APIs for uploads first)

4. Headless Drone API

Headless "master+drone" Hydrus APIs, that allows one "master" to distribute downloader scripts and "download orders" to them, and the "drones" will return either downloaded images and tags, or an error message, back to the "master"


59ad1a  No.11764

Also having an API similar to this would be useful https://github.com/mserajnik/hydrusrv/blob/master/API.md


d9f85b  No.11802


d9f85b  No.11843

>>11764

This is a one-upping https://github.com/rr-/szurubooru/blob/master/API.md now THIS is an API that we can learn from


b45f6a  No.12269

bumping this so that we can discover more uses of API better.




YouTube embed. Click thumbnail to play.

641234  No.12084[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v346/Hydrus.Network.346.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v346/Hydrus.Network.346.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v346/Hydrus.Network.346.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v346/Hydrus.Network.346.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v346.tar.gz

I had a great week. There's a bunch of fixes and thumbnail improvements, a more powerful manage urls dialog, and a prototype OR search to try out.

thumbnails

Last week's big thumbnail change seems to have gone fairly well overall. There do not seem to be any huge errors anywhere, and the simpler file system and code is great, but I was annoyed by how slow certain regeneration scenarios turned out to be, and how some thumbnail resizes were a bit blurry.

To address this, this week I have cleaned and improved a bunch of thumbnail loading code. It was much nicer to do now I only have one thumbnail file to juggle. Thumbnails should now be aPost too long. Click here to view the full text.

21 posts omitted. Click reply to view.

a0ba9a  No.12145

> Can you give a concrete example of an export search that is failing for you? Is it by chance a 'system:rating' only search?

I have two exports, both are a simple search for one tag.

One to sync tag "bg", another to sync the tag "potential bg".

Both export folders have been left empty or incomplete when I use hydrus. I have to poke around the export settings to make sure they actually worked before closing hydrus.


460898  No.12146

>>12135

>In 347, the 'evangelion' search will be run first, so the potential search domain of '-rei' is cut down from your entire collection to your x eva files.

Oh nice, especially with various system tags in place to limit the search, that'll cut down the time quite a lot for me.

On another topic, for the list of next big things, was the booru something you had in mind for working on? I had an idea about how it could be integrated more directly into the main UI, which I could write more about provided that there's interest in the booru right now.


641234  No.12250

>>12136

I would like to have that, particularly for my own use, but unfortunately this functionality does not seem to be built in to my ui library, and furthermore my thumbnail panel is all my own custom code. I will have to write both ends of a system like this, so I have it as a 'big job' for now. It is currently "Write some ui to allow selecting thumbnails with a dragged bounding box" here: >>12152


641234  No.12251

>>12145

Thank you, the 'delete after export run' bug should be fixed in 347 for export folders that make filenames that include subdirectories. Please let me know if you still get it in the new version.


641234  No.12252

>>12146

Yeah, now here as a new 'big job' to work on: >>12152 . However, I am not totally sure what to do with the current booru. It was always a prototype that then never got revisited. I think next step is probably to convert it to the nicer new Client API code just so it is all operating on the same system, and then work on templating so users have more CSS/HTML customisation options.

Then again, I do know of some users working on booru wrappers for the Client API as-is, so if they can do a better job than I can fit in, it may be better ultimately for me to abandon it. If the booru does not prove all that popular in the poll and some other users can fill in the gap, perhaps we can revisit the whole idea and instead I give more power to them via the Client API.

If you have particular ideas on integrating the booru into the actual client ui, I am interested to know what you are thinking of. Please post in that big poll thread if you would like.




File: 7ece33b770e9701⋯.gif (565.95 KB, 300x166, 150:83, 7ece33b770e9701da946ffa95c….gif)

89c662  No.12143[Reply]

I had a good week. I fixed some bugs, extended thumbnail resizing into a much smoother two-stage system, and fleshed out the OR search workflow as I had hoped.

The release should be as normal tomorrow.

1a0602  No.12148

What 'bout wildcards in parent tags?


89c662  No.12244

>>12148

That is an interesting idea. It might be dangerous if I were not careful, but I like it. Unfortunately, the tag siblings and parents system needs significant overhaul at the database level before I can do anything clever and new with it. If you are interested in new sibling and parent tools, please vote for "Improve tag siblings/parents and tag 'censorship'" here >>12152 .




YouTube embed. Click thumbnail to play.

1cdbfc  No.12153[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v347/Hydrus.Network.347.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v347/Hydrus.Network.347.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v347/Hydrus.Network.347.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v347/Hydrus.Network.347.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v347.tar.gz

I had a good week. OR search is essentially finished, and I cleaned up and fixed a variety of other things.

A new 'big thing to work on next' poll will be going up soon. If you are interested, please check out the discussion thread here:

>>12152

or search

As previously discussed, I have moved OR predicate construction to the standard dropdown list below the tag input. It now appears as the top result, where you can hit enter on it to submit it as-is. Also, while under construction, a 'cPost too long. Click here to view the full text.

17 posts and 3 image replies omitted. Click reply to view.

1cdbfc  No.12207

>>12167

This looks like a variant of an out-of-memory error. Were you zooming way in on a big file during dupe comparisons? Unfortunately, my image viewer is not good at big zooms atm, and if you like 2000% on a 4,000x3,000px png or something, it'll hit a limit somewhere trying to make a bigass bmp for the whole thing. You may have noticed the client getting laggy during bit zooms like this. A future iteration will have a tiled rendering system that only keeps the bit on screen in memory, but for now it is still basic.


1cdbfc  No.12208

>>12164

Oh yeah, your odds of getting the same image in a row are fairly high, as my current 'get some dupes to process' job batches decisions together into various similar types. You'll likely get several pairs in a row that feature the same file as A or B.


1cdbfc  No.12209

>>12190

Yeah, it'll say the action as well. In the filter it will be something like:

"Deleted in Duplicate Filter (better/worse): worse file deleted."

If you do a mass action from thumbnail right-click, it says this variant:

"Deleted from duplicate action on Media Page (action): one/both deleted."


180b1e  No.12238

Just did another clean up as hydrus was eating 6gb on launch and dups were starting to lag the program to much, its now starting sub 1gb with next to nothing changed since the last time I did this which got it down to 2.3~gb

still honestly see no reason for the watchers once they 404 to eat that much space, but good to see the program is getting even better with ram management.

>>12205

as far as the images themselves go, yea, they are likely changing 1 pixel and re saving it, my shotgun approach to downloading has managed to seemingly sweep up everyone ever made but there are a few times where I knew the image was never getting posted again I just straight deleted the shitposts, but this time…

I thought I got rid of all of them, but apparently it's somehow finding more.

>>12208

after a while I thought this too as I was getting somewhere of 455k images, it was really odd that there would be 3 or 4 images that were dups of the same image that would come in.


180b1e  No.12239

>>12204

I have to ask, is it possible to have a custom sup finding thing? sadly I went though these a number of weeks ago, but I was checking an import archive, and apparently what I used to download corrupted images, it's more or less the standard at some point the image below this line is no longer loaded and its solid grey lines.

with images like this, would it be possible to have a 'corrupt dup' setting where you either point out where the image is good, or where the image is bad (a fill tool may deal with bad better then good making it easier to do) and have it discount that section from a dup search? my logic here is if I have a large enough archive, or the interests see me downloading similar images regularly, there is a good chance I have the un corrupt version somewhere else already

and just to head this off, these are images that have gone from a 120gb maxtor, to a 1.5tb seagate and survived 3 hdd failures there, to be moved to a 4tb and finally to am 8tb hdd, and finally to a currently 3tb image archive drive. odds are one of the drives fucked with the files in some way, or it was my horrendous method of acquisition all those years ago so the issue isnt my hdd corrupting images, its more I have corrupt images that may have to much corruption to be dup found




File: 1426721772716.png (100.78 KB, 1624x1081, 1624:1081, 1327614072601.png)

7f2c0e  No.471[Reply]


Drag and drop windows with tag rules. Show two windows side by side and one window can be programmed with the rule "ADD tag foo" and the other one has the rule "REMOVE tag foo, ADD tag bar" and you can drag and drop files to them.

Deriving tags from regex of other tags/namespace tags. A file has the tag "filename:big_ugly_name" and we could regex that namespace for another tag.

Tag sets with hotkeys: save a set of tags under a hotkey so it's quick to add them to a file while filtering

Opaque window behind tag list in the corner so it doesn't get hidden by picture background

Option to default certain mime types to be excluded from slideshow and only open externally, will help with videos with odd codecs that don't preview in the slideshow correctly

Option to specify hamming distance in "find similar images", you can't change the option once it's in the filter window and you have to enter the hash manually in the "system:similar to" option
656 posts and 192 image replies omitted. Click reply to view.

5d012e  No.12098

For the Pixiv downloader, would it be possible to have Pixiv pages start with page:0 instead of page:1 the same way pixiv labels its image pages? Its just for in the event that I or someone else may need to export pixiv images back to their original file name made easier using the regex. I know you could probably reduce the number using regex when exporting but it just seems like extra work and would probably get confusing.


5d012e  No.12099

>>12098

Actually, I forgot the hydrus downloaders also grab url's which seems like a better option for exporting images. Would be cool if you could export based on url's, maybe some options to trim them down so you can get the original file name.


e339aa  No.12100

So lately I've been manually tagging a number of related images. The fucking clear button has become the bane of my existence.

Want to add 5 or 6 of the most recently added tags? You accidentally clicked the clear button because it is literally one line above the most recently added tag and have to manually type them all out again.

Can we either move the clear button to the bottom of the window so you can't accidentally click something you only need on rare occasions, or add a confirmation dialog?


690b74  No.12151

Working suggestion: make GUI font a little more hackable. To do that, add to the ClientConstants.py:


def get_default_font():
return wx.SystemSettings.GetFont(
wx.SYS_DEFAULT_GUI_FONT)

And then run magic


sed -r -i 's/wx\.SystemSettings\.GetFont\( *wx\.SYS_DEFAULT_GUI_FONT *\)/CC.get_default_font()/g' *.py

and commit the changes.

This will make it easier to, both, add a setting for the fonts, and make it easier to patch the font in the code.


7d0642  No.12170

It would be useful for mp3 files to use their embedded album cover art as a thumbnail if one exists.




File: d49758fd3e9ccfe⋯.png (54.11 KB, 2105x826, 2105:826, logo.png)

4ab0be  No.11035[Reply]

I was thinking since Pixiv has such a weird, complicated, and sometimes awfully designed site, us Pixiv bros should help each other out on figuring out ways to make it easier to use with Hrdrus. Post any tips, scripts, regexes, setups, etc.

7 posts and 2 image replies omitted. Click reply to view.

323f16  No.11786

>>11059

>>11054

>>11041

Not just that, but "artists' favorite tags" and "tags' main users", from that we can create a network graph for better tag and artist discovery


4ab0be  No.11787

>>11760

Those are nice scripts but my main issue is I need to keep turning new Japaneses tags into English using sibling. It just seems like such a clutter to keep turning Japaneses tags into English when they are already translated in that >>11759 2nd pic, unless there's a better way to deal with this and I'm going something wrong.


16c720  No.12060

>>11759

sauce for the first image?


f3b3ee  No.12127

>>12060

>>11759

able to post all images visible? I like what im seeing


2781f9  No.12131




YouTube embed. Click thumbnail to play.

033f89  No.12021[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v345/Hydrus.Network.345.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v345/Hydrus.Network.345.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v345/Hydrus.Network.345.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v345/Hydrus.Network.345.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v345.tar.gz

I had a good week. The big thumbnail update is done, which means for an important update. Users with smaller databases have less to think about, but anyone with 1,000,000+ files should definitely read all of the following:

thumbnails

This is a great week to make a backup before you update!

Until now, the hydrus client has used two simple thumbnails–one 'full-size', one 'resized'–for each file. Hardware has moved on since then, and so has my code, so it is now feasible to have a single smarter thumbnail that will resize itself on demand. The work for this was done this week. This makes for a simpler and lessPost too long. Click here to view the full text.

21 posts and 4 image replies omitted. Click reply to view.

2e9983  No.12079

>no next release announcement thread

is hydrus dev okay?


2e9983  No.12080

forget it


90978c  No.12081

is it possible to add in a option for color managment just to point to an ICC Profile? Would help greatly for people using an ICC profile on Windows color managment or or programs like displaycal.


033f89  No.12082

>>12076

>>12077

>>12081

Hey, I apologise, this ICC profile tech is completely new to me. I will make a job to look into this, specifically how much control I have over it with wx or OpenCV/PIL, for next week.


033f89  No.12083

>>12079

>>12080

I took some extra time yesterday evening (and just now) to finish some bugs and cleaning up thumbnail code. Going to test it a bit more, so release might be delayed a little bit. I feel good about it though, so no chance of anything going wrong™.




File: 4156de3f899388a⋯.jpg (285.02 KB, 1402x1865, 1402:1865, 4156de3f899388aeb0cfbb298e….jpg)

3cab50  No.12078[Reply]

I had a great week. With the new simpler thumbnails, I improved how thumbnails load and added some new thumbnail maintenance actions. I also extended the 'manage urls' dialog to work for multiple files and got the very first simple version of OR search working and ready for user testing!

I also fixed the unusual 'include current tags' and white-frame animation bugs.

The release should be as normal tomorrow.



YouTube embed. Click thumbnail to play.

893c3e  No.11975[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v344/Hydrus.Network.344.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v344/Hydrus.Network.344.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v344/Hydrus.Network.344.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v344/Hydrus.Network.344.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v344.tar.gz

I had a mixed week, but I got some good hydrus work done. It is basically all misc this week.

highlights

The Client API v1.0 got its last polish this week. I fixed some bugs and added a new parameter to control page selection to the /add_urls/add_url command. Future Client API work will be entirely in regular weekly work. I'd like to add wildcard and namespace tag searching, more system predicates, autocomplete tag searches, optional https to encrypt communication outside a LAN, cookie.txt import, and I am sure many other things. The basic bones are decent now though–I now need only hang new things off it.

Animation scanbars (thePost too long. Click here to view the full text.

17 posts and 2 image replies omitted. Click reply to view.

893c3e  No.12013

>>12005

I have very little experience with p2p, and I am not sure what good python options I have available. I'd like to integrate any client-to-client talking into the client ui and have some sort of 'trust' network so users have more control over who they are talking to (whether that is 'I am happy to help share the bandwidth load, I am on 24/7 and will talk to anyone' vs 'I want to share my private collection of dbz kino with my two friends') rather than linking users together automatically.

In any case, I haven't thought about it seriously yet. The bandwidth crush is not a big deal for me yet, but it will be if I trigger any sort of update regeneration.

>>12006

The problem I am concerned for is more clientside. A full PTR sync now takes ~14GB total in local hard disk space. Since most users only ever use a small fraction of those tags, I'd like to cut it down for technical reasons like processing speed and also convenience.

The server db, which stores a bit more info like account_id and so on with each content row, is now 28GB. I don't mind that bloating up to 100-200GB in future as the collection grows, but I don't want to say 'hey new user, if you sync with the PTR, prepare your expensive SSD angus for 64GB of random shit tags you'll never see'.

For me, this is all ultimately a series of stopgaps for the next ten years or whatever until auto-tagging tech takes over the easy stuff. The 'every client gets everything the server has' idea of tag repos has worked overall really great, but it is not sustainable at the current pace. It could be sustainable as >>12004 suggests, making a certain tag repo devoted to a limited set of files and rules that will grow slower than new hard drives are, but my PTR's free-for-all is too unmanaged to last forever.

That said, it seems that training databases for neural nePost too long. Click here to view the full text.


893c3e  No.12014

>>12011

That might be slightly more tricky because I think an SVG is really an xml document, right? There isn't therefore a neat reliable file header, but I'll make a job to look into it. Adding 'document' handling to hydrus is a long-term objective, so extending initial file parsing to say 'ok, it wasn't a video, let's see if it loads as xml and if it has any good file metadata tags' will come one of these days in any case.

A quick search suggests there are a couple python svg libraries around, but I don't know how good they are. If there's a great one out there, I can add support for this much quicker (and add native rendering) as I can offload the difficult stuff to that.


500ba1  No.12017

>>12014

Yeah, svgs are basically XMLs with a specially defined namespace at the beginning. If you ever get to document/xml handling, namespaces are probably the way to go to differentiate them from say, html files.

The mozilla developer network has a surprisingly good primer on the stuff imo: https://developer.mozilla.org/en-US/docs/Web/SVG/Namespaces_Crash_Course

I'm not versed enough in python to recommend a library, but considering how often python is used in math and how often svg is used to make charts, I'd wager there's a great one you can probably use.


19e182  No.12018

>>12014

For initial parsing, have you considered libmagic from the "file" unix utility?

It doesn't do much to extract metadata, but if it's just a matter of finding the proper MIME type it should do a good job.

https://github.com/file/file/tree/master/python


c9c79e  No.12019

>>12013

>have some sort of 'trust' network so users have more control over who they are talking to

Wouldn't it be easier to start with some "all of this is shared" type bittorrent or such integration? There are good libs for that that are easy to use.

You can define what is shared, but you'd omit the "with who" part for now.

Sharing in a web of trust or with friends requires a good bit of UI work to make any sense, and security flaws will also upset people a lot more.




Delete Post [ ]
[]
Previous [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
| Catalog | Nerve Center | Cancer
[ / / / / / / / / / / / / / ] [ dir / bbg / choroy / dempart / doomer / flutter / vfur / vietnam / vril ]