[ / / / / / / / / / / / / / ] [ dir / abdl / animu / asmr / boers / cafechan / kc / leftpol / vg ]

# /hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.
Winner of the 32sd Attention-Hungry Games
/animu/ - Anime and Otaku Culture

Name Email Select/drop/paste files here * = required field [▶ Show post options & limits]Confused? See the FAQ.
Embed (replaces files and can be used instead) Do not bump(you can also write sage in the email field)Spoiler images(this replaces the thumbnails of your images with question marks) (For file and post deletion.) Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdfMax filesize is 16 MB.Max image dimensions are 15000 x 15000. You may upload 5 per post.

New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Current to-do list has: 1,316 items

YouTube embed. Click thumbnail to play.

windows

os x

linux

source

I had a busy week that was ok overall. I have made a variety of small fixes and changes, added a bunch of debug code, and improved the tag siblings workflow.

siblings

The manage tag siblings dialog has received a similar workflow improvement as tag parents recently did. Rather than spending 5 or 10 seconds loading a gigantic 22,000-row list, it now only shows the existing siblings that are relevanPost too long. Click here to view the full text.

b1aeca No.8420

>>8419

I should note this was potentially with version 298 and possibly with 299

my ability to track when this bluescreen is kind of shit due to a few ways my computer will crash and memory dumps not showing up/working it may have started with 297 possibly 298, but at the same time I have xbox and ps3/4 controller drivers installed fairly recently too, those are in a pseudo quarantine where I disabled everything involved

going to go through the uninstall process next if I don't get a blue screen in a month of uptime with those drivers in quarantine.

but if hydrus could cause this, potentially makes my life so much easier.

File: 1426721772716.png (100.78 KB, 1624x1081, 1624:1081, 1327614072601.png)

Drag and drop windows with tag rules. Show two windows side by side and one window can be programmed with the rule "ADD tag foo" and the other one has the rule "REMOVE tag foo, ADD tag bar" and you can drag and drop files to them.

Deriving tags from regex of other tags/namespace tags. A file has the tag "filename:big_ugly_name" and we could regex that namespace for another tag.

Tag sets with hotkeys: save a set of tags under a hotkey so it's quick to add them to a file while filtering

Opaque window behind tag list in the corner so it doesn't get hidden by picture background

Option to default certain mime types to be excluded from slideshow and only open externally, will help with videos with odd codecs that don't preview in the slideshow correctly

Option to specify hamming distance in "find similar images", you can't change the option once it's in the filter window and you have to enter the hash manually in the "system:similar to" option
546 posts and 184 image replies omitted. Click reply to view.

9e3e67 No.8331

When loading a saved session (especially when hydrus first starts up and loads the default session), the various pages should be refreshed.

My default session includes a "new" tab that is just a search for "system:inbox", but on startup it always loads with whichever files WERE in the inbox when I first saved the session instead of the ones that are actually in it now. I have to manually refresh the page each time.

9f5f90 No.8378

I wonder how hard it would be to add a basic tag definition/wiki. Just a simple way to write a sentence or couple of sentences that define your tags. Sometimes I have forgotten my exact methodology I use when tagging shit and would like to keep things straight.

I am manually tagging all my images over time, a colossal undertaking, I sometimes have a bit of tag drift as I maybe shoehorn in some images into a tag that are edge cases with that gradually becoming standard for that tag. It results in a tag with mixed meanings that I might rather split into two tags or something. I can manually check out various imageboard tag definitions, but I kinda use my own definitions based on what's relevant to me.

826c52 No.8384

>>471

Multi-resolution for Tumblr

^(https?:\/\/)\d+\.media\.tumblr\.com(\/[0-9a-f]{32}\/tumblr_(?:inline_)?[0-9A-Za-z]+_(?:r\d+_)?)\d+(\.[a-z]+)1s3.amazonaws.com/data.tumblr.com$2raw$3

ff56d0 No.8385

Suggestions for Hydrus currently:

-this one still >>7547

-show tags for files while browsing the duplicates page.

Suggestions for the future downloader because I don't know where to put them:

if anyone can do either of these things currently via a different piece of software I'd be glad to hear it.

1d9037 No.8423

Not to harp on this >>7547, but a new behavior I encountered makes me curious:

>-Open tab silently behind current focused window, so I can keep middle-clicking.

>-And/or highlight a list of tags and have each tag be opened in a new tab.

Currently, if I highlight multiple tags and hit "open new search page for selected" it'll open up several tabs (which I want!) then closes them as it merges the search.

File: ab23431121b8563⋯.jpg (96.69 KB, 492x650, 246:325, ab23431121b8563a7bc3b7ac57….jpg)

Gonna start another thread because >>173 is becoming too big for most people.

In the tag manager, whenever multiple tags are selected for multiple files with different tags, if you press the del key, the program will still ask you if you want to delete them all or put them all in the selected files. This does not make sense, if I'm pressing the del key it's obvious that I want them gone.

344 posts and 108 image replies omitted. Click reply to view.

8fb0cd No.8409

>>8392

This has to do with windows and the fonts it uses. the heart is double booked with the 3 line symbol. I think I got foobar to display the heart properly by changing the font to segoe ui or something

85f3dc No.8411

File: c94c6e4fee4a5c3⋯.png (12.68 KB, 859x204, 859:204, heart.png)

File: 1dfc5a2899b4779⋯.jpg (958.43 KB, 1920x1278, 320:213, 1dfc5a2899b4779a39764d9c0a….jpg)

>>8356

>>8355

>>8381

Yeah, you can't merge URLs yet–URLs were not a hydrus-recognised 'datatype' when I wrote the dupe filter. I will have to revisit it when I am on better footing with what URLs go where.

I will add support for adding URLs with the new gallery parsers as I roll out the new download engine. It should be totally doable to recognise a source URL on a gallery page and associate it additionally as you download the file.

>>8363

Thanks for the screenshot. That is pretty weird! I assume this is some trickery in your driver somewhere that isn't happy reinitialising 'stay on parent' windows like review services. Some force-relayout call isn't being propagated down.

I have written a new entry at help->debug->force a layout for all tlws now for today's release. Please give it a go sometime and let me know if it changes this issue at all. It will print a bunch of related size/pos info to the screen as well–a screenshot of that (or copy/paste from the logfile, which will have the same info) would be helpful.

>>8382

Thank you for this report. Can you name the tumblr that does this? Does it have any unusual images at all, either in ratio or size? It looks like somehow I am asking CV to resize to negative coordinates, which maybe suggests one of these images is reporting a bad size. I will write a neater catch for this for today's release.

Have you ever seen this error in a different context, or only on an actively importing tumblr downPost too long. Click here to view the full text.

8fb0cd No.8413

>>8382

>>8411

>Thank you for this report. Can you name the tumblr that does this? Does it have any unusual images at all, either in ratio or size? It looks like somehow I am asking CV to resize to negative coordinates, which maybe suggests one of these images is reporting a bad size. I will write a neater catch for this for today's release.

It was ligbi.tumblr.com but none of the images seemed unusual. It had about 3,800 files so I doubt you want to sit through a download of it yourself.

>Have you ever seen this error in a different context, or only on an actively importing tumblr downloader?

Never seen it before, but I haven't used the tumblr downloader in quite some time.

Really though, I was whipping through images and deleting them via the media viewer quite rapidly as new ones were incoming. I only got the error about 5-6 different times during that download and I believe it switched focus off the media viewer when it popped up.

38e810 No.8421

>>8348

>Thank you. Do you mean is can crash as you click thumbnail right-click menu->share-export-files?

yes

>If you boot the client fresh and do nothing but try to export some files, can you still get a crash that way?

also yes

>Approx how many files are we talking, btw? Is it 1-100, or like 10k+?

this happens even when I export a single file

6767a4 No.8422

File: 3648c236e844daf⋯.png (21.24 KB, 408x406, 204:203, 2018-03-22 15_15_15-hydrus….png)

>>8411

Summary of what I did, which is something I do basically daily. Additionally, this behavior is repeatable if my monitor is simply being woken up.

-Unplugged displayport from work laptop.

-Plugged it into the displayport-out of my primary monitor.

-Windows refreshed the driver with the expected black screen.

-Went to Hydrus, checked if I would see no popups.

-Confirmed it, went to debug, and got this:

File: 96339c942441720⋯.png (362.43 KB, 700x700, 1:1, confused_6.png)

Here you can ask questions so that the board is not clogged with small threads.

>>2231 has reached its bump limit, so I made a new thread.

564 posts and 113 image replies omitted. Click reply to view.

3f1d6b No.8393

>>8388

i'd like to know this too. sometimes i accidentally delete pics that i shouldn't have deleted

9dc0f5 No.8397

>>8393

open a new page and search for system:trash or smth like that.

9dc0f5 No.8399

>>8397

better yet, F9, up, left

441e0f No.8404

>>8388

If it's a search page you can just refresh, otherwise you'll have to find it and drag it onto the page.

>>6021

Is there a twitter downloader? I hate navigating that site and changing URLs to get the original file (which is still compressed because fuck you). It would be nice to set up a watcher for certain art accounts.

Is it possible to alter delay rules? For example, if I'm scraping Sankaku Channel, and I start getting 429 errors from the site, Hydrus abandons it's sync and sets a delay on that subscription for 24 hours or more. I'd rather that delay be set to 30 minutes or so since that's all it needs for the 429 errors to go away.

YouTube embed. Click thumbnail to play.

windows

os x

linux

source

I had a difficult week. A bunch of IRL stuff fell on me, but I got some bug fixes and new ui stuff in for hydrus.

mostly misc this week

The 'file notes' system from last week seems to have worked out well. I've now made it so the 'manage_file_notes' shortcut action should OK the dialog as well as open it, and I have moved the text caret to the end of the document on dialog open. It should now be pPost too long. Click here to view the full text.

16 posts and 3 image replies omitted. Click reply to view.

1b179b No.8406

File: ac3c6e9fe2c7441⋯.jpg (30 KB, 700x525, 4:3, 8e1f78c0052493dc952e362433….jpg)

>>8387

>>8391 and also the fact that chrome, along with virtually every modern browser, is bloatware that needs a big chunk of your RAM to work and the longer you leave it on, the more RAM it needs to run well.

>>8398

>datamining

>not relevant

Anon I…

cb19a3 No.8407

>>8405

>None of the hotkeys seem to work while the tags window is in focus.

you can scroll over the image window to change to the next window when you turn Settings -> Devices -> Mouse -> Scroll inactive (…) If I remeber correctly my old ubuntu and kubuntu installs did this by default

a4e9db No.8408

>>8407

Thanks but I use windows and I can't find a setting like that

7fa2c9 No.8414

File: d1cc3332fb6836a⋯.jpg (68.06 KB, 450x200, 9:4, d1cc3332fb6836a16447ae8c29….jpg)

File: 899bf175a4d64b8⋯.jpg (78.72 KB, 447x686, 447:686, 899bf175a4d64b8699f838d063….jpg)

>>8377

>>8367

I didn't update it for a few weeks, so the count piled up. I am typically running at +15 or so a week atm. There's a bunch of dupe and already-done items in there that I haven't cleared out. I am nonethless completely overwhelmed by it all, and I don't know the solution, but I will keep pushing at it anyway.

>>8368

Thanks. I've got some new ui in for today's release that'll better specific the cookies your machine has for pixiv–I would appreciate either getting an emailed (or discord DM'd) screenshot, or if you just post a screenshot here with the 'values' of the cookies blurred out (for privacy).

If it looks like your cookies are good, I'll figure out some way to dump the actual page data that is being downloaded and we'll examine it closer.

>>8369

Nothing too clever, I'm afraid. I only have time to neaten workflow for the existing system atm. I do have plans to make this a lot better, though.

>>8379

Thanks–that looks like an ui for same stuff I do. If you are getting into python and want a neat way to make exes, that looks like a great way to do it. I like PyInstaller a lot–it works very well for me, despite some unusual stuff I am doing, on all three platforms I put releases out on. The 'pyinstaller -y -w…' bit is the same part I have in my build scripts.

PROTIP: Don't use One File unless you are making a tiny program.

1b179b No.8415

File: 3428a73cc9d4af6⋯.gif (568.8 KB, 250x158, 125:79, 4jtsQNX.gif)

>>8414

YAMERO

YouTube embed. Click thumbnail to play.

windows

os x

linux

source

I had a great week. I fixed some things and added a bunch of new ui stuff, including a basic new 'file notes' system.

Several users have reported that their subscriptions were receiving DEAD status despite seeming to be healthy. Moreover, attempts to revive their dead subs were not working. I have put some time into improving the logic here all around and I Post too long. Click here to view the full text.

15 posts and 5 image replies omitted. Click reply to view.

df34ce No.8353

got a question on the pixiv downloader, does this pick up every image?

I ask because I have no fucking idea how that retarded website works, japanese internet always seems so unintuitive and backwards in how it functions, but I have been noticing that its not downloading things Im pretty sure are there.

3bb0cc No.8354

>>8353

780ecf No.8359

>alpha file notes implemented

I deeply respect you, chief.

Thanks for the feature, will try it as soon as I can.

c30ff8 No.8360

>>8350

This works, kinda a pain with tagging and sorting the images but at least it works. Thanks

0969d6 No.8412

File: 7f8182bee17e313⋯.jpg (524.36 KB, 2257x1645, 2257:1645, 7f8182bee17e31361c9647ebb7….jpg)

>>8350

>>8349

>>8360

Yeah, this an issue with the 8chan API not providing direct links to files, so files that use the old storage system aren't pointed to correctly. Page of images works ok for now (doesn't do non-images right, though), but I will have other, proper fixes for this stuff out as I roll out more of the downloader overhaul.

>>8353

Tell me about it–every time I get into it, I find nip software and file format decisions are reliably bonkers.

Afaik, my pixiv parser does still images ok. Manga (multi-image) pages don't work, ugoira doesn't work, but I expect to have a fix for both in the new downloader engine.

Pixiv also have some mickey-mouse new layout that they are intermittently rolling out that uses dynamic element loading (i.e. the page is just a blank canvas, and then some javascript makes an API call for thumb/whatever info and draws the page with your CPU cycles), which is what broke the tag search. Also, they had some 'whiteapple' or something new layout that I think was eventually abandoned, but it was a site-wide thing that would sometimes kick in for users randomly as a beta. They do some page-referral shit as well that blocks image loading if you don't say you are coming from the post URL.

Once I have better parsing solutions in place, I would like to look up if they have an API, or if I can just cheat their unofficial internal API calls myself and skip all this shit.

File: 609e28f57f9de74⋯.gif (385.35 KB, 375x277, 375:277, 609e28f57f9de74ac349a9ab2c….gif)

I had a busy but ok week. I improved a bunch of error handling and debug, fixed and added some small stuff, prototyped some tag sibling workflow improvements, and wrote some prototype session/cookie review and edit ui for the new network engine.

The release may be late tomorrow. I need to spend some time catching up on messages (and any related urgent bugs) that have piled up.

File: 442eb99a34c2ff7⋯.jpg (120.39 KB, 379x834, 379:834, 442eb99a34c2ff7c026c477316….jpg)

ITT post/discuss/jerk off to/improve scripts to use with hydrus

One of my scripts was broken as shit and I just fixed, but I figured it was better to graduate from the Q&A thread and make it easier to find.

e65484 No.8400

File: 8b83d3d31991e3f⋯.jpg (101.06 KB, 379x834, 379:834, 8b83d3d31991e3f2d78458a9f1….jpg)

Here's the MakeNew script I just fixed.

It takes every line from a file named s.txt, compares each of those lines with the lines of old.txt(if it exists) and spits out a file named new.txt with the new lines. All the new lines will then be appended to old.txt.

It's mean to be used along with my md5 script from the Q&A thread(I want to mess around with it a bit so I'll post it here later) which creates a file named s.txt with the md5s of all files in your hydrus folder, so you can easily perform a md5 search to get tags for them. You should use that script first to generate the s.txt file, then you either rename that s.txt to old.txt(if it's the first time you're running the script) or run this one(if you already have a old.txt) to get all the fresh new hashes on new.txt while also saving them on old.txt.

The scripts are independent, so you can name them whatever(as long as the extension is ".py"), I just call it MakeNew since that's what I named mine, since it's pretty much what it does.

Sometimes I make seemingly retarded stuff, so feel free to ask me stuff if there's anything you don't understand about the script.

oldFile = open("old.txt", 'r')line = ""write = Falsefor line in oldFile:	continueif ((not(line[-2::] == "\n")) and (len(line) > 0)):	print("line:" + line)	print(len(line))	write = True	oldFile.close()write = FalsereadFile = open("s.txt", 'r')newFile = open("new.txt", 'w')for tempReadLine in readFile:		for readLine in tempReadLine.split('\n'):		if(len(readLine) < 1):			continue					newLine = True				oldFile = open("old.txt", 'r')		for tempOldLine in oldFile:			for oldLine in tempOldLine.split('\n'):				if(len(oldLine) < 1):					continue									if oldLine == readLine:					newLine = False				

e65484 No.8401

>>8400

>Sometimes I make seemingly retarded stuff

>

if ((not(line[-2::] == "\n")

And also actually retarded stuff sometimes.

File: 6b8429746cd3d3a⋯.png (76.84 KB, 1920x1080, 16:9, 001.png)

So I tried to imagine what would Hydrus might look like after after a few hundreds of implemented to-do list items. I consider it to be too presumptuous to spam with these in suggestons thread so I made separate one. It's more like referential pics for dev's inspiration.

77 posts and 77 image replies omitted. Click reply to view.

b7a7bc No.8358

>>8234

>>8235

This!

I suggested this many months, if not years, ago. Every image that is a slight variation, or that you want to be grouped, should be able to be grouped together.

For pages, etc. you can always give a tag and you're good to go, but for similar images, or images that have ALL the same tags but you want a few of them grouped for different reasons, this is a must and I believe chief will implement it in a year or two.

fb7d05 No.8372

>>8347

Thank you so much for your response! Indeed those .ai files were pdf-compatible and could be opened in 3rd party pdf readers, so this mistake is understandable. It's one of my biggest pains—there's simply NO 3rd party software capable of generating thumbnails for .ai correctly (even if claims so). Even Adobe's proprietary media browser can't recognize some ancient .ai's made in early 00's. What a bunch of obscurity exploiters!

I'll be patiently waiting till you put your hands on translation issues. Just bump this thread—I'll be here to help.

fb7d05 No.8373

And by the way, will you put russian version for help section on your site if I translate it?

9afa5a No.8394

1. Image archive clubs, with dedicated settings so that each member downloads different items for IPFS.

2. File extension to PDF, EPUB, MOBI and DJVU for >>>/pdfs/ and >>>/tdt/

3. Perfecting the tag translation and mapping system for compatibility between boorus, dA and pixiv.

4. Heavy-handed image deduplication and searching

https://pippy360.github.io/transformationInvariantImageSearch/

https://github.com/ascribe/image-match

https://github.com/dlau/mineye

5. Video, Fan-fction and Webcomic subscription/archiving (beause why not?)

9afa5a No.8395

Commonly used/favorite tag heat-maps

Automatic scheduled Booru scraping of preset parameter

tag-tag relationship diagram/cloud/cluster

File: b736f45439a883d⋯.gif (20.08 KB, 177x160, 177:160, b736f45439a883d3ddc83b246d….gif)

I had a difficult week. A bunch of IRL stuff fell on me, limiting my hydrus time, but I did fix some bugs and add some new ui stuff.

The release should be as normal tomorrow.

c24046 No.8352

yay

YouTube embed. Click thumbnail to play.

windows

os x

linux

source

I had a great week. You can now edit the text on thumbnails, and there are some new file import options.

file import options

I have moved the 'allow decompression bombs' option from the global options dialog down to regular file import options. So, if you encounter a suspected decompression bomb that you would still like to import, you can now much more quickly just set the allow state on that quePost too long. Click here to view the full text.

16 posts and 10 image replies omitted. Click reply to view.

711a01 No.8287

Just had a crash whilst using the duplicates filter, I just had my import folder import two new webms, and when I first updated (via the cog icon) one duplicates page, and after that a second one which was not in a page of pages but in the root page. just after pressing the update menu-item, and when the play icon on the search for duplicates had turned white and I pressed it, and then it searched. After completing the search it crashed, it did not yet show any new potential pairs, but when booting it again I now do see the four new potential pairs. I also have some thread watchers running in the background on 8chan and 4chan pages.

572944 No.8291

Hey dev, I just had a thought. I have been going through some of the image options, there are some options that would make images look better as they zoomed in. while this is good for just browsing, would there be a way to have a seperate 'not adding anything to make this shit look better' option for the duplicate filter? I don't have time to really look into it and compare images, and it would be a bit difficult to do, but something tells me stripping anything that makes the images look better would be helpful for finding out the differences between them.

17e978 No.8292

For files downloaded from HentaiFoundry with a HF "known url", before when you viewed them in the media viewer it said "Hentai Foundry url" (or similar) in the upper right corner of the screen. But since a couple of versions back it doesn't say anything there anymore.

I'm guessing maybe the urls don't match the "hentai foundry file page" url class? Here's a typical url from one of my subscriptions "http://www.hentai-foundry.com/pictures/user/Sparrow/579179/Assuka-Sucks". But the only difference I can see is the url class uses https.

That little line of text was useful when using the dupe filter…

cca384 No.8297

>>8257

>8183 again

another thought would be being able to paste them in more than 1 at a time–vim/emacs macros would make the repetition bearable, and I'm probably not alone in pre-staging them there to begin with so I don't forget any.

60fb9b No.8299

>>8262

I've done a bit for this in today's v297 release. You can now save and spam preview size settings a bit easier–let me know how it works out and if you would like more fine control of specific pixel values or whatever. Setting multiple options might be easily doable–I'll check it next week.

In working on this this week, I was also surprised that hidden pages still do re-layouts on parent window size events. Unfortunately, I don't control that stuff at my level, but it better explains why having 100 windows open can make ui resizes laggy (especially a few weeks ago, when file and tag import options were embedded on every page). I will consider this in some other ways going forward.

>>8263

Thanks for these examples–I expect to write a generic html imageboard parser in the coming weeks for rarer/bunker chans, so I will see if I can wrap support for these into that and write some URL Classes to capture them for the thread watcher as well. I'll roll it all into a normal update, so it should just start working one week. Let me know how they work for you as I roll this out.

>>8279

>>8285

Not easily yet, but the ongoing downloader overhaul will expand our options greatly here. It'll be easy to add 'rating:safe'-type tags and I think we should be able to do known url parsing as well.

>>8287

Thank you for this report. I happen to have fixed a duplicate filter crash for today's release (as well as a fix for the page not refreshing after a dupe files maintenance job is done). PleaPost too long. Click here to view the full text.

File: fb7c2639048ed42⋯.gif (628.93 KB, 700x630, 10:9, fb7c2639048ed42968521fb59d….gif)

I had a great week. I fixed some bugs (including, I hope, the DEAD subscriptions some users have seen), improved the new tag summary generators and several other ui controls, and added a simple prototype 'file notes' system.

I expect the release will be two or three hours late, as there is a lot to test and I also want to spend some time catching up on messages.

File: c7ab9e75b8ba6a6⋯.png (735.97 KB, 1200x1880, 30:47, decboo.png)

Hello Hydrus Network community.

I am a programmer familiar with booru development. I have written my own private booru software that scrapes public websites for images much like Hyrdrus Network, it currently has 10 million files, but this isn't what I am here to talk about.

I am very interested in Hydrus Network's goals of decentralized zero-authority sharing and collaborative organization of images as developments towards these goals benefits my own booru.

I have recently created a proof of concept decentralized booru called "Decboo", it uses IPFS for image distribution and the Ethereum blockchain for tag distribution.

Decboo is not the booru I spoke of earlier, it is an extremely basic program made only to demonstrate blockchain-based tag distribution, functionality that I hope the Hydrus Network developer(s) will integrate into the Hydrus Network software.

2 posts omitted. Click reply to view.

dc0b90 No.8120

>>7858

>the ethereum blockchain for tag distribution

how the fuck did you make a cryptocurrency distribute tags for images of taiwanese tapestries?

1f6b69 No.8122

>>7883

Most things cryptocurrency enthusiasts do with blockchains are not actually a good idea. They somehow generalize from "blockchains are the decentralized trustless cryptographically secure ledger" into "blockchains are the first decentralized thing" or "blockchains are the first ledger" or sometimes even "blockchains are the first cryptographically secure thing, satoshi nakamoto invented cryptography". Then they try to use them for things that already have long-standing solutions. Trying to get businesses or governments to store databases in centrally controlled blockchains when the only measure blockchains don't lose horribly to traditional databases on is decentralization, trying to cryptographically verify things by putting them in a blockchain instead of just using public-key encryption, etc…

Blockchains make a lot of sacrifices in terms of efficiency and speed to achieve that decentralized trustless shit. (Trustless here meaning it's infeasably hard for an attacker to send a fake currency transaction to defraud people, or in this case infeasably hard to… add fake tags to the tag list?) As you mentioned, 39-year-old Merkle trees (and thus Git, which uses them) can do everything in that list except "trustless", and do it with one billionth the electricity and hardware cost.

Blockchains are just not very useful for anything other than libertarian internet money. For any other task, there's a better tool (though the old tool is probably not currently as trendy or fashionable).

dc0b90 No.8123

File: 08c5d3c6ed7b354⋯.gif (647.26 KB, 500x289, 500:289, sensible chuckle.gif)

>>8122

>satoshi nakamoto invented cryptography

e0e4aa No.8147

I think a blockchain is probably overkill, especially since in the long run blockchains tend to grow very large and unwieldy. The bitcoin blockchain is currently more than 100GB.

While technically blockchains provide the strongest guarantee of decentralized trust, they are only necessary when you're expecting the network to be attacked by technically skilled people with lots of time and deep pockets. In practice the worst kind of attack a network like this could expect would come from particularly autistic shitposters with relatively little disposable income. Even things like the bittorrent network still function fine despite being targeted by the likes of Hollywood.

Proof-of-work algorithms are a good idea though. They can be used to increase the computational cost of attacks and prevent a malicious entity from doing damage faster than a few humans can undo it.

If I was designing something like this I'd require proof-of-work both for node ID generation (to prevent sybil attacks) and for any edits made to the image/tag database. Let leaf nodes query the network for free (within certain bandwidth limits), but make them generate a computationally intensive ID if they want to push any changes to it. Each node should adjust its trust in its neighbors depending on how often they agree with the consensus, blacklisting any nodes which consistently give bad responses.

5996db No.8274

>>7858

IPFS based image sharing is awesome

YouTube embed. Click thumbnail to play.

windows

os x

linux

source

I had a great week. I mostly fixed and dejanked stuff. Also, the thread watcher now supports 420chan!

fixes and cleanup

The RuntimeError popups some users were seeing after a restore-from-minimise events are completely fixed. Some related search page and tag autocomplete draw and timer code is also cleaned up, and a memory leak was fixed along the way.

34 posts and 9 image replies omitted. Click reply to view.

f433a4 No.8161

Dunno if it will be fixed in the new release that will come up in a few hours, but after Corrupt EXIF data it stops downloading altogether.

Causing the the "Waiting for subs to finish".

2018/02/28 19:03:50: /media/hydrusnetwork/PIL/TiffImagePlugin.py:742: UserWarning: Corrupt EXIF data. Expecting to read 2 bytes but only got 0.

File: 1887dc6aeb7b111⋯.jpg (102.13 KB, 800x1150, 16:23, 1887dc6aeb7b11145438720e9b….jpg)

>>8145

Thank you. I ran out of time this week, but I have recorded this all and will look at it closer hopefully next week.

The tracebacks you saw should be completely fixed in today's release.

>>8146

>>8150

>>8149

>>8151

>>8152

Thank you for these reports. I think maybe some unusual error states are pausing the whole system or otherwise setting some strange 'oh fugg, don't do any subs for a bit, I think the network is down' flag somewhere and then isn't recovering from that correctly. I will look at this soon. Please let me know if the situation improves or otherwise changes in the coming weeks.

>>8155

Thanks. You are absolutely right on the sqlite front. I could be convinced that a different db engine worked better for some of our needs and also had better python support, but the work to change it atm would be far more than just improving the current system. And since everyone is moving to SSDs for their system drive, I think making it easier to have a hydrus client with db on the fast drive and files on the slow drive is probably a good plan that'll ameliorate this anyway. My decent laptop (on an SSD) processes so fast I don't even think about it any more, it is like 30k rows/s or something. About 10s a day to keepPost too long. Click here to view the full text.

198b19 No.8164

File: 63ad6d5a880e02a⋯.png (42.61 KB, 1152x628, 288:157, Untitled.png)

File: 796af784cfb1cbf⋯.jpg (74.66 KB, 1050x1523, 1050:1523, 796af784cfb1cbfa4dae284c82….jpg)

>>8162

>If you click the cog icon on the download page and say 'get tags for files where url known and already in db (this downloader)' and try again, does it work then?

Yeah, that works. Thanks.

>Maybe it should be in 'tag import options'?

I think that be better. It's one of the two places I first looked for that option in, the second one being in the settings menus.

I didn't even notice the cog in there, though I am fucking blind to things in front of me.

>what ui improvements would you recommend that I could make so you would have spotted this the first time around?

Aside from putting it into the tag import options, the placement of the cog would help. You have these kind of clusters of buttons (red lines), which makes it cover a large area, so your eyes wander around there and you end up easily finding the buttons. That cog, however, is kind of in a desolate spot with nothing around it in it's subsection.

Putting it near something wide(e,g, the "stop after this many files" thing, like in the pic) would make it more visible, I think.

f433a4 No.8167

>>8162

Regarding the pausing due to an error. I stated it in >>8161, if the client gets "UserWarning: Corrupt EXIF data." while doing subs, it will freeze the sub, prevent the client from closing, and after a forcefull shutdown set the "paus subscription syncronisation" to true.

>>8164

👌﻿

>>8161

>>8167

Thank you–sorry, I missed this the first time around. I will check this out this week. Can you give me an example sub search or a specific page that will give a bad file like this?

File: ca41f7a80270ccc⋯.png (41.82 KB, 377x515, 377:515, Capture.PNG)

I've been looking around for a while but haven't found an answer; is it possible to import Windows tags into Hydrus?

Failing that, does anybody know a good way to export that data to a *.jpg.txt file so I can use that feature? Futzed around with ExifTool but it didn't look like it could do what I wanted.

I thought I was a genius when I started tagging my files ~7 years ago but now it's become an immense pain. This program is like a dream to me but I can't imagine starting over.

c1213c No.8156

>>8153

Maybe this helps you:

https://github.com/Dijji/FileMeta

If it works, you can export the metadata to XML. Of course then you still need to find an editor or script something to grab the right fields to .txt, but generally that's not terribly hard with all the pretty good modern parsers.

5aa1d9 No.8259

>>8153

>>8156

Once the new parser is done, I expect to add some kind of support for 'neighbouring html/xml/json' files or something, so hang on to everything and let me know how you get on here. I am breddy sure we can fix your problem one way or another if you can find a program to extract the tags to something simple we can parse. If it comes to it, I can knock up a one-time script to convert from xml to Hydrus Tag Archive or something (but I've no experience with NTFS tags, so I just can't help much there).

Delete Post [ ]
[]
Previous [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
| Catalog | Nerve Center | Cancer
[ / / / / / / / / / / / / / ] [ dir / abdl / animu / asmr / boers / cafechan / kc / leftpol / vg ]