Version 194

Name
Email
Subject
Comment *
File	Select/drop/paste files here
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field) Spoiler images (this replaces the thumbnails of your images with question marks)
Password	(For file and post deletion.)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf Max filesize is 8 MB. Max image dimensions are 10000 x 10000. You may upload 5 per post.

YouTube embed. Click thumbnail to play.

Version 194 hydrus_dev ## Board Owner 02/24/16 (Wed) 22:52:59 No.2103

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v194/Hydrus.Network.194.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v194/Hydrus.Network.194.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v194/Hydrus.Network.194.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v194/Hydrus.Network.194.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v194/Hydrus.Network.194.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v194.tar.gz

I had a good week. I mostly worked on IPFS, and successfully got pin and unpin working, and otherwise did a bit of cleaning up and added a new way to add tags on import.

ipfs pin/unpin

Now that I know this all works, I will write up a help page for IPFS, including what it even is and where to get it, for next week, so if you don't understand any of this, please hang on a little bit.

If you have an IPFS service added, right-clicking on thumbnails will give you the option to remote services->pin to ipfs. The workflow works just like hydrus repositories, so your 'to be pinned' files will gain a little icon and the pending menu will count them. Committing from the pending menu will upload them to your IPFS daemon when you are ready. Files that are pinned to IPFS will retain a small correct-colour IPFS icon. Unpinning works similarly.

Furthermore, right-clicking on pinned files will let you copy their IPFS multihash, so you can share them wherever you like. Here's some I did this week:

QmP6BNvWfkNf74bY3q1ohtDZ9gAmss4LAjuFhqpDPQNm1S

QmXH3NYyqyAnNgf9gWppufJp3kCHwJcJeqk2w3d9zQQF9u

QmaQNZLHQg3tXT2R3gCSKPDJ4Bs9FjmWnHbmNQbVS157ge

You can try to download and import those through the client, or you can check through the browser-compatible gateway the IPFS people run at:

https://ipfs.io/ipfs/[multihash]

I haven't set my dev machine's IPFS daemon to run all the time, but I presume the file will live on despite that, somewhat, precisely because of how IPFS works. I have read all three files through that gateway, so I think they get temporarily mirrored or something. Let me know if you can get them, and let me know how your own IPFS-hydrus stuff goes!

parse tags from neighbouring .txt files

For a while now, when you export files from the thumbnail right-click menu via share->export->files, you have been able to export the files' tags into neighbouring [exported_filename].txt files, as newline-separated lists. I have now added a checkbox to the import files 'path tagging' dialog (the one with all the regex stuff), that will try to do the reverse, loading any existing neighbouring .txt files and parsing them for newline-separated tags.

If you have been looking for a way to automate tag import from an unusual source with a non-python script, have a play with this!

full list

- ipfs pins and unpins can now be queued up like file repository pending and petitioned, through the regular thumbnail right-click menu, which also reports some/all ipfs pinned selection status

- this ipfs action queue is similarly summarised and commited at the normal service 'pending' menu

- ipfs's 'pinned', 'to pin', and 'to unpin' statuses are displayed on thumbnails with ipfs-specific icons

- you can copy the focussed file's ipfs multihash or all the selected files' ipfs multihashes from the thumbnail menu's share->copy->ipfs multihash

- added a .txt tag parser to the 'path tagging' import dialog–it will parse the same sort of txt files the export dialog produces

- the client's new 'requests' network code is harmonised, generally improved, and now produces hydrus-compatible exceptions

- updated help re the local server and boorus now defaulting to off

- db can now remember service-specific filenames (e.g. ipfs multihashes)

- cleaned up some overly complicated and confused thumbnail menu code

- the pending menu now specifies what it is about to do more plainly

next week

I want to support IPFS directory sharing, so you'll be able to post 'Here's my 400-file blah collection: [single_multihash]' to a thread, and also be able to efficiently download such links using hydrus. It needs a bit more research on my end and thinking about what workflows would be best and then experimenting with the IPFS API to figure out what is actually doable. That won't happen in one week, because as well as the help, I also want to add 'system:pinned to ipfs' search support and a better review services panel for IPFS.

I might also write some class outlines for the big client.db extraction.

Anonymous 02/25/16 (Thu) 01:49:57 No.2106

So I finally started playing with ipfs and huh, it's cooler than I thought. Can't wait for sharing the 400 files under one multihash through hydrus!

bug report though: for local files, my hydrus server no longer appears under

right click > remote services

So I cannot upload pictures to a hydrus server now.

Anonymous 02/25/16 (Thu) 01:57:37 No.2107

>>2106

also ipfs isn't under system:file service so I cannot easily list what files I have pinned!

Anonymous 02/25/16 (Thu) 04:51:18 No.2108

Might I suggest a feature where hydrus calculates the ipfs multihash of every file in the database? Or alternatively, uses ipfs multihash in the databse itself rather than whatever hash you use now?

I'm the guy who had a bunch of files disappear, but the hashes are still in hydrus' database file, it's just the files that are missing. If they were ipfs hashes, you could implement a "try to recover missing files from ipfs".

Anonymous 02/25/16 (Thu) 12:34:25 No.2109

>>2103

Hey, hate to bother you with my request again, especially with how busy you seem to have been lately, but have you had any time to look into adding a source field?

I understand there are a lot more important things that should be done first, but it'd be nice to know it hasn't been forgotten.

Anonymous 02/25/16 (Thu) 12:54:47 No.2110

can I ask for a option to toggle the ipfs multihash to be copied from the program with the generic gateway url prepended to it?

makes it easier on my side

also, a system tag to sort items that are pinned to a ipfs service, I assume system:file service is one that could be used

https://gateway.ipfs.io/ipfs/QmchWtr91Ht88MPDWvXUrUEX5jMsdxoK9T9oEjxTgM1pcT

Anonymous 02/25/16 (Thu) 15:34:20 No.2111

File: 1456414460748.png (100.7 KB, 1366x768, 683:384, Hydrus.png)

Hooray for false positives -_-;

Just curious, how hydrus doesn't have an internal update checker?

Anonymous 02/25/16 (Thu) 17:55:54 No.2112

I was a bit skeptical about ipfs but it looks kinda cool. Makes it easy to share stuff, certainly.

Also, how far back in the roadmap are the subscription system improvements that were discussed?

Anonymous 02/25/16 (Thu) 22:35:53 No.2113

Great work as usual! Looking forward to directory sharing.

For anyone wondering about IPFS, I found this article very useful for conceptually understanding how IPFS works.

https://medium.com/@ConsenSys/an-introduction-to-ipfs-9bba4860abd0

Anonymous 02/26/16 (Fri) 02:54:24 No.2114

>>2111

>Just curious, how hydrus doesn't have an internal update checker?

I believe it was because dev wanted the user not to be inclined to update if they didn't want to. the only reason people would really want to update outside feature scope would be due to a change in the networking, which isn't often

currently we're on network version 17, I believe last time we changed it was when we changed hashes/how they're transferred

Anonymous 02/26/16 (Fri) 03:21:46 No.2115

>- added a .txt tag parser to the 'path tagging' import dialog–it will parse the same sort of txt files the export dialog produces

I could suck your digital dick right now.

Anonymous 02/26/16 (Fri) 13:25:44 No.2116

Hi dev,

so, I've downloaded the PTR. Once it finished, the final db file was 4gb in size, and now everything is ridiculously unstable, sometimes it takes up to 3 minutes to initialize, and if try to look around it usually freezes for a while or outright crashes. Is there a way to purge the repository from my db file?

Anonymous 02/27/16 (Sat) 02:30:28 No.2120

File: 1456540228297.png (885.02 KB, 1100x551, 1100:551, e6747ae36716c89829961ddb32….png)

>>2106

Can confirm that regular file repos don't appear under remote services, only when attempting to upload though removing files seems to work fine.

>>2116

Are you synced with hydrus_dev's actual tag repo or did you import the tag archive into local tags, if the former you can go to services->manage services->remote->tag repositories and remove the service there.

Anonymous 02/27/16 (Sat) 03:37:14 No.2121

>>2111

Use an external package manager. No need to burden the dev with issues that already have solutions:

https://chocolatey.org/packages/hydrus-network

Anonymous 02/27/16 (Sat) 06:15:09 No.2125

>update hydrus

>now refuses to open

client.exe pops up in taskmanager and then closes after 2-3 seconds

Should I not have used the installer version to update?

Anonymous 02/27/16 (Sat) 06:48:04 No.2126

File: 1456555684466.png (33.75 KB, 757x448, 757:448, hydrus.png)

>>2125

Tried reinstalling, told me a previous uninstall wasn't complete (or something to that effect) and I had to retstart

>restart

>reinstall

works fine on initial loadup

>notice antivirus has blocked some hydrus files (pic related)

Set those to allow, restore those files, turn antivirus off just so it can't interfere further

>still fails to load

I do have a crash log though

Anonymous 02/27/16 (Sat) 10:33:25 No.2128

File: 1456569205548.png (311.86 KB, 409x322, 409:322, 1402600448994.png)

>>2108

Chiming in

IPFS allows for hashing only without adding the file content to the datastore with the command

ipfs add -n *files*

I'm assuming there's a way to do the same in the API.

>If they were ipfs hashes, you could implement a "try to recover missing files from ipfs".

This self healing nature is something that excites me about IPFS and its integration with hydrus.

I would also love to see something like that so that files could be gathered using IPFS even if you never had them.

For instance if you add a tag repository. search for all images containing *tag* and don't have a remote file repo that has them, if you had the IPFS hash it wouldn't matter, you'd just pull it from whoever has it, hydrus user or not, no need for a specific file repo to pull the file from.

The thumbnails might be a problem though, I guess since you have the full image though you could just generate it.

>>2110

This isn't really related to what you said but these addons were very helpful for me and dealing with bare hashes

From /tech/:

This script changes all bare ipfs hashes into clickable links to the ipfs gateway (on page loads)

https://greasyfork.org/en/scripts/14837-ipfs-hash-linker

These redirect all gateway links to your local daemon when it's on, it works well with the previous script.

https://addons.mozilla.org/en-US/firefox/addon/ipfs-gateway-redirect/

https://chrome.google.com/webstore/detail/ipfs-gateway-redirect/gifgeigleclkondjnmijdajabbhmoepo

>>2113

This one is also good imo

https://blog.neocities.org/its-time-for-the-permanent-web.html

>>2121

I really hope MS develops OneGet further and pushes developers to use it. Related thread >>1511

Anonymous 02/27/16 (Sat) 10:37:56 No.2129

>>2110

>>2128

I should mention I'd like a custom prefix option as well though, I didn't mean to imply one is not needed.

Allowing the user to set a custom one should cover all bases, that way you could copy links to your local gateway, a private one, the public one, one on non-standard ports, whatever. Having the gateway is easy for linking it out to people externally who don't use IPFS.

Anonymous 02/27/16 (Sat) 21:20:32 No.2132

>>2120

Thanks, tried to do that before, actually, but I just thought the program had crashed or something. But then I opened the task manager and there it was, consuming 1,3gb of ram, half of the processor and the hard drive at max speed. It's been 7 hours now and just went through 1/4 of the total journal.

I guess I'll just have to leave the computer running for a day, since I believe the database journal gets scrapped if it doesn't finish.

Anonymous 02/27/16 (Sat) 21:32:41 No.2133

Just updated and I noticed the installer didn't ask about the path like usual. Did I miss an option box or something, or was this intended?

It detected the right path and all, and I absolutely recognize that I'm gonna be in the minority here, but I'd like it if a tick box or something was added to let me specify where to install.

The reason being that I've toyed with the idea of having 2 separate hydruses installed. One for images in general, and one for doujinshi.

It's not a big deal at all, and I realize I can just download the zip file instead, but I also figure this wouldn't be a lot of work to add in.

hydrus_dev ## Board Owner 02/27/16 (Sat) 22:13:10 No.2135

File: 1456611190256.jpg (1.03 MB, 1818x2424, 3:4, 86605ec0e80acd30e2e84e2bd2….jpg)

>>2106

>>2120

Thank you for this report. Sorry for the inconvenience, it is fixed for next week!

>>2107

>>2110

I hope to add ipfs services to 'system:file service' this week.

>>2108

I had thought this technically difficult, since IPFS hashes are not trivial to figure out, but if I can quickly ask the IPFS daemon for hashes for files as >>2128 suggests, then I might be able to figure out some sort of sync. Unfortunately for your situation, however, you need the file content (i.e. the file itself) to figure out its hash. There's no way to calculate an IPFS hash if you only have the hydrus hash.

>>2109

I'll write something simple for this week. Something like a right-click menu option for 'show all known urls for this file' or something. That information is remembered if you download, so I'll figure out some ugly access for you, and you can let me know how you would like it beautified.

>>2110

Adding a prefix is a great idea, thank you. I'll add something to manage services.

>>2111

Someone recently emailed me with a similar report with the same anti-virus software, maybe that was you. I don't know what's going on there, but I think those are old files anyway, as the latest version doesn't include them. If that warning keeps coming up, you can do a clean install (delete everything except the 'db' folder, which contains all your files and settings, and reinstall), and you shouldn't see that again.

There's no auto-update because I never got around to writing one! I'm not opposed, particularly, but it just isn't high on my list.

>>2112

I think that will be a big job, so it will be on the next 'what to work on next' poll. If that's what you care about most when that comes around, please vote!

>>2113

That's a great article, thanks. I am writing the ipfs help now, so I'll make sure to include it.

hydrus_dev ## Board Owner 02/27/16 (Sat) 22:57:36 No.2136

File: 1456613856694.jpg (68 KB, 602x960, 301:480, e1dcc095fde004c748767f3ea8….jpg)

>>2116

>>2120

>>2132

Unfortunately, you have hit the problem I want to fix with the client.db breakup. Having everything wrapped into one 4GB blob is too clunky for a bunch of reasons. Running regular vacuums tends to reduce a lot of the lag, but that in itself is a heavy operation that needs a bunch of disk space and can take twenty minutes.

Deleting a service is how >>2120 says, but for now you'll just have to wait. Let me know if it crashes or something, and I can walk you through a manual process.

When everything is broken up, it'll be much quicker, because services will be on separate files in a subfolder, so the client will just have to delete the folder, and it'll be gone.

>>2125

>>2126

When you say crash log, do you mean the crash.log at hydrus's install_dir/crash.log? If so, please post its contents, or email it to me if there is private info in it.

>>2128

Thank you for this information. Doing hydrus-ipfs syncs and putting file repo/repo updates on ipfs is something I won't be working on in this initial phase, but it is something I would like to do, and it is nice to know that IPFS can generate hashes without me having to spam pin/unpin.

Although hydrus uses sha256 right now, and the key tables and network stuff all work off that, I think I'll ultimately move towards arbitrary hashtype, so repos can talk only in IPFS if the admin wants, or sha512 or whatever, and the local client can cross-reference that stuff as cleverly and invisibly as it can.

>>2133

I updated my dev machine recently, which included updating all my dev software, including InnoSetup, the installer I use. I noticed the installer felt different when I tested it, but I wasn't sure what it was and assumed it was just a Windows 10 graphical thing. I suppose the new version of Inno doesn't show that page if it detects an install already exists.

I had a look through the Inno docs and found a variable to force that page to display. Thank you for mentioning it. Let me know if it doesn't work for you.

Anonymous 02/28/16 (Sun) 05:43:44 No.2143

>>2126

I use webroot too, try copying over everything except the db folder from the .zip. The issue seems to be that the restore file option doesn't really restore the files, or at least not to where they were in the first place. For whatever reason, this happens every maybe 6 or 7 versions for me.

>>2136

> I suppose the new version of Inno doesn't show that page if it detects an install already exists.

I'm using hydrus on an external drive, and I've migrated to the extract install because the installer hasn't been able to install to that directory. You probably can't do anything about how Inno works to that extent, but it might be something to let them know about.

Also, I noticed that my performance seems to have increased dramatically, so great work!

Anonymous 02/28/16 (Sun) 09:19:00 No.2144

>>2136

>hydrus install_dir/crash.log:

Traceback (most recent call last):

File "<string>", line 23, in <module>