Version 198

Name
Email
Subject
Comment *
File	Select/drop/paste files here
* = required field	[▶ Show post options & limits] Confused? See the FAQ.

Embed	(replaces files and can be used instead)
Options	Do not bump (you can also write sage in the email field) Spoiler images (this replaces the thumbnails of your images with question marks)
Password	(For file and post deletion.)
Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf Max filesize is 8 MB. Max image dimensions are 10000 x 10000. You may upload 5 per post.

YouTube embed. Click thumbnail to play.

Version 198 hydrus_dev ## Board Owner 03/23/16 (Wed) 20:28:30 No.2316

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v198/Hydrus.Network.198.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v198/Hydrus.Network.198.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v198/Hydrus.Network.198.-.OS.X.-.App.dmg

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v198/Hydrus.Network.198.-.OS.X.-.Extract.only.tar.gz

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v198/Hydrus.Network.198.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v198.tar.gz

I had a good week. I added IPFS directory downloads and sped up file search result generation.

IPFS directories

If the IPFS downloader receives a directory multihash, it now asks IPFS for the full nested directory listing and throws that up on a new checkbox tree dialog. You select the files and folders you want to download, hit ok, and the IPFS downloader should attempt to get them all.

The new tree dialog is something new, and I do not know if it renders well on all platforms, so let me know if you have any problems. The overall workflow of this is also fairly rough–I expect I might eventually move this to a traditional queued download page, to support resumes, retries and better error reporting and so on.

The last thing I would like to do to round off 'IPFS plugin v1.0' is adding directory upload. Assuming I can figure out a nice way to do it, preferably with the API at port 5001, then I assume that will be done in the next few weeks.

faster file search results

One particular query inside file search result generation (where it says '256 of 2,000 files…' in the status bar) had become incredibly slow. I am not sure when it got so bad, nor exactly what the problem was (the sqlite query optimiser reported no problems, as far as I understand it), but I have slightly reworded it and the lag has disappeared. On my dev machine, the query went from taking 2.5 seconds per 256 files to 9 milliseconds! Anyone with a lot of tags will see a similar boost in speed when loading files or opening the manage tags dialog, which happens to use the same routine.

full list

- ipfs download now parses directory multihashes

- ipfs directory trees are thrown up on a new checkbox tree dialog for selection/filtering

- created a new download_urls (i.e. multiple urls in one popup) popup call, which the new ipfs directory downloader uses

- improved a very inefficient line of sql in the mappings fetch stage of query result building that was also used in the manage tags dialog. on my dev machine, fetching 256 files' mappings dropped from 2.5s to 9ms!

- regular gallery pages and 'page of images' pages now have a 'paste input' button to mass-add newline-separated queries from the clipboard

- fixed a typo in new txt_tags import folder code

- fixed bad autocomplete predicate sorting

- fixed some sibling autocomplete search_text matching logic

- some byte-based value/range presentation will now correctly convert to bytes, rather than regular decimal numbers

- improved linesep splitting code across the program

- fixed get/setlabel calls across the program to get/setlabeltext, so ampersand characters are handled properly

- wrote an 'all known files' autocomplete cache db and prepped a bunch of code for it–it should be easy to finish next week

- added a new regex practise link to the regex dialogs

- os x app release should display better on retina displays

- misc cleanup

next week

I wrote most of the 'all known files' autocomplete cache this week, so I will polish that off and then focus on some smaller stuff, which has been piling up.

Anonymous 03/24/16 (Thu) 13:29:27 No.2318

Awesome job!

Anonymous 03/28/16 (Mon) 07:05:19 No.2330

Got Fatal Error

"pyi_rth_twisted returned -1"

when starting hydrus

Used the extract verison…

Anonymous 03/28/16 (Mon) 23:53:28 No.2332

>>2330

Got the same error with the installer version of 1.95, I'm installing this one now to see if it'll fix it.

Anonymous 03/29/16 (Tue) 00:45:45 No.2333

>>2332

Just installed the extract version, it doesn't give me any errors, but the process just starts and closes itself after a second.

hydrus_dev ## Board Owner 03/30/16 (Wed) 17:58:13 No.2340

File: 1459360694120.jpg (212.66 KB, 1562x1293, 1562:1293, e89b3ec729a7696cc598f8c5d9….jpg)

>>2330

>>2332

>>2333

I am sorry to hear this. Which OSes are you using? Are you running 32-bit or 64-bit? In your install directory, do you have a crash.log? If so, what is inside it? What about install_dir/logs/client.log–does that have any extra error information?

Anonymous 03/30/16 (Wed) 22:21:36 No.2348

>>2340

>>2330

Win 8, 64Bit. Doesn't generate a crash log in either of those directories.

I also tried rolling back to the previous version by just extracting and replacing- still get the same error

Not sure if I should roll back 1 more version (this is the version I was using until I updated to 198) or should try the installer. Last time my av blocked stuff from the installer, I've since put in exceptions but the extract method was working fine up until now.

Anonymous 04/01/16 (Fri) 04:58:11 No.2358

>>2340

Wang 7 64bit. There's this log inside the folder


Traceback (most recent call last):
  File "<string>", line 23, in <module>
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\ClientController.py", line 3, in <module>
    import ClientDaemons
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\ClientDaemons.py", line 15, in <module>
    import HydrusServer
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\HydrusServer.py", line 2, in <module>
    import HydrusServerResources
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\HydrusServerResources.py", line 13, in <module>
    from twisted.web.server import NOT_DONE_YET
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\twisted\web\server.py", line 36, in <module>
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\twisted\web\util.py", line 304, in <module>
  File "site-packages\twisted\web\util.py", line 387, in FailureElement
  File "site-packages\twisted\web\template.py", line 401, in __init__
  File "site-packages\twisted\web\template.py", line 344, in _flatsaxParse
  File "xml\sax\__init__.py", line 93, in make_parser
SAXReaderNotAvailable: No parsers found

Anonymous 04/01/16 (Fri) 04:59:47 No.2359

flood

>>2340

Wang 7 64bit. There's this log inside the folder


Traceback (most recent call last):
  File "<string>", line 23, in <module>
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\ClientController.py", line 3, in <module>
    import ClientDaemons
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\ClientDaemons.py", line 15, in <module>
    import HydrusServer
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\HydrusServer.py", line 2, in <module>
    import HydrusServerResources
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "include\HydrusServerResources.py", line 13, in <module>
    from twisted.web.server import NOT_DONE_YET
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\twisted\web\server.py", line 36, in <module>
  File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
  File "site-packages\twisted\web\util.py", line 304, in <module>
  File "site-packages\twisted\web\util.py", line 387, in FailureElement
  File "site-packages\twisted\web\template.py", line 401, in __init__
  File "site-packages\twisted\web\template.py", line 344, in _flatsaxParse
  File "xml\sax\__init__.py", line 93, in make_parser
SAXReaderNotAvailable: No parsers found

hydrus_dev ## Board Owner 04/02/16 (Sat) 21:16:36 No.2369

File: 1459631797209.jpg (157.72 KB, 1268x1573, 1268:1573, 751ac2fe7d1b7b0f232fab4a35….jpg)

>>2348

>>2358

>>2359

That looks like a dll error–likely something missing, or something old that is conflicting with something new. I think I have seen that 'xml' error before. You should have lxml.etree.pyd in your install directory, at 854,528 bytes, and nothing that starts xml…. I think a clean install fixed it.

So, please try a clean install:

Back up your install_dir/db folder. This is where all your files and settings are stored, so make sure you have a safe copy, just in case something goes wrong.

Delete everything in your install directory except the db folder.

Extract a fresh v199 install to your install directory.

( https://github.com/hydrusnetwork/hydrus/releases/download/v199/Hydrus.Network.199.-.Windows.-.Extract.only.zip )

Try running client.exe.

And let me know how you get on. If this still gives problems, I can put a debug build together that should tell us more.

Anonymous 04/03/16 (Sun) 02:01:32 No.2371

File: 1459648893127.jpg (28.63 KB, 368x368, 1:1, 1459385857367.jpg)

>>2369

Tried a "clean" install with extract version, same error

Tried a "clean" install with installer version, same error

Tried an install without my db, same error

Uninstalled and reinstalled hydrus, same error

Uninstalled and reinstalled python & hydrus, same error

Not sure what else there is to try….

>>2358

>>2359

Just out of curiosity, would you happen to be using the aegis voat script to block all telemetry updates and Windows 10 upgrade components for Win 7/8?

That's the only "major" change I've made to my laptop but I don't see how it could be related.

Just fishing for ideas.

hydrus_dev ## Board Owner 04/05/16 (Tue) 18:28:40 No.2379

File: 1459880921228.jpg (110.2 KB, 642x650, 321:325, 480d8f3c378c7ca6bcb6260937….jpg)

I searched a little more, and apparently this problem might rather be caused by a missing pyexpat.pyd file (which also does xml stuff), also in your install directory. I can induce the exact same error mesage by removing that file from a fresh extract. It should be 62,424 bytes. It is in the v199 extract zip, but maybe your anti-virus is bopping it?

In any case, I have tried to extract the broken import (which is inside the client's http server code, for its local server or local booru) out of the boot code, so if/when it fails for you in v200, it'll just throw up an ugly popup error rather than a full crash.

Anonymous 04/06/16 (Wed) 06:06:30 No.2386

>>2379

I've got the file and the size matches. I've done both extract and install with my anti-virus off but I just gave it another go anyway with the same result, sadly.

It's odd that the error would show up when I go from v196->v198, but then also appear when I try going backwards to v197 and v196. Normally I'd think something's corrupted or conflicting but since I've done reinstalls of both hydrus and python I'm out of ideas…

Guess we'll see what happens with v200

hydrus_dev ## Board Owner 04/06/16 (Wed) 17:20:40 No.2388

File: 1459963255471.webm (1.68 MB, 1280x720, 16:9, 7c063a7dd41565abf58eedfc8….webm)

>>2386

This could be moot if v200 works for you, but perhaps there is a conflict between your python and hydrus–maybe when client.exe tries to load the dll–pyexpat.pyd or whatever–it is actually detecting your python install path and loading it from there (typically C:\\Python27\DLLs, where my pyexpat.pyd is 180,736 bytes) rather than hydrus's install_dir. But I'm afraid I don't understand the intricacies of PyInstaller (which I use to freeze client.pyw->client.exe) enough to know how to check this stuff.

I have python 2.7.11 64-bit. If you have 3.0 or 32-bit and this odd dll loading is happening, that might be it.

If you haven't tried running hydrus with python uninstalled, that might be worth a try, or if you need python for your own programming, you could even try just running hydrus from source, like I do on my dev machine. You should be able to do it just by going python client.pyw in the install_dir, but you'll need a ton of libraries as listed here:

http://hydrusnetwork.github.io/hydrus/help/running_from_source.html

I think I'm just idly gabbling–we'll see how v200 works.

Anonymous 04/06/16 (Wed) 23:11:53 No.2392

File: 1459984313547.png (308.28 KB, 447x524, 447:524, 1413700824426.png)