[ / / / / / / / / / / / / / ] [ dir / acme / agatha2 / animu / arepa / asmr / games / leftpol / mde ]

/qresearch/ - Q Research Board

Research and discussion about Q's crumbs
Winner of the 53rd Attention-Hungry Games
/d/ - egenerates

August 2018 - 8chan Transparency Report
Name
Email
Subject
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webm, mp4, pdf
Max filesize is 16 MB.
Max image dimensions are 15000 x 15000.
You may upload 5 per post.


First time on QResearch? 8chan? Click here, newfag.



File: 8e1a4c121b66428⋯.jpg (23.65 KB, 400x265, 80:53, ComputerProgramming16x9.jpg)

d1575b  No.2352371

This is a thread created by a programmer for programmers to interact in support of QResearch. Great place to perform live Q&A chat.

Allowed file types:jpg, jpeg, gif, png, webm, mp4, pdf

Max filesize is 16 MB.

Max image dimensions are 15000 x 15000.

You may upload 5 per post.

d1575b  No.2352393

TW, you there?


8ce561  No.2352397

TW here.


d1575b  No.2352403

WT, here. TW said:

>It's a searchable offline archive of posts and tweets, easy to seed with multiple collections. Shows time deltas. Qclock filter function. Caching proxy for images, rewrites URLs to fetch from archives when original picture is not available.

>I run it locally. I have no idea what the best packaging/distribution approach is, thoughts are welcome.

>Multiple levels of trust needed to share, run. Doing this safely for everybody is my main concern. Not everybody can inspect the code at every release, or would be willing to install/run on their machine.

Wow. Sounds involved. Let's get some basic info outta the way and maybe I can give you things to consider…


8ce561  No.2352404

So, WT, ideas on how I could share what I have written?


d1575b  No.2352409

>>2352397

First off, what platform(s) are you targeting?


d1575b  No.2352418

Second, please tell me it's going to be free to the masses.


d1575b  No.2352423

>>2352409

Windows? Android? Apple? Any browser?


8ce561  No.2352426

>>2352409

Format is made for desktop, haven't tested on mobile.

Frontend it will run on anything that has a relatively modern browser.

Backend needs node. Also ext4fs to store large collections of files, but this limitation can be removed.


d1575b  No.2352437

>>2352426

Windows Desktop?


8ce561  No.2352445

>>2352418

>Learn how to archive offline.

The recent discovery on twitter post correction delta and Qclock means the tool would also speed up the dig.


d1575b  No.2352450

>>2352445

>ext4fs

Linux?


8ce561  No.2352455

>>2352437

The frontend runs on any modern browser.

Electron could be used to package the app for any platform, frontend and backend. I've started working on it but it needs more work, if this is the best packaging approach.

I host the backend in a (linux) virtual machine, but it could be a server in the cloud. Having everything available offline was the driving motivation, so could hosting would defeat that purpose.


d1575b  No.2352462

Third, what's the dev platform?

I know alot about deployment on Windows (all flavors), Android and a little about Apple (if developing on Windows). Other stuff you'll have to leave a post here and wait to hear something.


d1575b  No.2352470

>>2352455

What's the front-end written in? GUI or cmd line UI?


8ce561  No.2352495

>>2352462

>>2352470

The frontend is written in JS (modern, requires babel/JSX) and runs in any browser. GUI. (The command line is used to fetch resources, but I'll integrate that to the UI.)

Electron embeds a browser and would allow it to run like a native app on Mac & Windows.

Distributing in a *safe* way is my main concern.

>How can user trust that they don't get a malicious version of the app?

>How can I avoid doxxing myself sharing this?


d1575b  No.2352506

>>2352495

Wow. Good questions. Hang on…


8ce561  No.2352507

If it were a safe solution I'd just put the code up on github, post a link here, and let someone address the packaging/distribution. I'm a simple codeanon.


d1575b  No.2352520

>>2352495

So you have a very unique problem, my man. I don't think I've ever been involved in a deployment that was anonymous. It may sound ridiculous but perhaps you need to think like a hacker in that you deploy an install program via torrent and include a pre-auth "safe certificate" from an anti-virus company.


8ce561  No.2352550

>>2352520

A simple bootstrapper with code signing? Yes, it seems to solve the "anonymous distribution" part of the problem.

Why would users trust that I don't get comped resulting in malicious code getting pushed in updates?


d1575b  No.2352567

>>2352550

Anti-virus companies will issue you (for a small fee) a certificate. Users go to the AV website, enter the KeyCode from your certificate and are told if its safe or not.


8ce561  No.2352571

>>2352550

More precisely, if the software is useful, how do I make sure it cannot be exploited as a troyan by Clowns? Or why is this not a concern?


d1575b  No.2352575

>>2352550

Oooh. Bad actors. Hmmm. That's going to have to be approached from a checksum/md5 point of view. MD5 is your friend for Linux dists.


8ce561  No.2352584

>>2352567

This moves the trust to the anti-virus company. Do the Clowns have a copy of the CA? Is it not safer to use a self-signed CA? I don't know enough about this topic to make a decision.


8ce561  No.2352596

>>2352575

Code signing solves the tampering problem, but it doesn't prevent malicious actors from getting at me and taking over the distribution infrastructure.

If sharing this is going to put me and everybody at more risk than keeping the code for myself, it seems rational not to?


d1575b  No.2352606

>>2352584

This certificate I'm talking about is not the same as a CA. It's for verifying that you app is not malware.

As for MD5, be sure you have finished all MD5 computations before dropping the 5 to 8 bucks for the AV cert.


d1575b  No.2352610

If you stick with MD5 and give appropriate warnings to ensure MD5's match before use, you should be at minimal risk.


d1575b  No.2352623

>>2352596

A malicious actor could do a man-in-the-middle attack on your database is he knows a specific person is using it. I dunno. The variables on attack vectors expand exponentially there.


d1575b  No.2352634

I don't want to mislead you–I'm stretching my knowledge here since I'm primarily a Windows/Android Dev. Sorry I couldn't be of more help.


8ce561  No.2352645

>>2352596

Writing a great app and distributing it in an *apparently* safe and anonymous way is an excellent vector to compromise autists. I don't know why the Clowns haven't already done that. Too much effort? Risk of being exposed?

>>2352606

The code signing certificate is trusted by a CA. Anyone with a copy of the CA can produce signed code.

You are suggesting MD5, but is a very weak hash function.


d1575b  No.2352681

>>2352645

I'm saying get a reputable company to certify your app as legit (not malware) since you'd be anonymously distributing it as a torrent.

Then, using MD5 can tell the end user that not a single byte has been altered in the current deployment. MD% is more than adequate for that small task.


d1575b  No.2352683

Gotta head to work. Hope I helped somewhat. Shadilay!


8ce561  No.2352696

>>2352623

There is no database, it's all in local plain files.

>>2352634

I understand. It is a difficult topic. I'll keep in mind the signed code bootstrapper idea, it is part of the solution.

Thank you for the discussion.

If Q team is reading this, maybe get in touch? Extract me and I'll happily write code for the community. Though it's perhaps not worth the hassle for you at this point.


8ce561  No.2352708

>>2352683

Thanks & Shadilay bro!


29f2fa  No.2354303

>>2352696

EA here. Can help with cloud / distribution / devops.

If its "all local plain files" (btw that's good for "store it all offline") then the attack vector changes from MITM to corrupting the file sources. So we'd want a way for original file owner to verify/checksum the distribution source, and the downloader to verify/checksum his copy.

>>2352520 Certificate can come from LetsEncrypt.org. Also it's free. I use this for some of my sites already.


8ce561  No.2354516

>>2354303

Thank you for the input.

Not sure about LE certificates, I believe they are tied to a domain due to the verification process? I don't know that I would be able to secure a domain anonymously. Also LE certs expire after 3 months, they are not meant for code signing.

I am hesitant to go with a self-signed CA, it seems maybe risky but I haven't thought it through yet.

Local plain files by design. I don't rule out using one or several local DB engines, but they would only contain information that can be reconstructed from the local plain files.

Indeed corruption at the source could be a problem. I have a (python) 8chan thread archival tool that could be made available as a service (ran by independent sources), integrating the hashing process. Cross-checking sources would help detecting comped ones.

I'll work on this aspect as soon as paying job permits.

Is there such a thing as anonymous github without going to the dark web? A trustworthy (NSA/MIL) git server would be awesome, but I have no idea how I would get access to that and have reasons to believe it is safe to use. Also I am neither US resident nor citizen.


dc0709  No.2359386

File: 15cdf7c04f82765⋯.jpeg (11.71 KB, 300x168, 25:14, lol.jpeg)

not trying to slide, but you are all geniuses! fantastic work you are all doing.

please keep it up, and when you have the time, offer some advice to a noob trying to become a master computer scientists such as you guys.

>what programming language to start

>what to learn besides programming languages

>where is some good places to learn CS

thanks and again great work


1aaba4  No.2360560

File: dd199e053c4a9b4⋯.png (1.94 MB, 1280x1920, 2:3, 2.png)

File: 9d58e3081e70b01⋯.png (2.89 MB, 1280x1920, 2:3, 027a0f6a2e1d249c98c6459eca….png)

>>2359386

damnit i am having brain fart so many steg progs installed cant remember which one subtracts one image from another

2.png needs subtracted from 027

or combined

from what i can tell with bitmasking its a girl kneeling with a fountain of blood spraying on her or out of her, the water is blood i seperated them HELP!

also zsteg is far far superior for detecting hidden shit

found out all kinds of smartphone virus's posting by clown bots, motorola assembly files and shit

binwalk

foremost

anyone running apache can you get this up and running? finding hidden twitter images

careful with 027 it has detected headers of phone executables but could be false

https://github.com/holloway/steg-of-the-dump


8ce561  No.2364880

File: 51ec050adb0ced8⋯.png (279.22 KB, 1656x1293, 552:431, qminer_demo.png)

Demo.


e4be4b  No.2366039

File: deb3d379242dafc⋯.jpg (399.54 KB, 2278x1780, 1139:890, patchwork client.jpg)

>>2354516

> Is there such a thing as anonymous github

I'm starting to evaluate git-ssb for myself. I'm not sure yet about how well it would withstand attacks by determined adversaries. It is decentralized and requires some extra software running, I'm not sure if that counts as "dark web".

https://git-ssb.celehner.com/%25RPKzL382v2fAia5HuDNHD5kkFdlP7bGvXQApSXqOBwc%3D.sha256

It's built on top of secure-scuttlebutt:

"A database of unforgeable append-only feeds, optimized for efficient replication for peer to peer protocols."

https://github.com/ssbc/secure-scuttlebutt

Social network application (kind of like twitter):

https://github.com/ssbc/patchwork

>>2352596

> Code signing solves the tampering problem, but it doesn't prevent malicious actors from getting at me and taking over the distribution infrastructure.

Secure Scuttlebutt has similar benefits to a blockchain. Anyone can distribute your messages (not tamper), and you can't rewrite your post history. Kind of like with Git how every commit has a hash that is based (in part) on the commit history. So if you were compromised, only messages you write from that point forward would be affected.

Conceivably, you could publish messages in the past if you create them in order. So for example you could create a "Q" identity (public/private keys) and import all Q posts in order and have them contain the desired timestamps. You wouldn't be able to change them once published to the network, kind of like with Git's commit history.


d5f7c2  No.2366235

>>2354516

Self-signed us useless since https will flag it as self-signed hence "questionable".

Not certificate authority trail to follow for credibility.

Basically as is server owner says "I vouch for my self"


d5f7c2  No.2366298

>>2359386

read some books . very few real "masters" here but some highly competent, self-taught but frequently "narrow" expertise.

Don't bother talking about Internet traffic routing, security or detains of IP protocol family. Very, very few here that speak that language.

Host-based programming/scripting mostly, and web stuff


d5f7c2  No.2366315

>>2360560

So you already discovered the password?


e4be4b  No.2366980

>>2366235

For code signing, PGP (gpg) is great. Git supports commit signing with it, and it is used by major linux distros by their package management utilities.


d38a5a  No.2367573

>>2359386

Start with Python. https://www.learnpython.org/

Also learn databases - start with sqlite, because it doesn't require a server. UnQLite is also a good db to start with - it's up to you to learn the difference between SQL and NoSQL. Learn html and javascript. There is even a programming language based on javascript called nodejs that can also be useful.

I believe these are good places to start:

https://www.tutorialspoint.com/index.htm

https://www.codecademy.com/


91b78f  No.2368501

Hello fellow programmers!

Just viewed this video https://youtu.be/MqmeteSv8cU and SerialBrain2 spotted that "FARMER" and "QANON" have the same English letter gematria.

Pretty cool, but who would notice that without a calculator or word list? Just made one! Based on English vocabulary file "enable1.txt" (used by Words with Friends game). https://anonfile.com/Z8H295f7b1/gematria.txt

It would be better to make a gematria word list based on Q post, especially since English gematria seems to often reference names…

There may be a certain intuition or Sense Motive to hone in on likely clues but let's up our game!


05a340  No.2368709

>>2366298

Anon, that was beautiful, you are a great 'first contact' Anon!


fa4e1b  No.2368845

File: 137be270b9a7de0⋯.png (865.2 KB, 2880x1800, 8:5, QAnalysis.png)

I posted as new thread before I saw this thread…

Tool for Q-searchers. Some fragments mapped, ball is in your court.

https://mega.nz/#!WfhRlASA!gaDqWNdpP-OSWY4bue0Oj0En8h9lXX85J0FFIXytRGI

Node JS, Electron, Cytoscape, GunDB

cd into extracted directory and run "npm run"


d5f7c2  No.2368941

>>2368501

Gematria originated as an Assyro-Babylonian-Greek system of alphanumeric code or cipher later adopted into Jewish culture that assigns numerical value to a word, name, or phrase in the belief that words or phrases with identical numerical values bear some relation to each other or bear some relation to the number itself as it may apply to Nature, a person's age, the calendar year, or the like. A single word can yield multiple values depending on the system used.

Although ostensibly derived from Greek, it is largely used in Jewish texts, notably in those associated with the Kabbalah. The term does not appear in the Hebrew Bible itself.

Some identify two forms of gematria: the "revealed" form, which is prevalent in many hermeneutic methods found throughout Rabbinic literature, and the "mystical" form, a largely Kabbalistic practice.

Though gematria is most often used to calculate the values of individual words, psukim (Biblical verses), Talmudical aphorisms, sentences from the standard Jewish prayers, personal, angelic and Godly names, and other religiously significant material, Kabbalists use them often for arbitrary phrases and, occasionally, for various languages.

A few instances of gematria in Arabic, Spanish and Greek, are mentioned in the works of some Hasidic Rabbis also used it, though rarely, for Yiddish.

However, the primary language for gematria calculations has always been and remains Hebrew and, to a lesser degree, Aramaic.

Numerology is any belief in the divine or mystical relationship between a number and one or more coinciding events. It is also the study of the numerical value of the letters in words, names and ideas. It is often associated with the paranormal, alongside astrology and similar divinatory arts.

Despite the long history of numerological ideas, the word "numerology" is not recorded in English before c.1907

So it would be correct to describe it as sort of a mystical, Kabbalistic form of numerology voodoo like reading chicken bones?

With no scientific or linguistic basis?

Just magic words and numbers?

And we are supposed to take it seriously?

OK


8ce561  No.2369040

>>2366039

Thank you Anon! SSB is precisely what I was looking for.

@ljtQyLKmVLKw/jGzA1lqugPLL+8sDO7AYnTJqr9lYcI=.ed25519

Still setting this up, I'll get there.


40c6d7  No.2369158

>>2368941

I don't know about mystical aspects but it sounds like some players are using it as a means of secret message. Q = 17 has come up a bunch of times too so it could be a communications technique.

Let's check 187…

If people are using gematria as a name-drop/signature and time deltas sometimes… Let's check 111… No luck. What are some good numbers to search for because it might match names.

All those 2 letter abrieviations would have values between 2 and 52… This isn't going anywhere. Wonder if any names equal 30 after those 30 days of silence… NP = 30, Namcy Pelosi? She isn't in the news at this time, bad loose thread…

If and when a real gematria indicator is present you would think other collaborating hints to be in same message… Maybe doing it backwards then. :/

Fine! I AM NOT SERIAL BRAIN! HE IS THE BRAIN AND I AM THE MOUTH! haha Maybe I got carried away there…


b82900  No.2369794

What if it's useful to LABEL the edges between nodes. And thus/then have multiple edges.

For instance, Strzok has multiple connections to the same person (ie Page etc) but each connection has a different flavor (ie co-conspirator, "textual" relationship). This is my personal gripe with https://DiscoverTheNetworks.org, that it doesn't provide a 10k foot view of how the quid pro quo work. Another example: WJC gives a "speech" on a date, Russia "pays" him for it, and then somehow uranium is exported to them.

Yes, populating this thing will be time-consuming, but we need not try to enter all the details all at once. Eat an elephant one bite at a time.

SSB looks like a winner.

GunDB is very interesting but doesn't have property lists for edges. It's possible to use intermediate nodes for edges, but then the challenge is data management or graph visualization.


6b3962  No.2370327

>>2364880

Looks interesting. Using the API?


7ceeb5  No.2370545

>>2359386

agree!

hope u dont mind muh lurking as well


6b3962  No.2370561

>>2359386

>>2367573

^^^^^^^^^^^

>Start with Python, SQL. Learn html and javascript. These 4 languages will get you FAR all by themselves.

Start with HTML/Javascript if you are starting out, then Python and SQL


fa4e1b  No.2371101

GunDB lets you have property lists for each node and many connections. Q Analysis app lets one manage those individual properties. For each node with only name and image displayed. Each node can cross link to another graph file or be linked to a url. All html css js. Can customize presentation layer.


8ce561  No.2371704

>>2370327

Yes, it loads a local JSON file that comes straight from qanon.news/api/posts. I plan to add support for more sources but time has been scarce lately.

Hopefully I can share this before Q starts wrapping up.


c9230e  No.2371963

>>2371704

Haha yeah, can we finish our tools before Q finishes dismantling a 2,000 year global conspiracy to enslave the entire species?


bb7b36  No.2377944

Programmer here, was working on self-redistributing OS and encountered a similar problem to yours on verification. You can't provide a guaranteed 100% non-comped program, all you can do is make it so it's extremely unlikely (don't let perfect be the enemy of good).

MD5 hash has been broken for years and data can be manipulated in transit or in standing. JavaScript payload is sent plaintext and can be intercepted (see 'problems with encryption in JavaScript' - you can't send a reliable JavaScript program without the reliable program having a reliable means of transport).

Electron is a Chrome spin-off and I'd advise away from 'Bridge' technologies.

It's unclear if you're after a standalone app or a webhosted app, so I'll attempt to bridge both. Webhosted is nearly impossible to give any real assurances of non-comped status (you or host can be compromised, can be hijacked on site, etc).

Standalone requires a webhost, so it hits the same issue.

So, tips:

1) Keep the app small. Smaller it is, the easier it is to do a byte-by-byte comparison. A 20MB app would be mere seconds, 800mb is going to be a pain.

2) Use anything stronger than MD5 for a hash. Avoid anything with the NSA's rubberstamp of approval (that means no SHA256 etc).

3) Supply more than one type of hash of the software.

4) Make sure you have an alternative source that keeps a copy of the current hash(es) [EG an archive page] so even if main website is comped, archive isn't.

5) Utilise multiple webhosts for hosting the main package itself (why? To compromise the software all hosts have to be hijacked). You can include free software hosts etc into this mix. Don't touch Mega because Kim Dotcom no longer owns it (NZ government gave it to a Chinese investor).

6) Include a copy of the hashes in a text file which is shared with the software package.

7) Use a 'read only' storage format (EG squashfs or an encrypted archive). In order to tamper with it, you'd need to replace the entire file.

8) Supply the individual with the means to build their own from scratch. So if the binaries are comped, it's possible to recreate a non-comped binary from source.

9) Archive copies of the source code.

10) Only push out major releases (so you're not overwhelmed with the archiving/hashes/mirroring).

Essentially to avoid software compromise, you need many alternative copies and many checks/balances. As for yourself being compromised, you need someone who acts as your watch guardian (they don't have control over the project, but they do have the power to say if you're compromised).

Having warrant canaries and a specific coded signal that you can mention that the community will know to mean you're compromised will also help.

If you sell out and none of the checks succeed, then the open source code or binaries would be analysed or decompiled by a white hat eventually and you'd become exposed anyway if that was the case.

But the most compromised product of all is the one you don't even produce. ; )


8ce561  No.2380208

>>2377944

Wow. Thank you Anon, this is a fantastic response, very well thought out. Will re-read often.

Docker? Small images, reproducible builds. Not much isolation, but probably enough. Not very easy to run.

Virtualized Alpine-Linux-based iso image? A little heavier, perhaps not so much with careful decisions (go backend instead of python). I think I prefer this approach.

I'll try harder. Again, thank you!


fa4e1b  No.2381203

>>2377944

Yes on all points. However, for my part, I’ve got too little time to maintain my self. Source is there. Any one can read and decide if they wish to utilize it. Its a start, hopefully others can help realize it. I can achieve what I set out to be able to achieve and figured others might get something out of it.


8ce561  No.2391576

I created a board for Q Research software development.

>>>/qcode/1


6b3962  No.2398158

Whatever happened to the idea of a Q Research Wiki? Did that whole thing die out?


bb7b36  No.2401309

>>2398158

Wiki software requires a dedicated PHP host of some sort to host (especially if you want to stay protected against censorship).

Wikia is a free alternative but it reeks of liberialism and I bet would censor in a heartbeat.

It would be nothing short of a full-time job to maintain (both against shills and keeping it up to date/organised).

I could guide you on the basics of setting up MediaWiki on a Linux box (LAMP + some light configuration), but I absolutely do not have the resources to support the endeavour, I am literally overstretched.


6b3962  No.2412127

>>2401309

I like the idea of automation. If we had a good way of trawling the breads to assemble info the wiki could build itself. That too, is probably a full time job.

What are the projects that are currently being worked on? Tying them together or collaborating would make things go faster. Lets pool our resources.


1bc038  No.2449345

just a question

looks to me like there are very advanced bots out in the open

if so, I might have bumped into such bots in public forum

- they follow an agenda

- they give likes

- they give dislikes

- they distract

- they talk to each other

- they answer questions

- they notice Leetspeak, but can't understand it


2e41e8  No.2469655

I made a neat script that could be useful for Linux users.

Many times if you are making a script to process a lot of files it is not so easy to fully use all your possible CPU power (cores->threads).

So after few tries I can up with this script that can be used as a base. Easy to modify to your needs and it uses all possible CPU that you allow.

for example if you have files to process (for example test thousands of images if they contain steganography), list the files, feed it as standard input to the script and let it spread the processing to all available cores / threads.

For example:

- list files using ls -1 *.png will list file names, pipe it to the script

- or any other line based info that you can process. the script can spread the items to process in your way to all cores.

here is the script:

#!/bin/bash

input="-"

tot=0

# THIS gets the available processing units, if 4 cores => 8 threads this will get value 8

# depending on your jobs, could try values like units -1, -2, +1, +2… or *2, *3, *4

units=$(nproc –all)

cat $input | while read line; do

tot=$(($tot + 1)) # just a counter to display total processed

./clean1.sh $line > $line.clean & # THIS line will run parallel jobs. Change it to anything you want, remember & at the end

jobs=$(jobs -p | wc -l | grep -o '[0-9]*') # currently running job

echo $tot $line "($jobs)"

#echo "Jobs $jobs"

while [ $jobs -ge $units ]; do # if jobs at the maximum value, wait

#echo -n '.'

sleep 0.01

jobs=$(jobs -p | wc -l | grep -o '[0-9]*')

done # when done, continue reading input and add more jobs.

done

wait

echo Done $tot


fa4e1b  No.2493793

>>2412127

AI can assist in this. I’ve been working on AI to monitor reddit for shills and abusive moderation. Talking point detection is on my list too.

If we can detect these things with AI we can respond to shills with evidence to refute their argument and with moderators, archive and record events to expose the infiltration.

I’m hesitant to post code yet, not sure it’s a good idea to put something out that can be weaponized by trolls the same as we use it as defense.

Right now I do an ok job of detecting when people are arguing vs debating, asking questions vs concern shilling. Starting to narrow in on detecting posts likely to be removed by a boards moderator but need more sample data.

None the less it’s possible. We just need great training data. Sample conversations with shills and sample moderated posts.


e7dd40  No.2494187

>>2391576

>honeypot


6b3962  No.2495240

>>2493793

Some javascript magic? I'd be interested in reading it.


fa4e1b  No.2535594

>>2495240

I’ll post source this weekend. Real life has been occupying free time.


6b3962  No.2537432

File: 08b4e0e7aba612d⋯.gif (130.52 KB, 220x268, 55:67, doit.gif)


b7fb97  No.2561043

>>2412127

Thread parsing automation (you'll need to install beautiful soup and mechanize, on Linux this is easy [do an apt-cache search for beautiful soup, and then mechanize]).

Here's HunterKiller bot (it can parse threads, but the code is sufficient enough you can probably re-engineer it to parse posts from threads). It was censored by the mods when they deleted the Q-branch thread:

https://pastebin.com/LmPFhtXm

Uses python 2.7 to my knowledge. Can't help with Windows, I ditched that shit OS years ago.


6b3962  No.2561161

>>2561043

>https://pastebin.com/LmPFhtXm

Interesting! Looks good - although you'll possibly miss some breads due to ebakes and what not if they don't match your "Q Research" restrictions.

I've been archiving the JSON from here since about FEB. I have over 3000 breads all archived locally and online in JSON - I just need to work out some logic on how to trawl with some context in order to make some sense of all the data we've found.

I'll take another look later on. Looks good!


b7fb97  No.2561191

>>2449345

Those are 'professional level' software bots, which have been in development for quite some time now (since at least 2010). The end goal of such software is to get 'natural speaking' bots that can engage targets (and 'talk' with other bots to make it seem like a legitimate conversation is ongoing).

I spent years lurking and interacting on dubious forums where such malicious activities were being tested. The bots aren't perfected (they have the same flaws as normal chatbots, presently), but there's an ongoing effort to make them 'more advanced'.

HunterKiller bot is homebrew, but it's based on several iterations of code which was based on observations of the so-called 'professional' bots. Such software is sold to both military and political activism groups (the bad kind: think Media Matters).

HunterKiller was my proposal to counter the bots: basically, a bot advanced enough to hunt other bots. What I've given you is a barebones example that should contain sufficient enough information for you to build your own variants.

It's my personal opinion that passion trumps corporate software development any day of the week. That code took me about 7 days to write (I have limited free time), but the potential to tack on other, more advanced Python libraries are there.

PS: Shills often use scripts (folders and pieces of paper in more primitive operations), more advanced shill operations use specially designed software that allows them to copy/paste generic garbage responses (usually across several accounts or IPs, depending on sitch), and very advanced shills have bots that automatically select what garbage to copy/paste with the shill acting as bot handler.

Check out the Clown College thread where I explain more on bots in my earlier posts.

From a strategic standpoint, you have the homefield advantage, because shills/bot posters reply on spam and generic replies or obvious tells. With a HunterKiller bot that is sufficiently well programmed, you can mass identify these bots and shills for some beatdown with administration tools.

Eventually you will experience shills who have tools that can 'thesaurus' the words around so it seems 'different' so don't reply on verbatim matches but perhaps even Markov chain analysis.

Hope this helps.


b7fb97  No.2561225

>>2561161

The point of the tool is to actually filter out the breads because it's a HunterKiller (you're not huntng/killing the breads: you're looking for the garbage threads). But you could modify it to investigate breads for shill posts or copy/pastes. It's up to you.

If you skim over the many anchored threads, you might notice it almost appears as if the admin are using such a tool (which greatly speeds up identification rates of trash threads). It's a pity they censored it as it was intended to help, not hinder (it cannot post and I won't build it so it can as that would only merely aid the shills).


b7fb97  No.2561418

HunterKiller's friends (variations) include:

ArchiveBot: mass collect all posts from all active threads in the catalogue (allowing you to do a raw text save of the data). Alternate version: bulk send archive requests of the thread URLs to archive.is/archive.org.

[Properly combined, you can keep a simultaneous offline/online version. Word of warning: when archiving to a website, be sure to only archive 'finished' threads on archive.is and to space out the requests over several minutes so it doesn't appear you are flooding/a bot. Automated and slow is better than manual and even slower.]

MonitorBot: keep track of which threads have 'moved position' and thus have 'new replies' (this is a technique used by shills to direct their limited resources to whichever thread is presently active; likewise, you can do the same).

KeywordBot: Have a bot that looks for specific keywords, image filenames or other triggers and then flag them up when it spots it (shills also use this technique to know when you're talking about a subject that they need to 'shill on').

Newsfeed/TrackerBot: use it to pull the latest news from websites (it's strongly recommended you use a news sites' RSS feed to do this as it keeps it nice and simple). Will require substantially more work and beware shitty unicode strings in the returned data.

Literally, whatever the hell else you could imagine. You're also not restricted to this board. If you change the URL to another board, the code should largely still work (albeit you might have to modify what data gets accepted as some boards don't have poster names).

If you want to get super anal, in theory you don't even need python to build an auto-parse tool. If you're batshit insane, you could even use wget coupled with a bash script.

Beautiful Soup and Mechanize are extremely powerful. Beautiful Soup does HTML parsing, and Mechanize is like a full-blown browser under the hood. You *can* post to a forum/board, but I've noticed the Media Matters shills appear to be doing everything manually (or whatever they have is absolutely shit), so I leave it as an exercise to talented chans to develop (highly advise you do NOT publish any posting capabilities as it will only arm the less well developed shills).

And believe me, this is just scraping the surface.


0e37ff  No.2566977

>>2561191

yes my friend … this makes sense

i was amazed on how 'smart' they interact and how vicious they attacked a deep state thread with just links. i managed to expose at least one bot and got banned by forum admin for 1 month before i was able to target more.


33e1de  No.2575758

Hey guise,

I've been chatting with a few Bakers and they would love to have the bread replies count at the top of the page so that they don't have to scroll to the bottom of the bread to check how many posts have been made, and when to start the fresh bread.

Anyone know if this can be added to the user js or a simple css fix?

Any ideas? Thanks guise.


2a3fd5  No.2587705

File: 45e06df9f1b20b4⋯.png (188.92 KB, 1359x355, 1359:355, script.png)

AntiSpam & ToastMaster Scripts Combined

https://mega.nz/#!OzIlkA5C!NDNVJN848S76siNypYvzGgxYewZHrTPdnghpS2n7Ky4


8ce561  No.2588476

>>2575758

Something like this?

$(function(){
$(document.head).append('<style>#post-counter{position:fixed;top:20px;right:10px;font:24px sans-serif;opacity:0.5;color:#f60;}</style>');
$(document.body).append('<div id="post-counter"/>');
function updateCounter() {$('#post-counter').text($('.thread>.post.reply').length);}
setInterval(updateCounter, 500);
});


8ce561  No.2589502

>>2588476

Cleaner version: >>2589214


6f830b  No.2598959

>>2566977

In this day and era we need to keep pace with the developments of military and commercial establishments. No point trying to fight bots as humans because that's exactly what they want you to do - waste time on bots.

Instead, you want bots to fight bots, so the humans can fight the humans. 9 times out of 10, all you need is an identification tool so admins can simply cap and ban and that's it.

I see HunterKiller has inspired a JavaScript variant which aids finding breads/spotting spam. I didn't build a bread spotter because it can allow shills to funnel their less competent members directly into a bread.

Be very wary of what tools you do realise, bearing in mind that shills have absolutely no qualms about stealing them, repurposing or reverse engineering them. They are largely conducting illegal activities, after all.

(PS: Create legal traps specifically targeted at paid shills in the licencing of your code, so if it ever gets found in their hands, you have means to 'return fire')


6f830b  No.2599230

Going to hand off some of my tasks to other anons, if they want to take it up.

Tools you might want to consider developing (basic tech specs included):

Online archival status checker:

1) Parses an entire bread

2) Pulls all links (feel free to add in a regex link filter so you can filter out irrelevant links)

3) Checks each link, in turn, against an archive host (EG archive.is, archive.org), to check if they have been archived online

4) If not, archives it

5) Generates a generic status report message that you can then copy and post to that bread (including full bread name) to let anons know you've done an online archive.

Missing bread/duplicate bread detector:

1) Parse catalogue for breads

2) does bread numbering (HK has this built in, so feel free to rip the python code I supplied for this)

3) Highlights missing numbers, duplicates as a 'status report' message (with direct links to dupe threads) which can then be posted to an admin to investigate/solve

One for the JavaScript anons:

Bread filter tool:

1) User supplies URL of bread

2) User has button that says 'update view of bread' (to stop laggy constant real-time updating)

3) Tool returns all posts in a bread that have pertinent research

4) (Optional) Have a set of toggle options that allows you to filter the posts depending on the following options:

[Has link] (allowed/not allowed)

Subset options of [Has link]: [Contains archive link](On/Off)[Contains old media link](On/Off)[Contains image link](on/off)[Other](on/off)

[Has no link] (allowed/not allowed)

[File upload only] (allowed/not allowed)

[Contains text] (allowed/not allowed)

[Contains gratitude] (allowed/not allowed)

You can probably think of other options, that's just to inspire.

5) (Optional) Have it work across all breads found on a catalogue (warning: will be extremely slow/laggy, will need serious optimisation)

Bread generation tool (best if kept offline or restricted to qualified bakers only to avoid bad breads):

1) Supply user with a form

2) Form contains fields that the baker can fill in (form should save reoccurring data to save time)

2a) (Optional) Have it so the tool can query a given bread URL to parse in order to pre-load/pre-generate the data in the fields

3) These fields are then used to compile either one or a series of text documents (.txt will suffice) that contain text that can be copy-pasted right into the comment box on 8chan.

This should give you guys some idea how to bring automation to your investigations and research. Once you can filter out the shills entirely, it's game over for them.


21d2e8  No.2638983

Anon who created the ToastMaster script:

What do you think about adding some sort of notification to the toast when Q posts in the breads shown? Maybe add underneath the bread number "(Q: [x number of posts])" or something along those lines?


44236a  No.2659375

>>2352645

>distributing it in an *apparently* safe and anonymous way

How do you manage the anonymous part?

I'm doing something similar, but it's a couple weeks out of date at the moment. I'm out of the country attending to a family emergency, and I couldn't take my production system with me.


9fc25d  No.2662002

>>2352645

Concur with the MD5 flaw.

It's strongly recommended you use a variety of hashes on your software, not merely whatever happens to be trendy. Hashes suffer from the Shannon problem (long story short: loss of resolution in data means substantially less accurate), which is why you should employ multiple hashes which means an attacker needs to not only pwn one hash, but several.

It becomes easier for an attacker to then just modify your hashes without you looking (rather than modify the code to fit the hash), at which point you have to make sure you keep backups of said hashes.

Done correctly, you will have enough hashes from enough algorithms that it's impossible to tamper with the code without tripping one or the other. MD5 and SHA1 are broken, but you can still use them… in conjunction with other non-broken hashes.

Sure, it's extra work, but it offers immunity to your reputation being compromised if it gets subverted.

>How do you manage the anonymous part?

PGP

Assuming you don't bury your identity somewhere in the PGP message (which should also contain the hashes). Of course, that introduces a reputation problem. You either have to trade a loss of trust for anonymity, or offer identity with reputation to engage in trust.

To be honest, I wouldn't recommend identifying yourself anyway, because even if you did, it's unlikely you have the reputational backing for it (if new) and it'd give too many clues if you're an 'old hand' (with a good rep).

Best to teach anons how to proofread, scruntise your code, make it open source, explain each line of code. Make the trust in the code, not you.


8ce561  No.2734127

>>2659375

For smallish stuff there is pastebin. If necessary, archive, encode as base64, paste with instructions at the top.

Larger files (~4MB) can be attached to posts here.

Tor may help. Or a VPN if you were able to open an account anonymously (prepaid credit card, fake identity if legal).

I do not know if Mega can be trusted. I would not trust AWS (S3).

Regarding the related problem of anonymous hosting (i.e., providing services anonymously), I've been thinking about writing a client to (ab)use 8chan as an anonymous communication/storage backend (hopefully with Ron's blessing). There are neat things to do in that direction, including anonymous software distribution.

>>2662002

I see how PGP helps with trust, but I do not see how it helps with anonymity. Can you explain what you had in mind?


7b8328  No.2752449

>I see how PGP helps with trust, but I do not see how it helps with anonymity. Can you explain what you had in mind?

You'd need a foster an alias that has a proven track record of reliability (like how I continuously try to post under this static 'name').

Once it's established as being reliable, PGP would allow you to prove it comes from that alias.

You'd need to be extremely careful not to connect your alias to your real self (I'm not worried about mine being connected, in-fact, it's important that it remains connected).

Nothing that can be done overnight, unfortunately, but that's how it is with trust, has to be (re-)earned.


d62bc5  No.2784392

Hey guise I've a question. Could someone write css to hide the name, subject and email fields on the qresearch board please. If BO could add it, it would save anons doxxing themselves and possibly hinder bots. Thanks in advance.


d1575b  No.2870643

>>2599230

Great post.


d1575b  No.2870793

Can we get the newest userjs posted here? At least the one with the PostCount.


1fcd70  No.2871218

>>2371963

This is probs a unique feel


07f8e1  No.2879212

>>2870793

https://pastebin.com/GCNvHRYv


1ecb50  No.2880218

>>2366039

Definitely a second on SSB. Also solves the archive everything offline issue Q keeps mentioning as well as provides advanced filtering.

If you mirror QResearch with SSB, you've got a winner!


d1575b  No.2882075

>>2879212

Perfect. Thanks, anon.


843ea1  No.2899194

>>2784392

Obsfucation is not security.

What's to stop someone opening up the dev window of a web browser and disabling/editing the CSS file?

Besides, Q needs access to the name field by default (and personally I would just keep appending my name to the bottom of my posts if the name field was disabled).

JavaScript for disabling email:

document.getElementsByName("email")[0].style.display = "none";

CSS selector:

input[type="email"]

{

display:none;

}

You're welcome. Leave the name field in please (maybe just append 'optional' to the name field?).


843ea1  No.2899223

To specify a placeholder attribute (as we cannot write directly to the label) using JavaScript:

document.getElementsByName("name")[0].placeholder = "Optional";

Note, both this and the prior code assume there is only one name="email" and name="name" element, and that it's the first element in the list.


ea40ba  No.2942084

new toastmaster

https://pastebin.com/JSRPEVLh


c515d7  No.3036792

>>2352495

>/tech/res/917924.html

Break their chains and spread this offline.


d1575b  No.3055030

File: f6a2ab6bb73c9b3⋯.jpg (410.63 KB, 2000x2476, 500:619, f6a2ab6bb73c9b3574e93a7b87….jpg)


8cc031  No.3056882

>>2377944

>2) Use anything stronger than MD5 for a hash. Avoid anything with the NSA's rubberstamp of approval (that means no SHA256 etc).

NSA works for Q. SHA-256 is fine; SHA-512 is faster on modern desktop computers.


d1575b  No.3101611

>>2942084

Is this still the latest toastmaster?




[Return][Go to top][Catalog][Nerve Center][Cancer][Post a Reply]
[]
[ / / / / / / / / / / / / / ] [ dir / acme / agatha2 / animu / arepa / asmr / games / leftpol / mde ]