Adds support for encoding (and optionally encrypting) user config values as
a single string that can be passed to any endpoint with the "preferences" url
param.
Co-authored-by: Ben Busby <contact@benbusby.com>
PEP-8 enforcements in the project are more of an annoyance than
anything. It doesn't really seem to add much value, and adds a lot of
friction to pull requests from developers who aren't familiar with the
style guide. Stylistic enforcements should just be done during PRs if
necessary (or a different style guide should be enforced).
Farside can now redirect quora links to querte instances and imdb links
to libremdb instances. This updates Whoogle to perform link replacements
for both services when site alts are configured.
Allows skipping the Tor startup script if WHOOGLE_TOR_SERVICE is set to
0. This is separate from WHOOGLE_CONFIG_TOR, which only allows
enabling/disabling user configuration of passing searches through
Tor.
Closes#631
Added password authentication for tor control port.
For user configuration of access to tor control port. This file should be
heavily restricted in file system.
Co-authored-by: MadcowOG <madcowog@Arch-Main.localdomain>
* Relativization of search results
* Fix JavaScript error when opening images
* Replace single-letter logo and remove sign-in link
* Add `WHOOGLE_URL_PREFIX` env var to support relative path redirection
The `WHOOGLE_URL_PREFIX` var can now be set to fix internal app
redirects, such as the `/session` redirect performed on the first visit
to the Whoogle home page.
Co-authored-by: Ben Busby <contact@benbusby.com>
Wikipedia, imgur, and translate alternatives were all still using
hardcoded URLs when replaced with their respective alternative frontend.
This updates them to use farside instead.
Also removes fosshost instance from readme
From @benbusby:
I'm unable to get in touch with fosshost support about the whoogle
instance being unavailable, and am no longer interested in
maintaining the instance due to the lack of communication.
The fosshost team decommissioned the region that Whoogle was hosted in,
but hasn't provided an option to transfer the domain record to the new VM. Until
that is fixed, the instance is inaccessible.
* Integrate Farside into Whoogle
When instances are ratelimited (when a captcha is returned instead of
the user's search results) the user can now hop to a new instance via
Farside, a new backend service that redirects users to working instances
of a particular frontend. In this case, it presents a user with a
Farside link to a new Whoogle (or Searx) instance instead, so that the
user can resume their search.
For the generated Farside->Whoogle link, the generated link includes the
user's current Whoogle configuration settings as URL params, to ensure a
more seamless transition between instances. This doesn't translate to
the Farside->Searx link, but potentially could with some changes.
* Expand conversion of config<->url params
Config settings can now be translated to and from URL params using a
predetermined set of "safe" keys (i.e. config settings that easily
translate to URL params).
* Allow jumping instances via Farside when ratelimited
When instances are ratelimited (when a captcha is returned instead of
the user's search results) the user can now hop to a new instance via
Farside, a new backend service that redirects users to working instances
of a particular frontend. In this case, it presents a user with a
Farside link to a new Whoogle (or Searx) instance instead, so that the
user can resume their search.
For the generated Farside->Whoogle link, the generated link includes the
user's current Whoogle configuration settings as URL params, to ensure a
more seamless transition between instances. This doesn't translate to
the Farside->Searx link, but potentially could with some changes.
Closes#554Closes#559
The default CSP is only helpful for some, and can break instances for
others. Since these aren't always necessary and are occasionally set by
the user's preferred reverse proxy, it is being disabled unless
explicitly enabled by setting `WHOOGLE_CSP`.
Fixes#493
https://whoogle.fossho.st is now an "official" public instance of
Whoogle, since it is the only instance maintained and validated by
the developer(s) of Whoogle (currently only me).
Closes#533
Activating minimal mode should also remove all collapsed sections, if
any are found.
WHOOGLE_MINIMAL now documented in readme and app.json (for heroku).
The "no JS" and "no cookies" portions of the readme warranted further
explanation. Since Whoogle uses JS and server-side cookies, it might be
confusing to a passerby what is actually meant by this.
Note that both JS and cookies can be blocked and Whoogle will still be
able to perform searches perfectly well.
Also updated the "theme" feature description
scribe.rip is a privacy respecting front end for medium.com. This
feature allows medium.com results to be replaced with scribe.rip links,
and works for both regular medium.com domains as well as user specific
subdomains (i.e. user.medium.com).
[scribe.rip website](https://scribe.rip)
[scribe.rip source code](https://git.sr.ht/~edwardloveall/scribe)
Co-authored-by: Ben Busby <noreply+git@benbusby.com>
Introduces a new config element and environment variable
(WHOOGLE_CONFIG_THEME) for setting the theme of the app. Rather than
just having either light or dark, this allows a user to have their
instance use their current system light/dark preference to determine the
theme to use.
As a result, the dark mode setting (and WHOOGLE_CONFIG_DARK) have been
deprecated, but will still work as expected until a system theme has
been chosen.
* Add support for Lingva translations in results
Searches that contain the word "translate" and are normal search queries
(i.e. not news/images/video/etc) now create an iframe to a Lingva url to
translate the user's search using their configured search language.
The Lingva url can be configured using the WHOOGLE_ALT_TL env var, or
will fall back to the official Lingva instance url (lingva.ml).
For more info, visit https://github.com/TheDavidDelta/lingva-translate
* Add basic test for lingva results
* Allow user specified lingva instances through csp frame-src
* Fix pep8 issue
The requests library requires both 'http' and 'https' values in any
included proxy dict, and whoogle was previously copying the http proxy
to https for simplicity. The assumption was that if the underlying
request wasn't able to connect via https, it would default to http
(otherwise why have the requirement to specify both?)
This led to connectivity issues for users with http only proxies as of
the latest urllib and requests package versions, which are a lot more
strict with connections over https. With the latest versions, if an
https connection cannot be made, the library returns an error.
As a result, the new proxy dict must look something like this for plain
http proxies:
{'http': 'http://domain.tld:port', 'https': 'http://domain.tld:port'}
where both http and https are identical, but both are still required.
* Replace hardcoded strings using translation json file
This introduces a new "translations.json" file under app/static/settings
that is loaded on app init and uses the user config value for interface
language to determine the appropriate strings to use in Whoogle-specific
elements of the UI (primarily only on the home page).
* Verify interface lang can be used for localization
Check the configured interface language against the available
localization dict before attempting to use, otherwise fall back to
english.
Also expanded language names in the languages json file.
* Add test for validating translation language keys
Also adds Spanish translation to json (the only non-English language I
can add and reasonably validate on my own).
* Validate all translations against original keyset, update readme
Readme has been updated to include basic contributing guidelines for
both code and translations.
* add view image option
* prevent whoogle links from opening in a new tab.
* remove view image template on mobile requests
* change loop values to be more robust to the number of images
* Update app/templates/imageresults.html
* fix "Basically the .cvifge class needs width: 100%; in order to expand the search input to fit the form width."
* Update app/templates/imageresults.html
* remove hardcoded string from template
* Add view image config var to app.json
* Add view image config var to whoogle.env
Co-authored-by: jacr13 <ramos.joao@protonmail.com>
Co-authored-by: Ben Busby <benbusby@protonmail.com>
* Block websites in search results via user config
Adds a new config field "Block" to specify a comma separated list of
websites to block in search results. This is applied for all searches.
* Add test for blocking sites from search results
* Document WHOOGLE_CONFIG_BLOCK usage
* Strip '-site:' filters from query in header template
The 'behind the scenes' site filter applied for blocked sites was
appearing in the query field when navigating between search categories
(all -> images -> news, etc). This prevents the filter from appearing in
all except "images", since the image category uses a separate header.
This should eventually be addressed when the image page can begin using
the standard whoogle header, but until then, the filter will still
appear for image searches.
The search language is now set using the WHOOGLE_CONFIG_SEARCH_LANGUAGE
environment variable. Interface language is still set using
WHOOGLE_CONFIG_LANGUAGE.
Fixes#260
This introduces a set of environment variables that can be used for
defining initial config state, to expedite the process of
destroying/relaunching instances quickly with the same settings every
time.
Closes#228Closes#195
This allows the user to enable their preferred settings in a variety of
ways, depending on their deployment preference. Values added to
whoogle.env can be enabled using WHOOGLE_DOTENV=1, in which case all
values in the env var file will overwrite defaults or user provided
settings.
Co-authored-by: Ben Busby <benbusby@protonmail.com>
The Heroku quick deploy branch has been changed from heroku-app to
heroku-app-beta, since a lot of users want to use features from develop
in their heroku instances.
The environment variables in the readme were updated to include the
reddit redirect var.
* Adds the ability to redirect reddit.com to libredd.it using the existing
"site alts" config setting.
This adds the WHOOGLE_ALT_RD environment variable for optionally
redirecting reddit links to libreddit
(https://github.com/spikecodes/libreddit).
* Include libreddit in home page site alt note
Repl.it has deprecated the anonymous experience so I have removed the part of the README stating that users can host on Repl.it without an account. -- @spikecodes
* Add ability to configure site alts w/ env vars
Site alternatives (i.e. twitter.com -> nitter.net) can now be configured
using environment variables:
WHOOGLE_ALT_TW='nitter.net' # twitter alt
WHOOGLE_ALT_YT='invidio.us' # youtube alt
WHOOGLE_ALT_IG='bibliogram.art/u' # instagram alt
Updated testing to confirm results have been modified.
* Add site alt vars to docker settings and readme
Environment variables should by default be disabled, since they are
optional and need further configuration by the user before enabling.
Readme was updated to reflect this approach, as well as moving the
documentation for the variables a bit lower and properly linking to them
in other areas of the readme.
* Add tor and http/socks proxy support
Allows users to enable/disable tor from the config menu, which will
forward all requests through Tor.
Also adds support for setting environment variables for alternative
proxy support. Setting the following variables will forward requests
through the proxy:
- WHOOGLE_PROXY_USER (optional)
- WHOOGLE_PROXY_PASS (optional)
- WHOOGLE_PROXY_TYPE (required)
- Can be "http", "socks4", or "socks5"
- WHOOGLE_PROXY_LOC (required)
- Format: "<ip address>:<port>"
See #30
* Refactor acquire_tor_conn -> acquire_tor_identity
Also updated travis CI to set up tor
* Add check for Tor socket on init, improve Tor error handling
Initializing the app sends a heartbeat request to Tor to check for
availability, and updates the home page config options accordingly. This
heartbeat is sent on every request, to ensure Tor support can be
reconfigured without restarting the entire app.
If Tor support is enabled, and a subsequent request fails, then a new
TorError exception is raised, and the Tor feature is disabled until a
valid connection is restored.
The max attempts has been updated to 10, since 5 seemed a bit too low
for how quickly the attempts go by.
* Change send_tor_signal arg type, update function doc
send_tor_signal now accepts a stem.Signal arg (a bit cleaner tbh). Also
added the doc string for the "disable" attribute in TorError.
* Fix tor identity logic in Request.send
* Update proxy init, change proxyloc var name
Proxy is now only initialized if both type and location are specified,
as neither have a default fallback and both are required. I suppose the
type could fall back to http, but seems safer this way.
Also refactored proxyurl -> proxyloc for the runtime args in order to
match the Dockerfile args.
* Add tor/proxy support for Docker builds, fix opensearch/init
The Dockerfile is now updated to include support for Tor configuration,
with a working torrc file included in the repo.
An issue with opensearch was fixed as well, which was uncovered during
testing and was simple enough to fix here. Likewise, DDG bang gen was
updated to only ever happen if the file didn't exist previously, as
testing with the file being regenerated every time was tedious.
* Add missing "@" for socks proxy requests
Arya K
João
Ben Busby
Kian-Meng Ang
J2D9
MadcowOG
PrivacyDevel
Nico
gdm85
domokosdcs0
Kainoa Kanter
Esmail EL BoB
DUO Labs
Albony Cal
Nuño Sempere
glitsj16
Ming Di Leom
alefvanoon
Ben Busby
Robert Blaine
KokoTheBest
fredster33
Vansh Comar
Oscar
drugal
Peter Dave Hello
FlawCra
Flux Industries
Ben Busby
Myzel394
Jerome Gravel-Niquet
Exonip
FlawCra
igosad
Shimul
FireMasterK
Spike
Ben Busby
Paolo Basso