Commit Graph

458 Commits (fd20135af0619de43fae0cb9ff08f4517d90389a)

Author SHA1 Message Date
Ben Busby 0e5630f33a
Add ability to listen on unix sockets
Introduces a way to tell the app to listen on unix socket instead of
host:port.

Fixes #436
2022-04-06 14:11:52 -06:00
Ben Busby 797372ecaa
Ignore blank alts if site alt config is enabled
If the alt for a particular service is blank, the original source is
used instead.

Example:
1. Site alts enabled in config
2. User wants wikipedia links, not wikiless
3. WHOOGLE_ALT_WIKI set to ""
4. All available alt links redirected to farside, except wikipedia

Fixes #704
2022-03-30 14:46:33 -06:00
green1052 0d6901aaa2
Add korean translation (#700) 2022-03-28 10:11:57 -06:00
138138138 5ecd4fe931
Add "nofollow noopener noreferrer" to all links (#698)
Old iOS 12 devices will pass the Referer HTTP header to the site user clicks.
Websites will know those traffic come from Whoogle search.
Adding "nofollow noopener noreferrer" solves the issue.
2022-03-28 10:11:09 -06:00
xatier e575fad324
Fix incorrect translation (zh-TW & zh-CN) (#697)
Translation for `maps` and `videos` were swapped in this commit.

11099f7b1d (diff-fcd1e088df6519cbd45d012f89a0d2722b7414c94189ee41595a3a101b4c11ad)
2022-03-28 10:10:18 -06:00
Ben Busby f5c47234de
Fix time filter background color
The time filter (past day/hour/month/etc) was using the result element
background color instead of the page background color, which wasn't
providing enough contrast with the default text color.
2022-03-25 12:14:57 -06:00
Ben Busby 0048c2f9aa
Update remaining alternative frontends to use Farside
Wikipedia, imgur, and translate alternatives were all still using
hardcoded URLs when replaced with their respective alternative frontend.
This updates them to use farside instead.
2022-03-21 10:08:52 -06:00
Ben Busby a58f70ca7e
Fix wikipedia->wikiless domain replacement
Was previously using wikipedia.com not wikipedia.org, causing wikiless
replacements to not occur.

Fixes #686
2022-03-21 10:01:21 -06:00
Ben Busby 2a0ad8796c
Switch to defusedxml for xml parsing
xml.etree.ElementTree.fromstring is considered insecure, see:
https://docs.python.org/3/library/xml.etree.elementtree.html

The defusedxml package contains several Python-only workarounds and
fixes for denial of service and other vulnerabilities in Python's XML
libraries: https://github.com/tiran/defusedxml

Fixes #670
2022-03-01 12:54:32 -07:00
Ben Busby f7e3650728
Only remove G links in footer
Links that were directed at G domains were previously removed
universally, when really they only needed to be removed from the footer
to reduce possible confusion caused by mixed Whoogle and G links.

Fixes #656
2022-03-01 12:48:33 -07:00
Ben Busby 69f845a047
Add test for empty bang behavior
Also fix pep8 issue
2022-03-01 12:13:40 -07:00
Ben Busby 809520ec70
Fallback to home page for empty bang searches
Bang searches without an actual query (i.e. just searching "!gh") will
now redirect to the home page. I guess people do this for some reason
and don't like that it redirects to the correct bang result URL, but
without an actual search term.

Fixes #595
2022-03-01 12:06:59 -07:00
Ben Busby b28fa86e33
Update ad filter
Recent changes to ads in search results caused Whoogle to display ads
for certain searches. In particular, ads recently started appearing
grouped into one div, as opposed to a singular ad per div. This was
accompanied by the div label "ads" (instead of just "ad"), which threw
off the existing ad filter. The ad keyword blacklist has been updated
accordingly, and has been enhanced to only check against alpha chars for
each label.

This only seems to have affected English language searches, and only for
very specific searches.
2022-02-25 23:02:58 -07:00
Ben Busby 9984158ec1
Ensure valid str->float conv in currency calc
Currency amounts returned by google seem to randomly include unicode
chars ('\xa0' noted in #642) which broke the currency calculator
included in the project. This ensures that only strings that can be
converted to float are ever used in the conversion.

Fixes #642
2022-02-17 16:33:44 -07:00
Nitish Yadav 0e711beca7
Give `Accept-Language` div its own class (#659)
Fixes accidental assignment of "get-only" class to the
"Accept-Language" config option
2022-02-16 09:23:38 -07:00
Ben Busby 23402e27e1
Check for updates using 24 hour time delta
Rather than only checking for an available update on app init, the check
for updates now performs the check once every 24 hours on the first
request sent after that period.

This also now catches the requests.exceptions.ConnectionError that is
thrown if the app is initialized without an active internet connection.

Fixes #649
2022-02-14 12:19:02 -07:00
Ben Busby d33e8241dc
Fix "my ip" search regression
Removes dependency on class names for creating the "my ip" info card in
the results list for searches pertaining to the user's public IP.

Adds test to prevent this from happening again.

Note to anyone reading this and looking to contribute: please avoid
using hardcoded class names at all costs. This approach of
creating/removing content just results in issues if/when Google decides
to introduce/remove class names from the result page.

Fixes #657
2022-02-14 11:40:11 -07:00
DUO Labs b2c048af92
Fix `collapse_sections` for `MINIMAL_MODE` (#654) 2022-02-11 14:44:08 -07:00
DUO Labs 7c5094d37b
Check for soup body in `remove_site_blocks` (#651)
Fixes error with `remove_site_blocks` in the Images tab
2022-02-11 14:42:11 -07:00
DUO Labs 502067addc
Clean "Show more results" of all site blocks (#646) 2022-02-08 10:57:00 -07:00
Joao A. Candido Ramos 11099f7b1d
Use consistent header for all result types (#535)
Introduces a header for switching between result types (i.e. "All", "News",
etc) that is consistent between the different result types. Previously, image
results had a tab header that was formatted in a drastically different manner,
which was jarring when switching from a different result page to the Images
page.

Created a G class enum to reference class names returned in search
results. As noted in the class doc, this should only be used/updated as
a last resort, as class names change frequently. For some instances,
such as replacing the tbm tab, it's a lot easier to just replace by
header name than attempting to replace it based on how the element is
structured.

Also updated a few styles to revert the latest styling changes being
applied by Google.

Co-authored-by: jacr13 <ramos.joao@protonmail.com>
Co-authored-by: Ben Busby <contact@benbusby.com>
2022-02-07 10:47:25 -07:00
සයුරි | Sayuri 4aa94a5d75
Fix Sinhala translation for farside search (#594) 2022-02-04 16:16:56 -07:00
DUO Labs 500942cb99
Update minimal mode for new Google formatting (#637)
Google's latest formatting changes broke the modifications made when enabling
`WHOOGLE_MINIMAL`. This updates the result filtering to work with the new
changes.

Fixes #634
2022-02-02 12:57:05 -07:00
Ben Busby b393e68d1d
Fix incorrect min-width for mobile screen sizes
min-width was previously set to 736px for all screen sizes, which forced
content off screen for smaller devices such as mobile phones. This
modifies the search stylesheet to only apply a min-width style to
devices > 800px wide.
2022-02-01 20:36:53 -07:00
Ben Busby e3394e29dd
Amend body width formatting in search css
`min-width` is a better field to override than `max-width`, since some
users prefer full width results.
2022-02-01 17:24:12 -07:00
Ben Busby 9ba73331aa
Override new Google search result formatting
There have been some recent formatting changes made by Google for search
results that do not look good (especially for dark themes). This
mostly overrides those styles to resemble the original Whoogle
result formatting.
2022-02-01 17:15:48 -07:00
Ben Busby 33f56bb0cb
Read `WHOOGLE_CONFIG_DISABLE` var as bool in app init
Fixes #636, which pointed out that the var was being interpreted as
"active" (config hidden) regardless of the value that was set.
2022-02-01 15:29:22 -07:00
Ben Busby 1af4566991
Bump version to 0.7.1 2022-01-26 10:41:41 -07:00
Ben Busby 863cbb2b8d
Remove trailing whitespace 2022-01-25 12:31:19 -07:00
Ben Busby 72e5a227c8
Move bangs init to bg thread
Initializing the DDG bangs when running whoogle for the first time
creates an indeterminate amount of delay before the app becomes usable,
which makes usability tests (particularly w/ Docker) unreliable. This
moves the bang json init to a background thread and writes a temporary
empty dict to the bangs json file until the full bangs json can be used.
2022-01-25 12:28:06 -07:00
Nitish Yadav fc50359752
Improve formatting of collapsible infobox (#612) 2022-01-18 13:47:35 -07:00
DUO Labs 257e3f33ef
Skip loading autocomplete.js if `WHOOGLE_AUTOCOMPLETE=0` (#611)
Bypasses autocomplete.js if `WHOOGLE_AUTOCOMPLETE` is set to 0
2022-01-18 13:39:56 -07:00
DUO Labs 74cb48086c
Introduce site alts for imgur and wikipedia (#609)
* Add `WHOOGLE_ALT_IMG` for a replacement for imgur.

* Add `WHOOGLE_ALT_WIKI` for Wikipedia
2022-01-14 09:59:03 -07:00
Ben Busby ded787547a
Exclude opensearch route from session validation
Fixes #588
2022-01-11 10:50:35 -07:00
Ben Busby f4b65be876
Catch invalid XML in suggestion response
As reported in #593, the XML response body returned for search
suggestions can apparently contain invalid XML elements. This catches
the error and returns an empty suggestion list instead of erroring.

Fixes #593
2021-12-28 11:38:18 -07:00
Ben Busby 8c92b381a2
Remove default country param
The country URL param ('gl') is no longer set to 'US' by default, and is
omitted from the search entirely unless explicitly set by the user. This
change was made in an attempt to cut back on the number of captchas
experienced by certain users self-hosting who experienced a decreased
amount of captchas when this configuration setting was removed.

Fixes #558
2021-12-23 17:01:49 -07:00
Ben Busby d02a7d90b9
Use UTF-8 encoding when loading json files
Fixes #581
2021-12-21 14:11:55 -07:00
Ben Busby 6d9df65d02
Catch `FileNotFound` when clearing invalid sessions
The server now consumes the FNF error if an invalid session is found but
is deleted in an earlier thread.

Fixes #577
2021-12-21 14:03:24 -07:00
Ben Busby 3d8da1db58
Bump version to 0.7.0 2021-12-08 17:57:22 -07:00
Ben Busby 634d179568
Use farside.link for frontend alternatives in results (#560)
* Integrate Farside into Whoogle

When instances are ratelimited (when a captcha is returned instead of
the user's search results) the user can now hop to a new instance via
Farside, a new backend service that redirects users to working instances
of a particular frontend. In this case, it presents a user with a
Farside link to a new Whoogle (or Searx) instance instead, so that the
user can resume their search.

For the generated Farside->Whoogle link, the generated link includes the
user's current Whoogle configuration settings as URL params, to ensure a
more seamless transition between instances. This doesn't translate to
the Farside->Searx link, but potentially could with some changes.

* Expand conversion of config<->url params

Config settings can now be translated to and from URL params using a
predetermined set of "safe" keys (i.e. config settings that easily
translate to URL params).

* Allow jumping instances via Farside when ratelimited

When instances are ratelimited (when a captcha is returned instead of
the user's search results) the user can now hop to a new instance via
Farside, a new backend service that redirects users to working instances
of a particular frontend. In this case, it presents a user with a
Farside link to a new Whoogle (or Searx) instance instead, so that the
user can resume their search.

For the generated Farside->Whoogle link, the generated link includes the
user's current Whoogle configuration settings as URL params, to ensure a
more seamless transition between instances. This doesn't translate to
the Farside->Searx link, but potentially could with some changes.

Closes #554

Closes #559
2021-12-08 17:27:33 -07:00
Vansh Comar 7bea6349a0
Add tools for currency conversion in search results (#536)
This implements a method for converting between various currencies. When a user
searches "<currency A> to <currency B>" (including when prefixed by a specific
amount), they are now presented with a table for quickly converting between the
two. This makes use of the currency ratio returned as the first "card" in
currency related searches, and the table is inserted into this same card.
2021-12-06 22:56:13 -07:00
Ben Busby 10a15e06e1
Fix incorrect request type for image searches
Previously had hardcoded POST requests for all requests that didn't use
the header template (which currently is only the image tab).

Also refactored how the Filter class works. It now requires a valid
Config model to be provided, which is then set up as a class var that
the filtering functions can use as needed, rather than setting specific
values from the config as individual values (which was confusing and
sloppy).

Fixes #561
2021-12-06 21:39:50 -07:00
Ben Busby b75ff0782d
pep8: fix CSP header line length 2021-11-29 15:58:19 -07:00
Ben Busby 3e20788857
Disable in-app CSP unless enabled via WHOOGLE_CSP
The default CSP is only helpful for some, and can break instances for
others. Since these aren't always necessary and are occasionally set by
the user's preferred reverse proxy, it is being disabled unless
explicitly enabled by setting `WHOOGLE_CSP`.

Fixes #493
2021-11-29 15:52:28 -07:00
Ben Busby f73e4b9239
Fix height for homepage logo 2021-11-29 15:34:13 -07:00
Ben Busby 27051363ff
Adjust logo css for mobile devices
Fixes #557
2021-11-27 20:03:06 -07:00
Ben Busby 9c96f0fd57
Improve default response headers
Reponse headers now include the following:
- X-Content-Type-Options: nosniff
- X-Frame-Options: DENY
- Strict-Transport-Security: max-age=63072000
  - Only when HTTPS_ONLY is set

https://infosec.mozilla.org/guidelines/web_security#http-strict-transport-security
https://infosec.mozilla.org/guidelines/web_security#x-content-type-options
https://infosec.mozilla.org/guidelines/web_security#x-frame-options
2021-11-26 08:38:26 -07:00
Ben Busby 73f631b1f9
Import logo stylesheet before applying custom css
This fixes #551, and allows custom css to be applied to the Whoogle
logo.
2021-11-24 12:38:56 -07:00
Ben Busby 3c06519130
Use 'gl' search param to set country
This switches the param used for the "country" config setting from "cr"
(which only filters results by the country the result is hosted in) to
"gl" (which overrides server/hosting location and produces results that
are more accurate for the user's current country).

Before this change, the country config setting was (imo) pretty useless.
Allowing a user to override an instance's hosting location with their
preferred country though is way more useful, especially for public
instances that are hosted in a different country than the user.

Closes #544
2021-11-23 13:48:54 -07:00
Ben Busby 1d3e7c0255
Pin config buttons to bottom of config menu
Previously the load/save/apply buttons in the config menu were hidden
below all available config options and required the user to scroll to
the bottom to save changes. This made for bad ux, since for new users,
it isn't immediately apparent that selecting a new dropdown value, for
instance, doesn't instantly save the new setting. The new layout should
make it more clear that hitting "Apply" is required to save config
changes.
2021-11-23 12:27:59 -07:00
Ilya Prokopenko 79a4a17311
Add Russian translation (#552) 2021-11-23 10:36:52 -07:00
Ben Busby 5a27d748d1
Create separate test workflow for docker
This expands on the current testing suite a bit by introducing a new
workflow for testing functionality within the docker container. It runs
the same test suite as the regular "test" workflow, but also performs a
health check after running the app for 10 seconds to ensure
functionality.

The buildx workflow now waits for the docker test script to finish
successfully, rather than the regular test workflow. This will hopefully
avoid situations where new images are pushed with issues that aren't
detected in regular testing of the app.
2021-11-22 00:26:25 -07:00
Ben Busby 6f5f3d8ca7
Fix incorrect redirect protocol used by Flask
Flask's `request.url` uses `http` as the protocol, which breaks
instances that enforce `https`, since the session redirect relies on
`request.url` for the follow-through URL.

This introduces a new method for determining the correct URL to use for
these redirects by automatically replacing the protocol with `https` if
the `HTTPS_ONLY` env var is set for that instance.

Fixes #538

Fixes #545
2021-11-21 23:21:04 -07:00
Ben Busby 0c5578937e
Remove 308 redirect for http->https
HTTPS upgrades should be handled outside of Whoogle, since Flask often
doesn't detect the right protocol when being used behind a reverse proxy
such as Nginx.
2021-11-20 16:43:57 -07:00
Ben Busby de28e06d8f
Improve cookie security when `HTTPS_ONLY` is set
Adds the "Secure" flag and "__Secure-" prefix if the `HTTPS_ONLY`
environment variable is enabled.

Fixes #539
2021-11-20 16:34:37 -07:00
Ben Busby e06ff85579
Improve public instance session management (#480)
This introduces a new approach to handling user sessions, which should
allow for users to set more reliable config settings on public instances.

Previously, when a user with cookies disabled would update their config,
this would modify the app's default config file, which would in turn
cause new users to inherit these settings when visiting the app for the
first time and cause users to inherit these settings when their current
session cookie expired (which was after 30 days by default I believe).
There was also some half-baked logic for determining on the backend
whether or not a user had cookies disabled, which lead to some issues
with out of control session file creation by Flask.

Now, when a user visits the site, their initial request is forwarded to
a session/<session id> endpoint, and during that subsequent request
their current session id is matched against the one found in the url. If
the ids match, the user has cookies enabled. If not, their original
request is modified with a 'cookies_disabled' query param that tells
Flask not to bother trying to set up a new session for that user, and
instead just use the app's fallback Fernet key for encryption and the
default config.

Since attempting to create a session for a user with cookies disabled
creates a new session file, there is now also a clean-up routine included
in the new session decorator, which will remove all sessions that don't
include a valid key in the dict. NOTE!!! This means that current user
sessions on public instances will be cleared once this update is merged
in. In the long run that's a good thing though, since this will allow session
mgmt to be a lot more reliable overall for users regardless of their cookie
preference.

Individual user sessions still use a unique Fernet key for encrypting queries,
but users with cookies disabled will use the default app key for encryption
and decryption.

Sessions are also now (semi)permanent and have a lifetime of 1 year.
2021-11-17 19:35:30 -07:00
Joao A. Candido Ramos 1f18e505ab
Include "chips" param in image search (#534)
"chips" is used in image tabs to pass the optional "filter" to add to the
given search term

Fixes #299
2021-11-17 16:17:27 -07:00
Ben Busby e93507f148
Catch connection error during Tor validation step
Validation of the Tor connection occasionally fails with a
ConnectionError from requests, which was previously uncaught. This is
now handled appropriately (error message shown and connection dropped).

Fixes #532
2021-11-12 17:19:45 -07:00
gnuhead-chieb 3f40a6c485
Add Japanese translation (#528) 2021-11-09 08:37:49 -07:00
Fabian Schilling 9ad1d60a47
Improve URL parsing for full size images (#521)
Skip URLs that are not two-element lists

Fixes #520
2021-11-02 16:22:24 -06:00
Vansh Comar 3784d897d9
Add "update available" indicator to footer (#517)
This checks the latest released version of Whoogle against
the current app version, and shows an "update available"
message if the current version num < latest release num.

Closes #305
2021-11-02 10:35:40 -06:00
Ben Busby b73c14c7cc
Set max height for config menu
The config menu has gotten out of control recently, but rather than
reducing functionality, I'm just going to set a max height for the div
and allow scrolling within the menu.

Ultimately though this indicates that the app is getting a bit too
complicated (imo). Striking a balance between customization and
minimalism is less of a priority for me nowadays though, hence why I'm
willing to let it slide for now. At some point, maybe when there are
more contributors, it could be nice to refactor this in some way so that
it isn't overwhelming to new users who are looking to customize their
instance (that's just me speculating btw, I haven't actually heard from
anyone who thinks there are too many options in that menu).
2021-11-01 16:55:33 -06:00
Ben Busby c766554eea
Bang refactor PEP-8 fix
Addresses PEP-8 formatting issue in previous commit
2021-11-01 16:53:19 -06:00
Ben Busby ddf951de35
Use `replace` in bang query formatting
Using `format` for formatting bang queries caused a KeyError for some
searches, such as !hd (HUDOC). In that example, the URL returned in the
bangs json was `http://...#{%22fulltext%22:[%22{}%22]...`, where
standard formatting would not work due to the misidentification of
"fulltext" as a formatting key.

The logic has been updated to just replace the first occurence of "{}"
in the URL returned by the bangs dict.

Fixes #513
2021-11-01 16:47:48 -06:00
gripped d1c9b7f803
Remove styling from NoJS liks (#511)
Fixes #510
2021-11-01 16:03:47 -06:00
Ben Busby 7fe066b4ea
Escape result html after bolding search terms
Fixes #518
2021-11-01 15:35:57 -06:00
gripped c2ced23073
Improve formatting with NoJS enabled (#509)
Removes line breaks, divider, and link location from all NoJS
links in results when NoJS mode is enabled
2021-10-29 09:28:05 -06:00
Ben Busby 0a78c524fa
Expand 'my ip' to work for proxied requests
Adds a check for the HTTP_X_FORWARDED_FOR header, and uses the value
from the request if found.
2021-10-28 21:31:24 -06:00
Ben Busby 26b560da1d
Pass response as str to bsoup for "my ip" card
Due to how the response is now reformed into a new bsoup object when
bolding search query terms, creating an ip card for "my ip" searches
threw an error due to how the new bsoup object was initialized for the
"my ip" card. This passes the response in as a string instead.

Fixes #504
2021-10-28 21:22:51 -06:00
Ben Busby cad1e2ab4d
Include translation mapping in nojs windows
The translation map was missing for links opened via the nojs feature,
causing a server error.

Fixes #507
2021-10-28 21:06:52 -06:00
DUO Labs 5189cdb072
Update "skip bolding" regex to fix some edge cases (#500)
Should address errors caused by the "bold query" feature replacing
tags and style elements, resulting in unformatted response pages.
2021-10-28 12:54:27 -06:00
Vansh Comar f04c7c5557
Support DDG style bangs with bang at the end (#503)
DDG style bang searches can now have the bang (!) at the end of
the search (i.e. "bologna w!" will now redirect to wikipedia just like
"bologna !w" would)
2021-10-28 12:39:33 -06:00
Ben Busby 190b684469
Reformat view templates 2021-10-27 12:30:55 -06:00
Ben Busby b96e3a0acb
Make base search url a member of the request class
Since the request class is loaded prior to values being read from the
user's dotenv, the WHOOGLE_RESULT_PER_PAGE var wasn't being used for
searches.

This moves the definition of the base search url to be intialized in the
request class to address this issue.

Fixes #497
2021-10-27 11:02:14 -06:00
DUO Labs d8dcdc7455
Skip bolding search terms that are not alphanumeric (#496)
Fixes #494
2021-10-27 10:50:21 -06:00
Ben Busby 1abd040428
Remove redundant loading of variables.css
variables.css doesn't need to be loaded by any template, since
WHOOGLE_CONFIG_STYLE loads those values by default when not set
explicitly. Loading the stylesheet caused the logo colors to be
persistent unless set individually.

Sorry @gripped for sneaking all of this unnecessary color in...

Fixes #492
2021-10-26 21:11:46 -06:00
Ben Busby 591ed4a6d6
Use f-string in bold query regex
by @DUOLabs333
2021-10-26 16:21:30 -06:00
Ben Busby f154b5f2e2
PEP-8 formatting fix 2021-10-26 16:17:38 -06:00
Ben Busby 6decab5a51
Improve regex for bolding search terms
Co-authored by @DUOLabs333
2021-10-26 16:15:24 -06:00
Ben Busby d16ef6d011
Unescape search response before rendering template
Fixes a small issue with the previous commit where bolded search terms
had the <b> tags escaped, rather than being applied as actual html.
2021-10-26 15:00:39 -06:00
DUO Labs 2c9cf3ecc6
Bold search query in results (#487)
This modifies the search result page by bold-ing all appearances
of any word in the original query. If portions of the query are in
quotes (i.e. "ice cream"), only exact matches of the sequence of
words will be made bold.

Co-authored-by: Ben Busby <noreply+git@benbusby.com>
2021-10-26 14:59:23 -06:00
Ben Busby 90441b2668
Add WHOOGLE_MINIMAL to docs, tweak min mode logic
Activating minimal mode should also remove all collapsed sections, if
any are found.

WHOOGLE_MINIMAL now documented in readme and app.json (for heroku).
2021-10-26 10:38:20 -06:00
DUO Labs 543f2b2a01
Add a "minimal mode" for condensing results (#485)
If WHOOGLE_MINIMAL is set, all non-link results are
removed from the view.
2021-10-26 10:35:12 -06:00
DUO Labs 5a05bfb6de
Allow setting number of results per page (#486)
Add `WHOOGLE_RESULTS_PER_PAGE` var, allowing users to 
specify the number of results per page. The default is 10.
2021-10-26 10:28:38 -06:00
Vansh Comar 5118ddb8b8
Allow setting "Accept-Language" header (#483)
Closes #445
2021-10-25 15:49:09 -06:00
Ben Busby 91002ec6be
Update default theme css
I've gotten a bit bored of the current light/dark themes, so I'm
switching the default theme over to the Doppelganger theme, which is a
better template/jumping off point for users to use when creating custom
themes since it also provides examples for coloring each of the Whoogle
logo letters.
2021-10-23 23:56:38 -06:00
Ben Busby 8f70236403
Update domains used for scribe.rip replacements
The levelup.gitconnected.com site is a Medium site that can also be
replaced with scribe.rip whenever privacy respecting site alternatives
are enabled in the config.

Also modified how link descriptions are updated when that config is
enabled (before it was missing replacements on quite a few
descriptions).
2021-10-23 23:23:37 -06:00
Vansh Comar 771bf34ce9
Show client IP for "my ip" searches (#469)
This introduces a new UI element for displaying the client IP
address when a search for "my ip" is used.

Note that this does not show the IP address seen by Google
if Whoogle is deployed remotely. It uses `request.remote_addr`
to display the client IP address in the UI, not the actual address
of the server (which is what Google sees in requests sent from
remote Whoogle instances).
2021-10-21 10:42:31 -06:00
Yadomin 284a8102c8
Block by result title or url using regex (#473)
Allows blocking search results using a regex filter for either
result title or result url
2021-10-20 20:01:04 -06:00
Vansh Comar 79fb7531be
Implement scribe.rip replacement for medium.com results (#463)
scribe.rip is a privacy respecting front end for medium.com. This
feature allows medium.com results to be replaced with scribe.rip links,
and works for both regular medium.com domains as well as user specific
subdomains (i.e. user.medium.com).

[scribe.rip website](https://scribe.rip)
[scribe.rip source code](https://git.sr.ht/~edwardloveall/scribe)

Co-authored-by: Ben Busby <noreply+git@benbusby.com>
2021-10-16 12:22:00 -06:00
Ben Busby ee6a27e541
Add link to user css themes in config menu 2021-10-14 20:20:12 -06:00
Ben Busby ff885e4fde
Disable autocomplete via WHOOGLE_AUTOCOMPLETE var
Setting WHOOGLE_AUTOCOMPLETE to 0 now disables the autocomplete/search
suggestion feature.

Closes #462
2021-10-14 18:59:10 -06:00
Ben Busby a76d39ec86
Fix missing translations in config menu
Closes #374
2021-10-13 21:07:42 -06:00
Ben Busby 9097c3ae23
Add /home endpoint to header template
Used in header templates for navigating back to the home page when
behind a reverse proxy config where the app is running from a subpath of
a domain (i.e. "https://something/whoogle/")

Fixes #403
2021-10-13 20:55:26 -06:00
Ben Busby 334aabacb7
Bump version to 0.6.0 2021-10-11 17:44:57 -06:00
Albony Cal c89353cfec
Add hindi translation (#448) 2021-10-11 14:32:03 -06:00
rn83 f18400b1f1
Strip SKIP_PREFIX for SITE_ALTS only (#452)
Domain prefixes (www, mobile, m) are now striped for site alternatives only.
2021-10-11 14:25:21 -06:00
Ben Busby b189ea3963
Fix hardcoded search method in header template
Should use GET if user has configured "GET only" in their config

Closes #446
2021-09-29 20:40:56 -06:00
BorislavGeorgiev 10b60d9373
Add Bulgarian translation (#440) 2021-09-27 20:39:38 -06:00
BlissOWL f12b0e62c5
Make bang searches case insensitive (#438)
Bang searches now ignore the capitalization of the operator

Co-authored-by: Ben Busby <noreply+git@benbusby.com>
2021-09-27 19:39:58 -06:00
Ben Busby 27d978f232
Hide overflow on all result divs
Mostly addresses the small amount of visible overflow on sections like
"Top Stories".
2021-09-18 21:19:05 -06:00
Kang-min Liu c3fd84b942
Update name of "Taiwan" in country list (#429) 2021-09-16 16:22:44 -06:00
Kang-min Liu 5289b4ceb3
Add zh-TW translations (#428)
There are a few conventional choices but this one should be friendly
and generally accepted by local reader.

Previous version is still comprehensible but lesser users (perhaps
used in Japanese documents) and may give local users a pause.
2021-09-15 15:30:53 -06:00
Ben Busby 9f84a8ad83
Remove form action from csp
Restricting form-action to 'self' in the content security policy
prevented Chrome (and likely other browsers) from using !bangs on the
home page.

Fixes #408
2021-08-31 07:57:50 -06:00
Ben Busby ad2b2554c1
Use UTF-8 encoding when loading languages json
Fixes #371
2021-08-30 17:23:19 -06:00
alefvanoon 981c7d28f8
Add persian (farsi) translation (#400) 2021-08-30 16:17:14 -06:00
davidfrickert 71070ee921
Fix portuguese translations (#405) 2021-08-30 16:11:32 -06:00
alefvanoon be3714f074
Fix rtl lang problem in search box (#399)
Adds auto dir to index, search and header input html
2021-08-30 16:10:07 -06:00
Trottel f34490d4f1
Add Czech translation (#397) 2021-08-30 16:05:19 -06:00
Darkempire 4f5ed37c0a
Add French translation (#391) 2021-08-24 09:12:34 -06:00
gripped 8d24f8abdd
Fix white background on dropdown for result selectors, time etc (#384) 2021-08-24 09:07:34 -06:00
Laurent le Beau-Martin 1a3790c7b1
Only open external links in a new tab (#380) 2021-08-24 09:06:41 -06:00
සයුරි | Sayuri 8e91564600
Update translations (#373) 2021-07-22 09:13:09 -06:00
Ben Busby 694642ccb3
Set bg color for "top stories" elements 2021-07-05 00:18:28 -04:00
Ben Busby 38c38a772f
Find valid parent element when collapsing result content
Previously if a result element marked for collapsing didn't have a valid
"parent" element, the collapsing was skipped altogether. This loops
through child elements until a valid parent is found (or if one isn't
found, the element will not be collapsed).
2021-07-04 15:20:19 -04:00
Ben Busby 13202cc6b1
Ensure existence of static build dir 2021-07-02 16:21:38 -04:00
Ben Busby 68fdd55482
Use cache busting for css/js files
On app init, short hashes are generated from file checksums to use for
cache busting. These hashes are added into the full file name and used
to symlink to the actual file contents. These symlinks are loaded in the
jinja templates for each page, and can tell the browser to load a new
file if the hash changes.

This is only in place for css and js files, but can be extended in the
future for other file types if needed.
2021-06-30 19:00:01 -04:00
Ben Busby c41e0fc239
Allow theme to mirror user system settings
Introduces a new config element and environment variable
(WHOOGLE_CONFIG_THEME) for setting the theme of the app. Rather than
just having either light or dark, this allows a user to have their
instance use their current system light/dark preference to determine the
theme to use.

As a result, the dark mode setting (and WHOOGLE_CONFIG_DARK) have been
deprecated, but will still work as expected until a system theme has
been chosen.
2021-06-28 10:26:51 -04:00
Ben Busby afd01820bb
Collapse long result sections into details/summary elements
Sections such as "People also asked" and "related searches" typically
take up a lot of room on the results page, and don't always have the
most useful information. This checks for result elements with more than
7 child divs, extracts the section title, and wraps all elements in a
"details" element that can be expanded/collapsed by the user.

Note that this functionality existed previously (albeit not implemented
as well), but due to changes in how Google returns searches (switching
from using <h2> elements for section headers to <span> or <div>
elements), the approach to collapsing these sections needed to be
updated.
2021-06-23 18:59:57 -04:00
Ben Busby d894bd347d
Handle error when parsing image result url 2021-06-16 10:40:18 -04:00
Ben Busby b21b4f4f57
Skip parsing user agent if absent from request 2021-06-16 10:37:33 -04:00
Ben Busby bcb1d8ecc9
Add lingva translation support in search (#360)
* Add support for Lingva translations in results

Searches that contain the word "translate" and are normal search queries
(i.e. not news/images/video/etc) now create an iframe to a Lingva url to
translate the user's search using their configured search language.

The Lingva url can be configured using the WHOOGLE_ALT_TL env var, or
will fall back to the official Lingva instance url (lingva.ml).

For more info, visit https://github.com/TheDavidDelta/lingva-translate

* Add basic test for lingva results

* Allow user specified lingva instances through csp frame-src

* Fix pep8 issue
2021-06-15 10:14:42 -04:00
deluxghost 82ccace647
Add zh-CN translation (#355) 2021-06-11 11:33:01 -04:00
Aikatsui a6b4252210
Add Sinhala translation (#353) 2021-06-11 10:22:25 -04:00
Ben Busby 904091f440
Bump version to 0.5.4 2021-06-06 13:45:03 -04:00
Ben Busby 44b0fe519c
Revert changes to default language config
A recent issue brought up a good point about how the latest changes to
setting default language to english break functionality for bilingual
users. The change was likely not the best solution for users who were
being affected by IP geolocation on their instances -- the right
solution for that would be to configure the interface/search language to
their preference instead.
2021-06-06 13:39:06 -04:00
Ben Busby e7a604d428
Fix handling of http (vs https) proxy creation
The requests library requires both 'http' and 'https' values in any
included proxy dict, and whoogle was previously copying the http proxy
to https for simplicity. The assumption was that if the underlying
request wasn't able to connect via https, it would default to http
(otherwise why have the requirement to specify both?)

This led to connectivity issues for users with http only proxies as of
the latest urllib and requests package versions, which are a lot more
strict with connections over https. With the latest versions, if an
https connection cannot be made, the library returns an error.

As a result, the new proxy dict must look something like this for plain
http proxies:

{'http': 'http://domain.tld:port', 'https': 'http://domain.tld:port'}

where both http and https are identical, but both are still required.
2021-06-04 15:30:21 -04:00
Ben Busby a64a86efb6
Bump version to 0.5.3 2021-06-04 11:31:03 -04:00
Ricardo 9d024cffce
Add Portuguese translation (#345)
* Add Portuguese translation

* Update translations.json
2021-06-04 11:16:58 -04:00
Ben Busby 614dceeb70
Add fallback interface/search lang + cleanup
Since the interface language defaults to IP geolocation by google, the
default language is now set to english. Still not sure if this is the
best solution, but at least temporarily should clear up some confusion
for users with instances deployed in countries outside of their own.

Also performed some minor cleanup:
  - Updated name of strip_blocked_sites to clean_query
  - Added clean_query to list of jinja template functions
  - Ensured site block list doesn't contain duplicate filters
2021-06-04 11:09:30 -04:00
bruvv 3892355199
Add Dutch translation (#343) 2021-06-03 09:24:59 -04:00
Myzel394 7103d9eccb
Add German translation (#339)
* Added german language

* Fixed translations.json path

* Fixed German name
2021-06-01 19:57:48 -04:00
Ben Busby cbe32a081e
Hotfix: extract only 'q' element from query string
Occasionally the search results will contain links with arguments such
as 'dq', which was being erroneously used in attempts to extract the 'q'
element from query strings. This enforces that only links with '?q=' or
'&q=' (elements with a standalone 'q' arg) will have the element
extracted.

I also refactored the naming of this element once extracted to be just
'q'. Although this seems counterintuitive, it makes a little more sense
since this element is the one we're extracting. It's a vague url arg
name, but it is what it is.

Bump version to 0.5.2 for hotfix release
2021-05-29 12:22:37 -04:00
Ben Busby 43faaee77f
Hotfix: remove site filter for maps links
The new site filter breaks links to Maps results, so filter.py needed
to be updated to handle these links as a unique case. A new method was
introduced to easily remove any "-site:..." filters from the query,
which is now also used to format queries in the header template rather
than manually removing the blocked site list within the template itself.

Bumps version to 0.5.1 for releasing the bugfix

Fixes #329
2021-05-27 12:01:57 -04:00
Federico Torrielli cf55765933
Add italian localization (#327) 2021-05-25 09:51:05 -04:00
Ben Busby 4649d96dda
Support basic localization (#325)
* Replace hardcoded strings using translation json file

This introduces a new "translations.json" file under app/static/settings
that is loaded on app init and uses the user config value for interface
language to determine the appropriate strings to use in Whoogle-specific
elements of the UI (primarily only on the home page).

* Verify interface lang can be used for localization

Check the configured interface language against the available
localization dict before attempting to use, otherwise fall back to
english.

Also expanded language names in the languages json file.

* Add test for validating translation language keys

Also adds Spanish translation to json (the only non-English language I
can add and reasonably validate on my own).

* Validate all translations against original keyset, update readme

Readme has been updated to include basic contributing guidelines for
both code and translations.
2021-05-24 17:03:02 -04:00
Joao A. Candido Ramos 448efb8f2a
Add "view image" functionality (#268)
* add view image option

* prevent whoogle links from opening in a new tab.

* remove view image template on mobile requests

* change loop values to be more robust to the number of images

* Update app/templates/imageresults.html

* fix "Basically the .cvifge class needs width: 100%; in order to expand the search input to fit the form width."

* Update app/templates/imageresults.html

* remove hardcoded string from template

* Add view image config var to app.json

* Add view image config var to whoogle.env

Co-authored-by: jacr13 <ramos.joao@protonmail.com>
Co-authored-by: Ben Busby <benbusby@protonmail.com>
2021-05-21 11:19:45 -04:00
Ben Busby fcfa3783e3
Bump version to 0.5.0 2021-05-21 10:50:07 -04:00
Ben Busby d5eebe9fe5
Add iframe-able search page for insertion into other sites
Introduces a new html template, search.html, which provides a very basic
form for submitting search queries.

Closes #319
2021-05-21 10:35:46 -04:00
Ben Busby 1fdf226802
Use curl-based healthcheck w/ new non-auth route
The wget method seemed to have a possible issue with creating endless
index.html copies (despite being specified to output to console only),
so this has been updated to use curl instead.

Also uses new non-authenticated "healthz" route to perform the
healthcheck.

Fix #316

Fix #313
2021-05-18 11:48:15 -04:00
bruvv 27b6d05b6a
Fix EU consent bug (#320)
* Update request.py

* Use current date to format EU consent cookie

Co-authored-by: Ben Busby <benbusby@protonmail.com>
2021-05-18 10:52:24 -04:00
Harsh Barsaiyan 4466bbc8f4
Add divider to user-defined CSS (#310) 2021-05-11 12:26:37 -04:00
Ben Busby 05995649f3
Hotfix: check for site filters before modifying query
The previous method of removing all site filters from the search query
removed the last letter of the search. This only applies the substring
filter if any site filters are present in the query.

Fixes #306
2021-05-10 12:07:55 -04:00
Ben Busby c8da53d4b0
Block websites from search results via user config (#304)
* Block websites in search results via user config

Adds a new config field "Block" to specify a comma separated list of
websites to block in search results. This is applied for all searches.

* Add test for blocking sites from search results

* Document WHOOGLE_CONFIG_BLOCK usage

* Strip '-site:' filters from query in header template

The 'behind the scenes' site filter applied for blocked sites was
appearing in the query field when navigating between search categories
(all -> images -> news, etc). This prevents the filter from appearing in
all except "images", since the image category uses a separate header.
This should eventually be addressed when the image page can begin using
the standard whoogle header, but until then, the filter will still
appear for image searches.
2021-05-07 11:45:53 -04:00
Ben Busby a7bf9728e3
Allow 'data:' for img src in app CSP
Disallowing base64 images in the app resulted in broken image
placeholders for things like pronunciation guides, business reviews,
etc.
2021-05-05 12:51:11 -04:00
Angel Mario d6d7110e22
Add option to disable changing config from client (#295)
* Add option to disable changing of configuration

Introduces a test to ensure the correct response code is found when
attempting to update the config when disabled, and ensure default config
is unchanged when posting a new config dict.

Attempting to update the config using the API when disabled now returns
a 403 code + redirect.

Co-authored-by: Ben Busby <benbusby@protonmail.com>
2021-04-27 10:36:03 -04:00
Ben Busby 8ae7b5947e
Separate interface language from search language in env vars
The search language is now set using the WHOOGLE_CONFIG_SEARCH_LANGUAGE
environment variable. Interface language is still set using
WHOOGLE_CONFIG_LANGUAGE.

Fixes #260
2021-04-26 11:38:55 -04:00
Ben Busby f56e913521
Remove gap between input and result types
Enforces 0 margin for the search input form on the result page, which
removes the weird gap that is seen by default.

Also made minor changes to the border styling. Desktop searches now have
a single bottom border in dark mode rather than an all around border,
and the border around the mobile search result input was removed
entirely.
2021-04-22 16:24:43 -04:00
Ben Busby 5b963b441c
Focus search input after clearing w/ reset btn
See #291
2021-04-22 10:02:15 -04:00
Ben Busby 01fe0c02a5
Add button to clear search input on mobile
This was unfortunately a bit more complex than just adding an HTML reset
button, since reset buttons only "reset" input content to its original
value rather than clearing it. This doesn't work for Whoogle's needs,
since inputs on search result pages are auto populated with the search
content as their default value.

A reset button was introduced anyways, but is controlled by a few lines
of javascript to allow completely clearing the search input. The button
will only appear on mobile searches.

At the moment, it isn't particularly pretty, but is functional. It uses
just a plain "x" character and is always visible on mobile search result
pages. This leaves plenty of room for improvement moving forward.

Fixes #291
2021-04-21 11:38:19 -04:00
Ben Busby 7136197e5d
Fix missing text style for active search suggestions 2021-04-21 10:49:27 -04:00
Ben Busby 2eb33007f7
Disable autocorrect on mobile search inputs
Fixes #292
2021-04-21 10:48:26 -04:00
Ben Busby d2fac809ca
Fix mishandling of empty config environment variables
The recent change to cast bool config vars as ints to handle a '0' or
'1' value was shortsighted, since it doesn't allow for instances where
the variable is set to an empty value (or '' or any invalid/non-int
value).

This introduces a read_config_bool method for reading values that should
be a '0' or '1', but will default to False if not a digit (otherwise the
value will be cast as bool(int(value)) if "value" is a digit str).

Fixes #288
2021-04-14 10:42:41 -04:00
Ben Busby baa7a87efb
Fix incorrect config bool env var casting
Config boolean environment variables need to be cast to ints, since
they are set or unset using 0 and 1. Previously they were interpreted as
(pseudocode) read_var(name, default=False), which meant that setting
CONFIG_VAR=0 would enable that variable since Python reads environment
variables as strings, and '0' is truthy. This updates the previous logic
to (still pseudocode) int(read_var(name, default='0')).

Fixes #279
2021-04-12 16:40:59 -04:00
Ben Busby b7e48a9597
Replace remaining hardcoded theme values
Both light and dark themes have been updated to remove the leftover
hardcoded values (mostly related to the search suggestion styling).

See discussion in #247.
2021-04-12 10:22:34 -04:00
Ben Busby 1030118d0b
Expand custom css theming support
Also adds new default dark theme designed by @gripped.
2021-04-09 11:00:02 -04:00
gripped 13abb0ae7f
Add .BVG0Nb to dark-theme.css 2021-04-09 10:57:23 -04:00
Ben Busby ed32fb927c
Disable logging from imported modules
The logging from imported modules (stem, in particular) has caused quite
a few users to assume there are errors where there aren't any. The logs
from stem also aren't helpful, as everything in the library works as
expected despite the implication from the logs that it is not working.
2021-04-09 09:26:16 -04:00
Ben Busby a321d55f13
Hotfix: Send generic "Mozilla" in user agent
Randomizing the "Mozilla" portion of the user agent changed the
character encoding to GB2312. Setting it to plain "Mozilla" enforces
UTF-8 encoding.

Bump to version 0.4.1 for release of bug fix

Fixes #267
2021-04-08 09:43:41 -04:00
Ben Busby 30be540b97 Bump version to 0.4.0 2021-04-05 11:00:56 -04:00
Ben Busby 0b9600b564 Expand custom css variables and functionality
Squashed commit of the following:

commit 37e22d2945b077a94d9997d064f4355ff8819bae
Author: Ben Busby <benbusby@protonmail.com>
Date:   Mon Apr 5 10:27:05 2021 -0400

    Pass user config to logo template

commit 2406fee05c3e221112fbe802fbf2ecca1df99127
Author: Ben Busby <benbusby@protonmail.com>
Date:   Mon Apr 5 10:24:54 2021 -0400

    Fix incorrect contrast text in dark theme

commit 91dd677e22c2e99819123154e03e9f519f95a9bd
Author: Ben Busby <benbusby@protonmail.com>
Date:   Fri Apr 2 17:21:38 2021 -0400

    Remove inline onclicks, fix svg sizing

commit 91bbf9c0fae36febd6a6a0d8e6a560babe8622d5
Merge: 72637df b1227bd
Author: Ben Busby <benbusby@protonmail.com>
Date:   Fri Apr 2 15:35:37 2021 -0400

    Merge remote-tracking branch 'origin/develop' into custom-css-tweaks

commit 72637df213f4b9e83e4b58fe76973de02f63ec8e
Author: Ben Busby <benbusby@protonmail.com>
Date:   Fri Apr 2 11:38:38 2021 -0400

    Use svg logo w/ custom styling on results pages

commit 666a7ceac4a6e4d3fe1975dcee91e6094b66149e
Author: Ben Busby <benbusby@protonmail.com>
Date:   Fri Apr 2 11:10:37 2021 -0400

    Split whoogle-accent into whoogle-element-bg and whoogle-logo

    See discussion on #247
2021-04-05 11:00:56 -04:00
Ben Busby 50c888f9a7 Revert heroku app https upgrade fix 2021-04-05 11:00:56 -04:00
Ben Busby df0b7afa50 Switch to single Fernet key per session
This moves away from the previous (messy) approach of using two separate
keys for decrypting text and element URLs separately and regenerating
them for new searches. The current implementation of sessions is not very
reliable, which lead to keys being regenerated too soon, which would
break page navigation. Until that can be addressed, the single
key per session approach should work a lot better.

Fixes #250

Fixes #90
2021-04-05 11:00:56 -04:00
Ben Busby ed4432f3f8 Hotfix: Upgrade heroku apps to https for all endpoints
The previous implementation of the is_heroku check in
search.needs_https() was implemented to only match URLs ending in
'.herokuapp.com', and skipped upgrading to HTTPS for other endpoints.
2021-04-05 11:00:56 -04:00
Ben Busby 7b9ee37beb Allow defining initial config state w/ env vars
This introduces a set of environment variables that can be used for
defining initial config state, to expedite the process of
destroying/relaunching instances quickly with the same settings every
time.

Closes #228

Closes #195
2021-04-05 11:00:56 -04:00
Shimul 8a10efaa01 Allow setting environment variables in whoogle.env (#237)
This allows the user to enable their preferred settings in a variety of
ways, depending on their deployment preference. Values added to
whoogle.env can be enabled using WHOOGLE_DOTENV=1, in which case all
values in the env var file will overwrite defaults or user provided
settings.

Co-authored-by: Ben Busby <benbusby@protonmail.com>
2021-04-05 11:00:56 -04:00
Ben Busby 8ad8e66d37 Improve static typing throughout repo
Eventually this should be part of a separate mypy ci build, but right
now it's just a general guideline. Future commits and PRs should be
validated for static typing wherever possible.

For reference, the testing commands used for this commit were:

mypy --ignore-missing-imports --pretty --disallow-untyped-calls app/
mypy --ignore-missing-imports --pretty --disallow-untyped-calls test/
2021-04-05 11:00:56 -04:00
Shimul 892b646a4e Configure PWA for mobile browsers (#234)
Fix PWA issue for mobile phones
Fix icon loading issue
Update app/static/img/favicon/manifest.json

Co-authored-by: Ben Busby <benbusby@pm.me>
2021-04-05 11:00:56 -04:00
Ben Busby e7c63afc1a Re-add search css to results page
The results page search css was removed during the refactor to allow for
user defined css. This adds that back.
2021-04-05 11:00:56 -04:00
Ben Busby 083c3758a1 Return 503 if response is blocked by captcha
Also added in a slight modification to the dark theme style, which
should only apply the border radius in the header.

Closes #226
2021-04-05 11:00:56 -04:00
Ben Busby 62a9b9e949 Allow user-defined CSS/theming (#227)
* Add custom CSS field to config

This allows users to set/customize an instance's theme and appearance to
their liking. The config CSS field is prepopulated with all default CSS
variable values to allow quick editing.

Note that this can be somewhat of a "footgun" if someone updates the
CSS to hide all fields/search/etc. Should probably add some sort of
bandaid "admin" feature for public instances to employ until the whole
cookie/session issue is investigated further.

* Symlink all app static files to test dir

* Refactor app/misc/*.json -> app/static/settings/*.json

The country/language json files are used for user config settings, so
the "misc" name didn't really make sense. Also moved these to the static
folder to make testing easier.

* Fix light theme variables in dark theme css

* Minor style tweaking
2021-04-05 11:00:56 -04:00
Shimul 337d0ebe37 Handle manifest-src in CSP (#231) 2021-04-05 11:00:56 -04:00
Ben Busby e5d1f6a292 Add healthcheck to Dockerfile
See #184
2021-04-05 11:00:56 -04:00
Ben Busby f8dfc78539 Improve naming of *_utils files, update fn/class doc
The app/utils/*_utils weren't named very well, and all have been updated
to have more accurate names.

Function and class documention for the utils have been updated as well,
as part of the effort to improve overall documentation for the project.
2021-04-05 11:00:56 -04:00
Ben Busby dcb80ac250 Send CSP header in all responses
Introduces a new content security policy header for responses to all
requests to reduce the possibility of ip leaks to outside connections.
By default blocks all inline scripts, and only allows content loaded
from Whoogle.

Refactors a few small inline scripting cases in the project to their own
individual scripts.
2021-04-05 11:00:56 -04:00
Ben Busby d146016860 Remove auth req for accessing opensearch
Requiring authentication for accessing the opensearch template prevents
the browser from accessing the file when adding as a default search
engine. This removes the authentication requirement from the opensearch
route, which should never provide any sensitive information anyways.
2021-04-05 11:00:56 -04:00
Ben Busby ecb7885a56 Allow bang operator anywhere in query
Bang operator can now be placed anywhere in the query, to allow for peak
efficiency in stream of consciousness querying (i.e. `big !reddit
chungus` will search reddit for big chungus`).

Fixes #196
2021-04-05 11:00:56 -04:00
Ben Busby 64567a63ea Ensure G logo doesn't appear in mobile img results
Adds a separate check to remove all images sourced from www.gstatic.com,
which is where the mobile logo in particular is coming from.
2021-04-05 11:00:56 -04:00
Tomasz Borychowski 03bd4b6871 fix 'j' and 'k' inside search input 2021-04-05 11:00:56 -04:00
Roman Štefko 7f3a284e04 Do not autocapitalize on index page search bar (#200) 2021-04-05 11:00:56 -04:00
Tomasz Borychowski 5538ac862e add basic keyboard support 2021-04-05 11:00:56 -04:00
Ben Busby 6600d8580c Add ability to redirect reddit.com to libredd.it (#180)
* Adds the ability to redirect reddit.com to libredd.it using the existing
 "site alts" config setting.

This adds the WHOOGLE_ALT_RD environment variable for optionally
redirecting reddit links to libreddit
(https://github.com/spikecodes/libreddit).

* Include libreddit in home page site alt note
2021-04-05 11:00:56 -04:00
Ben Busby b57c86a1d0
Bump version to 0.3.2 2021-04-02 12:57:15 -04:00
Ben Busby fdd4ee590f
Hotfix: Set EU consent cookie to pending for all requests
See discussion on #243
2021-04-02 12:32:59 -04:00
Ben Busby 0a6575d219
Hotfix: Move language/country json to app dir
Pip installs of whoogle search were missing access to the misc/ folder,
which previously contained the language and country json files. These
have been moved to app/misc, and the previous root level misc/ was
renamed to config/ (since it now only contains the tor config files).

Bump to 0.3.1.
2021-02-07 18:55:27 -05:00
Ben Busby 329c38efb0
Hotfix: Enforce https in heroku opensearch template
Heroku instances were using the base http url when formatting the
opensearch.xml template. This adds a new routing utility, "needs_https",
which can be used for determining if the url in question needs
upgrading.
2021-01-23 14:50:30 -05:00
Ben Busby 5c69283e80
Hotfix: Add hidden submit btn for nojs searches
With javascript disabled, searches could not be submitted on the results
page using the "Enter" key. Adding a hidden submit button to the header
template resolves this issue.
2021-01-19 11:11:13 -05:00
Ben Busby 406e236666
Bump version to 0.3.0 2021-01-17 23:07:43 -05:00
Ben Busby 440c4e9c50
Remove lxml dependency
The lxml dependency in the project was fairly unnecessary, and made the
initial build time for the project considerably slower. This replaces
all instances of lxml with either the default html parser (for bs4
constructors) or the built in xml.etree package (for search suggestion
parsing).
2020-12-29 18:43:42 -05:00
Ben Busby 2bbc649903
Add support for UPS/USPS/FedEx tracking queries
Introduces a new javascript "utils" file, which includes a check for
matching the query against a set of tracking number regexes on page
load. If a match is found, the script prepends a link to the
(presumably) appropriate tracking page.

Referenced in #98
2020-12-27 18:00:35 -05:00
Ben Busby 6e7ec9918a
Move language/country settings to app config
Moves the language and country dicts from the config model to json files
that are loaded during app init and stored in the app config dict. This
substantially improves the readability of the config model and allows
for much more sensible loading of the language/country options.
2020-12-17 16:42:05 -05:00
Ben Busby 375f4ee9fd
PEP-8: Fix formatting issues, add CI workflow (#161)
Enforces PEP-8 formatting for all python code

Adds a github action build for checking pep8 formatting using pycodestyle
2020-12-17 16:06:47 -05:00
Ziga Zajc b55aad3fdf
Use #222 for dark mode bg (#159) 2020-12-17 16:03:05 -05:00
Ben Busby b695179c79
Add ability to collapse "people also ask"
This adds a step in the filter process to wrap the "people also ask"
section in a <details> element, which automatically collapses the
contents of the section. Clicking/tapping the details element expands
the view as normal.

See #113
2020-12-15 11:09:48 -05:00
Ben Busby 3978241d28
Fix black text in dark mode dropdowns
Closes #145
2020-12-15 10:48:29 -05:00
Ben Busby 5b5c2588ed
Fix nojs lxml constructor
The BeautifulSoup constructur in gen_nojs needed to explicitly set
features='lxml' to silence a warning from the library.

Also temporarily disabled the site alts test since the results are too
unreliable. This should be moved to a unit test instead.
2020-12-11 19:21:32 -05:00
Ben Busby e6db3112f7
Fix pagination bug for pages > 3
The pagination footer on the results page after page 2 has three actions
(beginning, next, previous). The footer filter was updated to remove
items with more than three actions to fix this.

See #131
2020-12-07 20:38:57 -05:00
Ben Busby 6c429e6dd1
Allow setting site alts using environment vars (#155)
* Add ability to configure site alts w/ env vars

Site alternatives (i.e. twitter.com -> nitter.net) can now be configured
using environment variables:

WHOOGLE_ALT_TW='nitter.net' # twitter alt
WHOOGLE_ALT_YT='invidio.us' # youtube alt
WHOOGLE_ALT_IG='bibliogram.art/u' # instagram alt

Updated testing to confirm results have been modified.

* Add site alt vars to docker settings and readme
2020-12-05 17:01:21 -05:00
Ben Busby 44a5da1895
Fix heroku https upgrade, add funding options
Heroku app instances have been notoriously bad at having the instance
automatically upgraded to https. This adds a step in the before request
decorator to always upgrade heroku apps, since they're always deployed
with the certificate, but never configured to upgrade automatically.

Fixes #153
2020-12-05 15:53:42 -05:00
Ben Busby 54109874fb
Move screenshots/branding to separate docs folder 2020-12-04 10:53:12 -05:00