Go to file
Ben Busby 0c0ebb8917 Added POST search, encrypted query strings, refactoring
The implementation of POST search support comes with a few benefits. The
most apparent is the avoidance of search queries appearing in web server
logs -- instead of the prior GET approach (i.e.
/search?q=my+search+query), using POST requests with the query stored in
the request body creates logs that simply appear as "/search".

Since a lot of relative links are generated in the results page, I came
up with a way to generate a unique key at run time that is used to
encrypt any query strings before sending to the user. This benefits both
regular text queries as well as fetching of image links and means that
web logs will only show an encrypted string where a link or query
string might slip through.

Unfortunately, GET search requests still need to be supported, as it
doesn't seem that Firefox (on iOS) supports loading search engines by
their opensearch.xml file, but instead relies on manual entry of a
search query string. Once this is updated, I'll probably remove GET
request search support.
2020-04-28 18:19:34 -06:00
.github/ISSUE_TEMPLATE Update issue templates 2020-04-17 13:36:45 -06:00
app Added POST search, encrypted query strings, refactoring 2020-04-28 18:19:34 -06:00
test Fixed search results test 2020-04-26 18:11:02 -06:00
.dockerignore Refactored routes, added filter class for returned results, added dockerignore 2020-04-10 14:52:27 -06:00
.gitignore Fixed gitignore, added required files 2020-04-24 19:03:22 -06:00
.travis.yml Fixed travis CI 2020-04-24 17:27:21 -06:00
Dockerfile Deployment related refactoring, fixes to Dockerfile 2020-04-24 17:23:08 -06:00
LICENSE Update LICENSE 2020-04-11 12:38:31 -06:00
README.md Added POST search, encrypted query strings, refactoring 2020-04-28 18:19:34 -06:00
app.json Updated app.json to use container stack 2020-04-24 17:53:45 -06:00
heroku.yml Added heroku.yml 2020-04-24 17:56:16 -06:00
requirements.txt Deployment related refactoring, fixes to Dockerfile 2020-04-24 17:23:08 -06:00
run Fixed config bug in filter, updated run script to work on mac os 2020-04-16 18:50:31 -06:00

README.md

Shoogle Banner


Latest Release License: MIT Build Status codebeat badge

Get Google search results, but without any ads, javascript, AMP links, cookies, or IP address tracking. Easily deployable in one click as a Docker app, and customizable with a single config file. Quick and simple to implement as a primary search engine replacement on both desktop and mobile.

Contents

  1. Features
  2. Setup
  3. Install/Deploy
  4. Usage
  5. Extra Steps
  6. FAQ
  7. Screenshots

Features

  • No ads or sponsored content
  • No javascript
  • No cookies
  • No tracking/linking of your personal IP address
  • No AMP links
  • No URL tracking tags (i.e. utm=%s)
  • View images at full res with one click
  • Dark mode
  • Randomly generated User Agent
  • Easy to install/deploy
  • Optional location-based searching (i.e. results near <city>)
  • Optional NoJS mode to disable all Javascript in results

Setup

If using Heroku/Heroku Quick Deploy:

  • A Heroku Account
    • Allows for free hosting of the web app and single-click deployment.
    • Alternatively, you can host the app using a different service, or deploy it to your own server (explained below).

If deploying manually:

Install

There are a few different ways to begin using the app, depending on your preferences:

A) Heroku Quick Deploy (Free)

Deploy

B) Manual Setup (non-Docker)

Clone the repo and run the following commands to start the app in a local-only environment:

git clone https://github.com/benbusby/shoogle.git
cd shoogle
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
./run

C) Manual Setup (Docker)

  1. Ensure the Docker daemon is running, and is accessible by your user account
  • To add user permissions, you can execute sudo usermod -aG docker yourusername
  • Running docker ps should return something besides an error. If you encounter an error saying the daemon isn't running, try sudo systemctl start docker (Linux) or ensure the docker tool is running (Windows/macOS).
  1. Clone and deploy the docker app using a method below:

Using Heroku CLI

heroku login
heroku container:login
git clone https://github.com/benbusby/shoogle.git
cd shoogle
heroku create
heroku container:push web
heroku container:release web
heroku open

This series of commands can take a while, but once you run it once, you shouldn't have to run it again. The final command, heroku open will launch a tab in your web browser, where you can test out Shoogle and even set it as your primary search engine.

Using your own server, or alternative container deployment

There are other methods for deploying docker containers that are well outlined in this article, but there are too many to describe set up for each here. Generally it should be about the same amount of effort as the Heroku deployment.

Depending on your preferences, you can also deploy the app yourself on your own infrastructure. This route would require a few extra steps:

  • A server (I personally recommend Digital Ocean or Linode, their cheapest tiers will work fine)
  • Your own URL (I suppose this is optional, but recommended)
  • SSL certificates (free through Let's Encrypt)
  • A bit more experience or willingness to work through issues

Usage

Same as most search engines, with the exception of filtering by time range.

To filter by a range of time, append ":past

Extra Steps

Set Shoogle as your primary search engine

Update browser settings:

  • Firefox (Desktop)
    • Navigate to your app's url, and click the 3 dot menu in the address bar. At the bottom, there should be an option to "Add Search Engine". Once you've clicked this, open your Firefox Preferences menu, click "Search" in the left menu, and use the available dropdown to select "Shoogle" from the list.
  • Firefox (iOS)
    • In the mobile app Settings page, tap "Search" within the "General" section. There should be an option titled "Add Search Engine" to select. It should prompt you to enter a title and search query url - use the following elements to fill out the form:
      • Title: "Shoogle"
      • URL: "https://<your shoogle url>/search?q=%s"
  • Others (TODO)

Customizing and Configuration

Shoogle currently allows a few minor configuration settings, accessible from the home page:

  • "Near"
    • Set to a city name to narrow your results to a general geographic region. This can be useful if you rely on being able to search for things like "pizza places" and see results in your city, rather than results from wherever the server is located.
  • Dark Mode
    • Sets background to pure black
  • NoJS Mode (Experimental)
    • Adds a separate link for each search result that will open the webpage without any javascript content served. Can be useful if you're seeking a no-javascript experience on mobile, but otherwise could just be accomplished with a browser plugin.

Prevent Downtime (Heroku only)

Part of the deal with Heroku's free tier is that you're allocated 550 hours/month (meaning it can't stay active 24/7), and the app is temporarily shut down after 30 minutes of inactivity. Once it becomes inactive, any Shoogle searches will still work, but it'll take an extra 10-15 seconds for the app to come back online before displaying the result, which can be frustrating if you're in a hurry.

A good solution for this is to set up a simple cronjob on any device at your home that is consistently powered on and connected to the internet (in my case, a PiHole worked perfectly). All the device needs to do is fetch app content on a consistent basis to keep the app alive in whatever ~17 hour window you want it on (17 hrs * 31 days = 527, meaning you'd still have 23 leftover hours each month if you searched outside of your target window).

For instance: */20 7-23 * * * curl https://<your heroku app name>.herokuapp.com > /home/<username>/shoogle-refresh will fetch the home page of the app every 20 minutes between 7am and midnight, allowing for downtime from midnight to 7am. And again, this wouldn't be a hard limit - you'd still have plenty of remaining hours of uptime each month in case you were searching after this window has closed.

FAQ

What's the difference between this and Searx?

There are quite a few similarities and differences between Shoogle and Searx. Both are built to respect user privacy as much as possible, and both are capable of being individually deployed. Additionally, a lot of privacy features are shared between the two services, but since Shoogle is still in its infancy, a lot of features are missing, but will gradually be added in over time.

The core feature of Shoogle that sets it apart and will be supported through all added features is the extremely simplified deployment of the app for individuals at any experience level, not just the technically inclined.

It is strictly intended to be self-hosted only, and as such, needs to be (and is) excruciatingly easy to host yourself.

Why does the image results page look so bad on mobile?

A lot of the app currently piggybacks on Google's existing support for fetching results pages with Javascript disabled. To their credit, they've done an excellent job with styling pages, but it seems that the image results page - particularly on mobile - is a little rough. Moving forward, with enough interest, I'd like to transition to fetching the results and parsing them into a unique Shoogle-fied interface that I can style myself.

Screenshots

Desktop

Shoogle Desktop

Mobile

Shoogle Mobile