What I built

Casper, the Astral Librarian, is a Discord bot for searching the internet. Unlike other web searching discord bots, Casper does not restrict search results to a single search engine, instead returning results from a whole host of search engines.

Out of the box, Casper supports a number of features provided by the underlying search engine, SearXNG, including searching specific engines by formatting queries like this:

!<ENGINE_NAME> my search

For example,

!reddit nvidia 3000 series review

These features are described in greater detail in the SearXNG documentation

Here is a list of some of the search engines Casper currently supports:

• google
• bing
• duckduckgo
• qwant
• reddit
• stackoverflow
• github
• bitbucket
• gitlab
• docker hub
• deviant art
• imdb
• pypi
• npm
• packagist
• rubygems
• yahoo
• vimeo
• youtube
• twitter
• wolfram alpha

In addition to these features, I plan to add some exciting features like allowing users to add their own search engines to Casper as well as user and channel based customization options.

If things go really well, I may also port Casper to other messaging platforms too!

Category Submission:

• Wacky Wildcards
• Integration Innovators

App Link

Discord invite link:
Invite Casper to your Server

Source code:
https://github.com/speratus/indexor-discord-bot

Screenshots

Here is a series of screenshots demonstrating Casper in Use:

Screenshots of Kubernetes config

List of nodes in Kubernetes cluster according to Kubernetes control panel (identifying information has been removed):

Same list, but showing the nodes running as instances running in Linode's control panel (identifying information has been removed):

Description

Casper is a Discord bot built with Python, discord.py, Docker, Kubernetes, and SearXNG.. The main goal of Casper is to allow users to search for web results directly from within Discord. Secondary goals include a high degree of flexibility, enabling users to search using their preferred search engine, and, to that end, supporting the widest array of search engines possible.

Finally, Casper allows users to share search results with each other, enabling them to establish trust and common sources of information as well as a discussion of the quality of sources.

There are several major components of Casper:

• The bot itself
• The search library containing the main search features. This library is what will enable Casper to be ported to other messaging platforms
• The SearXNG Instance, powering web-wide searches

Link to Source Code

Main Casper source code:

Search functionality core library:

pip install git+https://github.com/speratus/indexor-core.git#egg=indexor_core

from indexor_core.config import Config
from indexor_core.searcher import search

search_xng_instance_url = 'https://example.com'

c = Config(engine_url=search_xng_instance_url)

results = search('hello world', c)

Permissive License

MIT License

Background

I came up with the idea for Casper on a whim, thinking how interesting it would be to get search results directly in a Discord server.

I also saw this as an opportunity learn a couple of technologies I had been interested in, but had never created any projects with: Docker and Kubernetes.

How I built it

I deployed Casper on top of Linode's Kubernetes engine. I chose Linode Kubernetes engine for this project so that it would be easy to scale, update and rollback as necessary. This also lets me exercise my interest in learning Kubernetes and deploying an application using it.

To deploy Casper on Linode's Kubernetes engine, I had to overcome several obstacles:

• Packaging the source code as a Docker image to which I could transfer necessary project information while keeping it out of the source code for the sake of privacy

docker build -t casper:1.0.2

• Creating a deployment for the SearXNG instances that had the configuration required to be compatible with the Casper source code

apiVersion: apps/v1
kind: Deployment
metadata:
  name: searxng-deploy
spec:
  replicas: ...
  selector:
    matchLabels:
      app: searxng
  template:
    metadata:
      labels:
        app: searxng
    spec:
      containers:
        - name: searxng
          image: searxng/searxng
          env:
            - name: INSTANCE_NAME
              value: 'casper-engine'
            - name: BASE_URL
              value: 'http://n.n.n.n'
          volumeMounts:
            - name: settings
              mountPath: /etc/searxng/settings.yml
              subPath: settings.yml
      volumes:
        - name: settings
          configMap:
            name: searxng-settings

• Creating a Kubernetes service for the SearXNG instances so that they could be load balanced as well as simultaneously enabling the Casper source to communicate with the instances and preventing the SearXNG instances from being accessible outside of the Kubernetes Cluster

apiVersion: v1
kind: Service
metadata:
  name: lb-searxng
spec:
  selector:
    app: searxng
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8080

• Creating a configmap for the SearXNG instance that could be read as the settings.yml file and which also includes the required values to enable Casper to function.

kubectl create configmap searxng-settings --from-file=PATH/TO/FILE/searxng/settings.yml

• Creating a Kubernetes secret to store the Discord bot's token without exposing it to outside access.

kubectl create secret generic ...

• Creating a deployment for the bot itself which drew from the necessary secrets and had the proper environment variables to communicate with the SearXNG instances.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: casper-deploy
spec:
  replicas: ...
  selector:
    matchLabels:
      app: casper
  template:
    metadata:
      labels:
        app: casper
    spec:
      containers:
        - name: casper-bot
          image: speratus/casper:1.0.2
          env:
            - name: DISCORD_TOKEN
              valueFrom:
                secretKeyRef:
                  name: ...
                  key: ...
            - name: ENGINE_URL
              value: 'http://n.n.n.n'

Once I had tested out each of these steps in detail and identified working settings for each component, deployment was as easy as running a few commands. Once that was done, Casper was live with no headaches.

NOTE: In these samples, I replaced IP address with n to indicate that anybody wishing to follow along should use their own information instead. I also used ... to indicate regular values that need to be replaced with the information specific to your context as well.

Additional Resources/Info

SearXNG's documentation for creating and running an instance:
https://docs.searxng.org/admin/index.html

SearXNG Source:

searxng / searxng

SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled.

---

---

Privacy-respecting, hackable metasearch engine

If you are looking for running instances, ready to use, then visit searx.space Otherwise jump to the user, admin and developer handbooks you will find on our homepage.

---

Contact

Come join us if you have questions or just want to chat about SearXNG.

Matrix

#searxng:matrix.org

IRC

#searxng on libera.chat which is bridged to Matrix.

Differences to searx

SearXNG is a fork of searx. Here are some of the changes:

User experience

• Huge update of the simple theme:
  • usable on desktop, tablet and mobile
  • light and dark versions (you can choose in the preferences)
  • support right-to-left languages
  • see the screenshots
• the translations are up to date, you can contribute on Weblate
• the preferences page has been updated:
  • you can see which engines are reliable or not
  • engines are grouped inside each tab
  • each engine has a description
• thanks to the anonymous metrics, it…

View on GitHub

Planned Features

• Multiple pages of results
• Results caching for faster response times
• Improved support for getting results from specific search engines
• User-added search engines (Any sites supporting the OpenSearch standard)
• Server and User based settings
• Direct messaging Casper to get results in addition to using the slash command
• Support for other messaging platforms such as Slack, Telegram, Matrix.org, and others.

Unsupported features

Currently, Casper does not support SearXNG's special queries features.