DEV Community

Cover image for Surfing web inside a terminal, because why not?
Bhupesh Varshney 👾
Bhupesh Varshney 👾

Posted on • Originally published at bhupesh.me on

Surfing web inside a terminal, because why not?

I recently wrote a python script to surf the web (see search results) directly into the terminal.

Here is a demo

The script is available on my github

It’s not that big of a deal since all credit goes to searx a privacy respecting FOSS metasearch engine

Why Searx

Why not?

  • It respects your privacy. No user-tracking
  • Open-source
  • You can even set up your own (private/public) searx instance on a VPS. List of public instances
  • Aggregates results from more than 70 search services (DuckDuckGo, Wikipedia, etc)

Installation

The script doesn’t require any dependencies (except Python 3), just download the script and run it.

Note that surf was written for/as a Linux user. It should work on Mac without any problems. If you are on Windows consider sending patches if the script breaks. Thanks!

wget -q https://raw.githubusercontent.com/Bhupesh-V/.Varshney/master/scripts/surf

chmod +x surf && mv surf $HOME/.local/bin/
Enter fullscreen mode Exit fullscreen mode

If you are more of a bad boy/girl then add the location of surf script to your PATH

export PATH="DIR_PATH_WHICH_CONTAINS_SURF:$PATH"
# e.g
export PATH="$HOME/Documents/.Varshney/scripts:$PATH"
Enter fullscreen mode Exit fullscreen mode

You can view help & usage on surf by doing a simple surf --help. Let’s go through all the options available.

Usage

  • Just run the script, without any arguments. surf will prompt you for a search query:
  $ surf
Enter fullscreen mode Exit fullscreen mode
  • Show link description.
  $ surf -d
Enter fullscreen mode Exit fullscreen mode

surf-demo

  • Provide search query as an argument (also --query):
$ surf -dq "how to change the world"
Enter fullscreen mode Exit fullscreen mode
  • You can also pipe data to surf ¯\_(ツ)_/¯:
$ echo "search this" | surf
Enter fullscreen mode Exit fullscreen mode
  • Use -c to specify search category (default: general):
$ surf -c "videos" -dq "how to make a pizza"
Enter fullscreen mode Exit fullscreen mode

Other categories include: “news”, “files”, “images”, “map”

surf-demo

  • Update searx instance cache list before fetching results:
$ surf -udq "when is doomsday"
Enter fullscreen mode Exit fullscreen mode

Updating cache is not usually required, I recommend running it once/twice a week to get a list of active instances.

The cache is located at $HOME/.local/.searx_instances on linux and $HOME/Library/Caches/.searx_instances on Mac

surf-demo

  • Show only top N results (also --top):
$ surf -t 5 -dq "check if key exists in map"
Enter fullscreen mode Exit fullscreen mode

searx supports those sneaky search tricks as well,

surf -dq "site:youtube.com how to make pizza"
surf -dq "site:stackoverflowcom <my-error-message>"
surf -dq "intitle:best vim plugins"
surf -dq "inurl:docs.djangoproject.com templates"
Enter fullscreen mode Exit fullscreen mode

Well now you would be asking “what’s the use of this bhupesh, If I have to open the link in the browser at the end”

But what I realized is that it may accidentally fit right into my/your workflow if you use Vim, let me show how

Browsing Inside Vim, say whaat

We need a way to open links right from our vim terminal. Add this to your vimrc or init.vim

" Open hyper link in current line
function! OpenLink()
    let links = []
    try
        call substitute(getline('.'), 'http[s]\?:\/\/[^) \"]*', '\=add(links, submatch(0))', 'g')
        echo "Opening " . links[0]
        exe "silent! !xdg-open " . links[0]
    catch E684
        echo "No link found :("
        return
    endtry
endfunction
Enter fullscreen mode Exit fullscreen mode

The line

exe "silent! !xdg-open " . links[0]
Enter fullscreen mode Exit fullscreen mode

will directly open your default browser with the link.

If you are not using X11, then you can just swap it directly with the browser of your choice or your default file manager

  1. Firefox
exe "silent! !firefox --new-tab " . links[0]

Enter fullscreen mode Exit fullscreen mode
  1. Chromium
exe "silent! !chromium-browser " . links[0]

Enter fullscreen mode Exit fullscreen mode

Also, I recently found a less bulky approach on vimtricks.com

function! OpenURLUnderCursor()
  let s:uri = expand('<cWORD>')
  let s:uri = substitute(s:uri, '?', '\\?', '')
  let s:uri = shellescape(s:uri, 1)
  if s:uri != ''
    silent exec "!open '".s:uri."'"
    :redraw!
  endif
endfunction
nnoremap gx :call OpenURLUnderCursor()<CR>

Enter fullscreen mode Exit fullscreen mode

Searching Error messages inside Vim

Say I am debugging something and end up with a never before seen Error message. I can quickly send the error string to surf and get links to StackOverflow or GitHub!

Here is a demo of me using flake on surf & searching about rules:

surf demo vim terminal

How this works is pretty straightforward. We just need to get the current visual selection.

function! GetVisualSelection()
    " Thanks: https://stackoverflow.com/a/6271254/8209510
    let [line_start, column_start] = getpos("'<")[1:2]
    let [line_end, column_end] = getpos("'>")[1:2]
    let lines = getline(line_start, line_end)
    let lines[-1] = lines[-1][: column_end - (&selection == 'inclusive' ? 1 : 2)]
    let lines[0] = lines[0][column_start - 1:]
    return join(lines, "\n")
endfunction

xnoremap <leader>t <esc>:split <bar> call SendQueryToTerm()<CR>

function! SendQueryToTerm()
    let selection = GetVisualSelection()
    exe ":term surf -dq \'" . selection . "\'"
endfunction

Enter fullscreen mode Exit fullscreen mode

This will open a split above with a new terminal buffer.

Convinced yet?

Maybe not

If you liked surf make sure to subscribe to the Atom Feed or watch my dotfiles repo for the latest development or better fork & hack your own features!!

Top comments (4)

Collapse
 
moopet profile image
Ben Sinclair

I got through about half this post thinking, "why not just use lynx|links?" and then saw your Vim integration and switched to, "hey that's an idea".

I think I might take this idea play with it - instead of using xdg-open, make it figure out what browser is available and use links if you're on a remote system through ssh - or maybe to take results and put them in the quickfix window?

Collapse
 
kovarpavel profile image
Pavel Kovar

With more and more ugly big banners and ads, this is the way how to find relevant information on the internet

Collapse
 
brandonwallace profile image
brandon_wallace

Nice article. This is good information.

Collapse
 
bhupesh profile image
Bhupesh Varshney 👾

Thanks :)