Simple XDP Firewall with Golang

For almost 20+ years, the traditional security features of a Linux system were centered around iptables, which has been the de facto packet filtering mechanism in the Linux kernel.
However, the increase in network speed and the transformation of the type of applications running in a Linux server has led to the consciousness that the current implementation may not be able to cope with the modern requirements, particularly in terms of scalability.
That's where XDP & eBPF come in.

So what is XDP?

XDP is a part of the upstream Linux kernel, and enables users to inject packet processing programs into the kernel, that will be executed for each arriving packet, before the kernel does any other processing on the data.

Writing the Code

So, enough nerd-talk about the technology, how do we use it?

To demonstrate, we'll be writing a simple firewall that'll drop incoming packets based on their IP address using XDP.
The program is going to consist of 2 parts:

• The Kernel Space Code
• The User Space Code

The Kernel side is written in C or Rust (We'll be using C), and is compiled into the eBPF byte code format that is verified and JIT-compiled in the kernel.
I'll be writing the XDP application with a User-Space controller written in Go.
I should mention that writing complex eBPF programs requires much more context than what we'll be doing today. There is a lot to know about eBPF and XDP. We will barely scratch the surface.
All the code for this project can be found in here.

First, let's discuss the kernel side of things.

Kernel Space Code

As discussed earlier, we are a bit restricted in the language we use in here. Possible languages that can be used are Rust and C.
We'll be using C because there's a C program out there for almost every standard use-case scenario for us to grab and use, except if what we're trying to do is very specific to our usecase. (Cloudflare's L4 DDos Mitigation platform showcases this very well)
Our kernel-space program is gonna consist of one function called firewall and one map called blacklist.
We already know what a function is, but what is a map and why do we need it?

Maps are a “generic data structure for storage of different types of data” and can be used to share data between eBPF programs as well as between kernel and userspace. The key and value of a map can be of arbitrary size as defined when creating the map. The user also defines the maximum number of entries with max_entries.

The blacklist is gonna store all IP addresses that need to be dropped, and the firewall function will check each packet and see if the source IP of that packet matches any of the IP addresses stored in the blacklist map.
We'll be using the following XDP actions:

• XDP_PASS if the IP address matches none of the IP addreses in the blacklist map.
• XDP_DROP if it does match an IP address in the map.
• XDP_ABORTED if the ipv4 header or ethernet header of the packet is malformed.

First, let's take a look at the defintion of the blacklist map:

BPF_MAP_DEF(blacklist) = {
    .map_type = BPF_MAP_TYPE_LPM_TRIE,
    .key_size = sizeof(__u64),
    .value_size = sizeof(__u32),
    .max_entries = 16,
};
BPF_MAP_ADD(blacklist);

The defintion is quite simple.

we'll first define the type of the map.
Here's the list of types of maps, found in the bpf_helpers.h header file:

enum bpf_map_type {
  BPF_MAP_TYPE_UNSPEC = 0,
  BPF_MAP_TYPE_HASH,
  BPF_MAP_TYPE_ARRAY,
  BPF_MAP_TYPE_PROG_ARRAY,
  BPF_MAP_TYPE_PERF_EVENT_ARRAY,
  BPF_MAP_TYPE_PERCPU_HASH,
  BPF_MAP_TYPE_PERCPU_ARRAY,
  BPF_MAP_TYPE_STACK_TRACE,
  BPF_MAP_TYPE_CGROUP_ARRAY,
  BPF_MAP_TYPE_LRU_HASH,
  BPF_MAP_TYPE_LRU_PERCPU_HASH,
  BPF_MAP_TYPE_LPM_TRIE,
  BPF_MAP_TYPE_ARRAY_OF_MAPS,
  BPF_MAP_TYPE_HASH_OF_MAPS,
  BPF_MAP_TYPE_DEVMAP,
  BPF_MAP_TYPE_SOCKMAP,
  BPF_MAP_TYPE_CPUMAP,
  BPF_MAP_TYPE_XSKMAP,
  BPF_MAP_TYPE_SOCKHASH,
  BPF_MAP_TYPE_CGROUP_STORAGE,
  BPF_MAP_TYPE_REUSEPORT_SOCKARRAY,
  BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE,
  BPF_MAP_TYPE_QUEUE,
  BPF_MAP_TYPE_STACK,
  BPF_MAP_TYPE_SK_STORAGE,
};

Then we'll define the .key_size, .value_size and .max_entries.

Now, take a look at the firewall function.
Before we define the function, we need to define 2 structs:

• ethhdr Ethernet header
• iphdr IPv4 header

// Ethernet header
struct ethhdr {
  __u8 h_dest[6];
  __u8 h_source[6];
  __u16 h_proto;
} __attribute__((packed));

// IPv4 header
struct iphdr {
  __u8 ihl : 4;
  __u8 version : 4;
  __u8 tos;
  __u16 tot_len;
  __u16 id;
  __u16 frag_off;
  __u8 ttl;
  __u8 protocol;
  __u16 check;
  __u32 saddr;
  __u32 daddr;
} __attribute__((packed));

We're going to need to define these structs so we analyze the received packets. All of this logic happens in the firewall function.
Let's take a look at the defintion of the function:

SEC("xdp")
int firewall(struct xdp_md *ctx) {
  void *data_end = (void *)(long)ctx->data_end;
  void *data = (void *)(long)ctx->data;

  struct ethhdr *ether = data;

  // Check if the Ethernet header is malformed
  if (data + sizeof(*ether) > data_end) {
    return XDP_ABORTED;
  }

  if (ether->h_proto != 0x08U) {  // htons(ETH_P_IP) -> 0x08U
    // If not IPv4 Traffic, pass the packet
    return XDP_PASS;
  }

  data += sizeof(*ether);
  struct iphdr *ip = data;

  // Check if the IPv4 header is malformed
  if (data + sizeof(*ip) > data_end) {
    return XDP_ABORTED;
  }

  struct {
    __u32 prefixlen;
    __u32 saddr;
  } key;

  key.prefixlen = 32;
  key.saddr = ip->saddr;

  // Lookup the IP Address in the blacklsit map
  // we defined earlier.
  // If the source IP matches the IP in the blacklist map
  // We drop the packet!!
  __u64 *rule_idx = bpf_map_lookup_elem(&blacklist, &key);
  if (rule_idx) {
    __u32 index = *(__u32*)rule_idx;  // make verifier happy
    return XDP_DROP;
  }

  return XDP_PASS;
}

Do not be intimidated!
C code can be intimidating, but taking a closer look at the code we can see what the function is trying to accomplish.

I should mention that this XDP program only supports ipv4 packets, so any other type of traffic (ipv6 included ) will be passed with XDP_PASS.

We're almost finished with the Kernel-Space side.
We just need to compile our code eBPF byte code.
But before we do that, we need to include the bpf_helpers.h header file at the start of C file:

#include "bpf_helpers.h"

This is not the same bpf_helpers.h header file in the bcc github repo. You can find the header file in here.
Either download it and place it in your project, or clone the repo and compile.
We'll be using Clang to compile our code to eBPF byte code.
Install it, and run the following command:

$ clang -I ../headers -O -target bpf -c xdp.c -o xdp.elf

the -I points to the directory where we downloaded the bpf_helpers.h file from earlier, so mind that. xdp.c is the file name of our C source code, and the output will be in the ELF format, that's what our User Space code will load into the kernel.
We can use the readelf command to verify the file:

$ readelf -a xdp.elf 
...

Symbol table '.symtab' contains 5 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS xdp.c
     2: 0000000000000130     0 NOTYPE  LOCAL  DEFAULT    3 LBB0_5
     3: 0000000000000000   312 FUNC    GLOBAL DEFAULT    3 firewall
     4: 0000000000000000    40 OBJECT  GLOBAL DEFAULT    5 blacklist

If you're having issues compiling, check out the source code and check if there's anything missing, or comment down below.

Let's place these 2 files inside a directory called bpf and move on to the exciting part of the tutorial!

User Space Code

Now comes the exciting (and less frusterating) part of our program!! Writing Go code.

Before we dive in, let's first take a look at all the eBPF/Go libraries provided for us by the lovely open-source community around the world.

Here's a list of the most popular Go eBPF libraires:

• ebpf-go by Cilium which is a pure-Go library to read, modify and load eBPF programs and attach them to various hooks in the Linux kernel.
• gobpf by iovisor which provides go bindings for the bcc framework as well as low-level routines to load and use eBPF programs from .elf files.
• libbpfgo by aquasecurity which is built around libbpf - the standard library for interacting with eBPF programs
• goebpf by Dropbox - A nice and convenient way to work with eBPF programs / perf events from Go.

I've decided to use goebpf, the one from Dropbox in this tutorial, mainly because of it's simplicity and it's easy beginner-friendly approach to eBPF.

Let's install the goebpf library by Dropbox:

go get github.com/dropbox/goebpf

Now that we have that out of the way, create a file called main.go and let's start coding!

Note: make sure the main.go file and the rest of C and ELF files are not in the same directory as that can cause errors and warnings, place the main.go file next to the bpf directory we made earlier.
So the directory hierarchy will look something similar to this:

.
├── bpf
│   ├── xdp.c
│   └── xdp.elf
├── go.mod
├── go.sum
├── headers
│   └── bpf_helpers.h
└── main.go

Let's first define two variables, the interface we'll attach our XDP program to and a slice containing the IP addresses and their subnet.

package main

import (
    "fmt"
    "log"
    "os"
    "os/signal"

    "github.com/dropbox/goebpf"
)

func main() {

    // Specify Interface Name
    interfaceName := "lo"
    // IP BlockList
    // Add the IPs you want to be blocked
    ipList := []string{
        "8.8.8.8",
    }

...

For my example, I'm going to attach my XDP program to my system's loopback device, and I'm gonna drop any packet coming from the source IP address 8.8.8.8.
Now let's load our XDP program:

// Load XDP Into App
    bpf := goebpf.NewDefaultEbpfSystem()
        // According to our directory hierarchy,
        // The xdp.elf file is placed in the bpf directory 
        // next to the main.go file
    err := bpf.LoadElf("bpf/xdp.elf")
    if err != nil {
        log.Fatalf("LoadELF() failed: %s", err)
    }

        // Load blacklist map
    blacklist := bpf.GetMapByName("blacklist")
    if blacklist == nil {
        log.Fatalf("eBPF map 'blacklist' not found\n")
    }

        // Load firewall function
    xdp := bpf.GetProgramByName("firewall")
    if xdp == nil {
        log.Fatalln("Program 'firewall' not found in Program")
    }
    err = xdp.Load()
    if err != nil {
        fmt.Printf("xdp.Attach(): %v", err)
    }

        // attach the program to the interface
        // in this case, the lo interface
    err = xdp.Attach(interfaceName)
    if err != nil {
        log.Fatalf("Error attaching to Interface: %s", err)
    }

The code is pretty self explanatory.
We load the eBPF ELF file from the bpf directory. We then load and assign the blacklist map & the firewall function to the variables blacklist & firewall respectivly, and finally we attach the program to the interface, in this case the loopback interface (lo).

We just need to create a function that takes the IP addresses in ipList and adds them to the blacklist map. The rest is handled in the C code we wrote earlier.
Here's the function:

func BlockIPAddress(ipAddreses []string, blacklist goebpf.Map) error {
    for index, ip := range ipAddreses {
        fmt.Printf("\t%s\n", ip)
        err := blacklist.Insert(goebpf.CreateLPMtrieKey(ip), index)
        if err != nil {
            return err
        }
    }
    return nil
}

Pretty simple isn't it.
Now we can use this function after attaching our program to the interface:

    ....
    err = xdp.Attach(interfaceName)
    if err != nil {
        log.Fatalf("Error attaching to Interface: %s", err)
    }

    BlockIPAddress(ipList, blacklist)

    defer xdp.Detach()

We're pretty much done. The only thing missing is that if we build and run our program it'll exit immediately, and with it the XDP program will be detached( see defer xdp.Detach ).
We don't want that, we want the program running until we interrupt it. We'll use channels for that.
Here's the final Go code:

package main

import (
    "fmt"
    "log"
    "os"
    "os/signal"

    "github.com/dropbox/goebpf"
)

func main() {

    // Specify Interface Name
    interfaceName := "lo"
    // IP BlockList
    // Add the IPs you want to be blocked
    ipList := []string{
        "8.8.8.8",
    }

    // Load XDP Into App
    bpf := goebpf.NewDefaultEbpfSystem()
    err := bpf.LoadElf("bpf/xdp.elf")
    if err != nil {
        log.Fatalf("LoadELF() failed: %s", err)
    }
    blacklist := bpf.GetMapByName("blacklist")
    if blacklist == nil {
        log.Fatalf("eBPF map 'blacklist' not found\n")
    }
    xdp := bpf.GetProgramByName("firewall")
    if xdp == nil {
        log.Fatalln("Program 'firewall' not found in Program")
    }
    err = xdp.Load()
    if err != nil {
        fmt.Printf("xdp.Attach(): %v", err)
    }
    err = xdp.Attach(interfaceName)
    if err != nil {
        log.Fatalf("Error attaching to Interface: %s", err)
    }

    BlockIPAddress(ipList, blacklist)

    defer xdp.Detach()
    ctrlC := make(chan os.Signal, 1)
    signal.Notify(ctrlC, os.Interrupt)
    log.Println("XDP Program Loaded successfuly into the Kernel.")
    log.Println("Press CTRL+C to stop.")
    <-ctrlC

}

// The Function That adds the IPs to the blacklist map
func BlockIPAddress(ipAddreses []string, blacklist goebpf.Map) error {
    for index, ip := range ipAddreses {
        err := blacklist.Insert(goebpf.CreateLPMtrieKey(ip), index)
        if err != nil {
            return err
        }
    }
    return nil
}

Build and run the Go code with sudo:

$ go build
$ sudo ./xdp-firewall
2022/12/02 02:36:46 XDP Program Loaded successfuly into the Kernel.
2022/12/02 02:36:46 Press CTRL+C to stop.

Now while the program is running, open another terminal and run the following command:

$ ip link list dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 xdpgeneric qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    prog/xdp id 173

We can see the XDP program with an id of 173 is now attached.
If we go back to the last terminal and exit the program, and run the command again:

$ ip link list dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

The XDP program has disappeared from the lo interface.

And this is pretty much it!
If you'd like to read more on this topic, I'll add a few links below for you to check out.

• Brandan Gregg's Blog (One of the pioneers of the eBPF/XDP space)
• https://github.com/dropbox/goebpf
• https://github.com/cilium/ebpf
• https://docs.cilium.io/en/v1.12/bpf.html
• https://ebpf.io/
• Repo for this project