Whitelisting by country region with ipset and iptables and bash automation

Recently I needed to restrict one server to country region. So I need to place range restrictions. For that purpose I used the publicly available database of ipdeny.com. There you can find all ip ranges by country region. And use each to black list a country with iptables or maybe some other kind of firewall. So I used the revers the logic… To use that lists to whitelist a country and forbid all others. Which was kind of easy. I just needed to download the right list (in my case Bulgaria). After that I just needed to create new iptables rules set. Create ipset with the downloaded list And apply the chain with ipset. And drop other. So I decided that I’ll use the opportunity to try to make this article as a automation process in bash. Suitable to deploy with LXD. And maybe illustrate a simple process of thinking while automation. Without further blabbering, lets dive in the process….

We start by adding execution flags. And declare a variable which will hold the country initials. After that will need to check a bit for validity. Maybe we can add some help menu as well in the flags… But some other time… After we’ve gain the country (name), we proceed to preparing the folder/file structure, that will hold our project files.

name=false
while getopts h:c: flag
do
    case "${flag}" in
        c) name=${OPTARG};;
    h) "help menu :::TODO";;
    *) exit;;
    esac
done
if [[ "$name" == false ]];
then
    printf "use -c country\n"; exit 1;
else
    prepare_folder_sctructure $name
fi

Here we start to prepare the folder/file structure. But maybe its better to check first, if that’s actually the first time that we’ve ran that script. Maybe its not… Who knows. So we first will check if there isn’t previous folder structure. If there is, we will leave it. And add some more countries or overwrite the same one… But more importantly. We are leaving option to add more regions if needed.
If it’s the first time, its stand for reason that maybe dependencies are not install as well. So we better check for that as well.
And then proceed downloading preparations.

function prepare_folder_sctructure()
{
    if ! [ -x "$(ls -a $PWD | grep zones)" ];
    then
        printf "There are no zones\n"
        dependencies
        mkdir $PWD/zones
        mkdir $PWD/zones/txt
        touch $PWD/zones/zone_scripts.sh && chmod a+x $PWD/zones/zone_scripts.sh
        prepare_download_zones $1
    else
        prepare_download_zones $1
    fi
}

As I’ve said, at one point or the other, will need to know if some dependencies, that we need are present. And if not lets install them.
We start by declaring a list. Which will hold just the names of the packages. And then we will walk all over it. And for each resource if its not install, install it. And then we will need to have some knowledge about the current security status of the server. After all there may be an incompatibility between the role sets. So we proceed to the “security_check”.

function dependencies()
{
    declare -a resources=("ipset" "curl");
    for i in "${resources[@]}"
    do
        if [ $(dpkg-query -W -f='${Status}' $i 2>/dev/null | grep -c "ok installed") -eq 0 ];
        then
            echo "No $i."
            apt install $i -y
        fi
    done
    security_check
}

Here we will check if firewall is on. Ufw and iptables at the same time, usually its not good idea. So we check if ufw is on. If it is, we disable it. Then we proceed making a service.

function security_check()
{
    firewall=$( ufw status )
    if [ "$firewall" == "Status: active" ];then ufw disable
    fi
    rc_locale_service
}

Here we prepare the rules for download… Remember the check “if the folder is not empty”. Now here, we check if the file created previously in the “folder_structure” is not empty. Because if its not empty, we wouldn’t want to completely overwrite the file… More so append new data to it.

After that we will actually start, first reading the entire html page in ipdeny, which holds the links with txt files that holds the ranges. After we find the right one by initials, we download it. Manipulate the name for the needs of automation.. After all the kind people from Ipdeny are not obliged to use naming schemas suited just for me and my needs. Remember always to forward errors in separate file or just redirect them to /dev/null. Otherwise, debuting is nightmare. And further more, you risk unnecessary script interruptions.
After that we proceed to prepare that list as a chain of rules.

function prepare_download_zones()
{
    if [ $( du -hb $PWD/zones/txt | col1 ) -eq 2 ];
    then
        curl -s 'https://www.ipdeny.com/ipblocks/' | \
            sed -n 's/.*href="\([^"]*\).*/\1/p' | \
            awk '$0="https://www.ipdeny.com/ipblocks/"$0' | \
            head -n -5 | \
            sed -e '1,11d' | \
            grep "$1" > $PWD/zones/download

        while IFS="" read -r p || [ -n "$p" ]
        do
            wget "$p" -P $PWD/zones/txt/ 
        done < $PWD/zones/download
        for f in $PWD/zones/txt/*.zone;
            do
                mv "$f" "$(echo "$f" | sed 's/aggregated.//g' )" 2>/dev/null;
            done
        prepare_zone_scripts_file $1
    else
        for f in $PWD/zones/txt/*.zone; 
        do 
            mv "$f" "$(echo "$f" | sed 's/aggregated.//g' )" 2>/dev/null;
            done
        prepare_zone_scripts_file $1
    fi
}

Here we build our service. This is needed because iptables rules are not usually up open reboot. And its not very automated method if will have to run it every restart… So will need to create it first. This is by RC_locale. You can do it with system.d for more standert approach. But honestly, this will do the trick as well… And I chose rc, because the syntax is more familiar to me.

function rc_locale_service()
{
    touch /etc/systemd/system/rc-local.service
    touch /etc/rc.local
    echo "#!/usr/bin/bash" | tee -a /etc/rc.local
    cat <<EOF > /etc/systemd/system/rc-local.service
[Unit]
 Description=/etc/rc.local Compatibility
 ConditionPathExists=/etc/rc.local
[Service]
 Type=forking
 ExecStart=/etc/rc.local start
 TimeoutSec=0
 StandardOutput=tty
 RemainAfterExit=yes
[Install]
 WantedBy=multi-user.target
EOF
    chmod +x /etc/rc.local
    systemctl enable rc-local
    systemctl start rc-local
}

Here we finally start to prepare chain rules by zones. For each line, we set new rule in the chain. And add it to a new script that is in the folder structure, called “zone_scripts.sh” While we are reading the txt file with the ranges, we append them in the sell script file with the ipset syntax. And just for more presentable output, we print it while appending it.. Its not need but… Why not :D
 After that we execute the newly created script that will load the new chain in IPset. And then we create new iptebles rules list and apply it for the entire tcp protocol for your eth0 (if need be, just change the name of the adapter). Then we match the chain rules with those in ipset. And add them to our new rule list with rule RIDIRECT to all ports on server. Lastly we set everything else to DROP

function prepare_zone_scripts_file()
{
    zone="zone-"$1
    line_one='$IPSET create '$zone" hash:net\n"
    line_two='$IPSET flush '$zone"\n"
    if [ $( du -hb $PWD/zones/zone_scripts.sh | grep -c 1) -eq 0 ];
    then
        cat <<EOF > $PWD/zones/zone_scripts.sh
#!/bin/sh
IPSET="/usr/sbin/ipset"
EOF

    echo '$IPSET' "create zone-$1-zone hash:net" | tee -a  $PWD/zones/zone_scripts.sh
    echo '$IPSET' "flush zone-$1-zone" | tee -a $PWD/zones/zone_scripts.sh
    fi
    for f in $PWD/zones/txt/*-zone;
        do
        while read -r line; do $( echo '$IPSET' "add zone-$1-zone $line" | tee -a  $PWD/zones/zone_scripts.sh ); done < $f 2>/dev/null
        done
    tail -n +2 zones/zone_scripts.sh | tee -a /etc/rc.local
    $( sh $PWD/zones/zone_scripts.sh )
    iptables -N whitelist
    iptables -A INPUT -i eth0 -p tcp -m state --state NEW -j whitelist
    iptables -A whitelist -m set --match-set zone-$1-zone src -j RETURN
    iptables -A whitelist -j DROP
}

This is it. The entire script will be available here. 
Now I want to make several notes. First of all I intently did not make dynamic the name or our rule list. It is better to apply to it, then to add new. This will make problems in future use… But if you wish, go for it. Change it… Play with it :)
 Secon (again intentionally) I did not apply actually rules permenatly. No meter that we created the service. This was for illustrating purposes. After all, I don’t want to stop your net or something :D But you can easy do that by adding a folder in “etc” called iptables. And you use the iptables-save to append the situation in “/etc/iptables/rules.v4”.
Third: this script (as it is written ) does not apply the rules for UDP protocol. Again this is easily fixed. Basically add it as additional command in the set.
This concept can be applied for a web servers… More importantly it can be applied with dnsmasq instead of iptables and be used as blacklist for domain names. For example you can just use dnsmasq and ipset to add a chain rules with ad list of YouTube and block the ads in your server… Or even maybe use it over bind9 and block ads for your entire network…
I hope that this article will help someone in need. And honestly I wanted to write it just to illustrate, on practice, automation of tasks. The principles are fairly easy to understand when you have a grasp on the end result…. What you want to achieve.
Fill free to take, play, overwrite etc.. And if you find it useful, drop me a line in the comments… Its always appreciated. :)