DEV Community

Osman Akin
Osman Akin

Posted on

Setting Bulk Mail Server — using Sendy and AWS (SES & Ubuntu Web Server)

I started freelancing this month and was assigned this project by a client. It's quite interesting and not a stressful process.

I am grateful to this genius who shared their experience. I decided to write about it due to some changes i observed while setting this up for my client. Also, There has been some update on amazon, the gui and some of the software installed. I attached a link to their blog below.

Let's start:

  1. Login to AWS account.
  • Select the region where emails will be generated. Although I ended up using the us-east-1 region. (At the right hand corner you will see your region and once you click on it you can select whichever region you want)
  • Next - Click on services - Scroll and click on Networking & content delivery - click on VPC then on the left panel under security - you click on security group.
  • Alternatively on the search for service - type VPC or EC2 - Scroll and you will see Security group

Click Create Security Group.

Enter the following details and rules:

Name: SendyMail-sg
Description: HTTP, HTTPS, SSH, Virtualmin
VPC: default
Inbound Rule: HTTP - port 80, (Anywhere-Ipv4 or Custom)
Inbound Rule: HTTPS - port 443, (Anywhere-Ipv4 or Custom)
Inbound Rule: SSH - port 22, For security best practices (select My IP) — this will be used to SSH into the computer and only really should be your IP address. If you have multiple locations that you work from, you can add all.
Inbound Rule: Custom TCP - Port 10000, For security best practices (select My IP) — this will be used to access Virtualmin
Outbound Rule: Leave as default, There is no need to create any Outbound rules.
Tag: key: Name, Value: sendyserver-sg

Image description

Click Create.

Image description

All done!

NEXT

Create an EC2 Instance

Select region in my scenario, I stuck to N. Virginia (us-east-1). (At the right hand corner you will see your region and once you click on it you can select whichever region you want)

Next - Click on services - Scroll and click on compute - Under Compute - click on EC2.
Alternatively, on the search for service bar - type EC2 - Select EC2 from the services menu.

Image description

On EC2
From the left-hand side menu, select Instances.
Click Launch Instance.
Select Launch Instance and click on launch instance
You will then be asked to Choose an Amazon Machine Image (AMI) and instance type, instance name e.t.c.

Image description

Name - Sendywebserver
Under Application and OS images (AMI):
Select Ubuntu
Choose, Ubuntu, 18.04 LTS (HVM).
Architecture type - 64-bit (x86) - (Leave default)

**
Instance type:
**Choose t2.small

Key Pair login:
There are 2 options - Create a new key pair or choose existing Keypair.

They keypair are your keys to connect to an ec2 instance. Please keep it safe.

If you haven't use aws before or a new account, Create a new key pair, give it a name and download the file or choose existing Key Pair

Network Setting:
click on edit, select vpc, subnet, enable auto-assign public-ip.
select existing security group - choose the security group created earlier.

Configure storage:
I selected 30GB - gp2

Advanced details:
Termination protection - Enable (Protect against accidental termination.)

There are a lot of settings here but I won't be going into them for now.

Image description

Verify all selected are correct.

click Launch

Image description

Image description

Setting up SSH Access to your AWS Ubuntu Server

I use WSL on my Windows PC.

On linux : Open Terminal window

copy or mv the keypair to your current working directory -

Input: chmod 400  then press Enter

Image description

Login to the Ubuntu server

Input: ssh -i and your webserver public ipv4 dns. Then press Enter

e.g ssh -i cloudnet.pem ec2-35-174-106-112.compute-1.amazonaws.com

You will be asked to accept the ECDSA key fingerprint: type and Enter: Yes

Image description

You should now be logged into your Ubuntu server.

Image description

There are some extra steps which are efficient to save your fingerprint so it does not ask you every time you want to login and also how to copy your keypair into sshd files so you don't have to type your keypair everytime you want to login..

Attach Elastic IP To Your AWS Server

An Elastic IP address is a static/constant IPv4 address.designed for dynamic cloud computing.
Elastic IPs are totally free, as long as they are being used by an instance. However amazon will charge you if reserved and not in use.

Task 1: Attach an elastic ip address to your server
Task 2: Creating a DNS record in Cpanel or Godaddy or route 53.

On your EC2 dashboard on AWS, Under Network & Security - select Elastic IPs.

Image description

Click Allocate Elastic IP address.
Select Amazon’s pool of IPv4 addresses.

Image description

Click Allocate.

Image description

Now click on the selection box to select the newly allocated IP address then click on Actions, A list of options will be displayed, click on Associate Elastic IP Address.
Select your Ec2 Instance (Ubuntu server for sendy)
Image description

Click on Associate.

Go back to your Ec2 Dashboard, and select the ubuntu server for sendy, Verify the Public IP address is now your associated Elastic IP address.
 
Associating Elastic IP to your domain’s DNS records

You have to associate your domain or subdomain with this IP address. This is the domain you will use to access Sendy. It's  recommended creating it as a subdomain to your website, e.g sendy. since your main domain is where you would ideally host your website.

Now login to your domain host control panel and navigate to DNS settings/records.

*Create an A record. *

Record Name: sendy.
Record Type: A Record
Content|IP address: Paste the Elastic IP address.
TTL: Leave as default or set to 300

Click Add Record.

Updating to your sendy server.

Open your terminal window (Mac or Linux), Putty (Windows)

You will have to ssh into your ubuntu webserver via ssh using your new Elastic IP address as you have already been logged out since you changed the public ip address.
Image description

Note: This login style is if you haven't store your keypair at sshd

Once logged in - We have to update the operating system.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get full-upgrade -y

Image description

sudo apt-get full-upgrade
Image description

Configuring & Installing Ubuntu's Host Files For Virtualmin

You have to change the hostname for the server: sendy.

sudo nano /etc/hostname
Image description

remove the private ip address and change to your new hostname
Image description
Save and Exit

Now you update the host file: Add 127.0.1.1 sendy.
sudo nano /etc/hosts
Image description
Add 127.0.1.1 sendy.domain.com
Image description

After this update kindly reboot for effective changes.

sudo reboot! Image description

Installing Virtualmin

After Reboot. Login into the server via ssh or putty. you will server name has changed from the private ip address to the subdomain name.
Image description

You need to download and install the virtualmin file.

wget http://software.virtualmin.com/gpl/scripts/install.sh
Image description

Once completed. Run the install file with this command:

sudo sh ./install.sh --bundle LEMP -y
Image description

It will display some messages about Memory size. : if you had selected the free tier (t2.micro), it will complain about memory is below 1024 MB, and full installation might not be possible thereby you may have to use --minimal

It will show other information about creating swap. Enter y.

Most times you will be asked for FQDN -  enter your sendy url.
The one i used: sendy.

The Installation has 3 phases and takes some time to complete the installation.
Image description
Image description

Once installation is complete

Install or update php & phpmyadmin
Image description

We are using nginx so don't select either apache or lighttpd, select ok.
Image description

Then you change Virtualmin’s root password — sudo /usr/share/webmin/changepass.pl /etc/webmin/ root
Image description

sudo rebootImage description

Post Installation..

Once the server has rebooted.

Login and go to your internet browser and enter the virtualmin installation url:

In my task, i used sendy.:10000

It will show an insecure connection. click on ADVANCED then PROCEED TO sendy.:(unsafe)

NOTE: if you are unable to access the page. check if your server has rebooted and is active. else try using your Elastic IP Address: http://01.23.45.67:10000

Virtualmin Post-Installation Wizard

You will have to login with your username and password.

username : root
Password : ! This is the password you created earlier after the installation of Virtualmin in previous step.

Once logged in, you will see the post-installation page:

Page 1 - introduction. Click Next.
Page 2 - Memory use settings: Preload Virtualmin libraries:- NO and Run email domain lookup server:- YES. Click Next
Page 3 - Virus scanning setting: Run ClamAV server scanner:- NO
Page 4 - Spam filtering setting: Run SpamAssassin server filter:- NO
Page 5 - Database servers: Run MySQL database server:- YES and Run PostgreSQL database server:- NO
Page 6 - MySQL root password:- Create a root password for MySQL. (NOTE: This password should be different from the root password you created for Virtualmin)
Page 7 - MySQL database size:- I selected Large system (1G) on which MySQL is heavily used
Page 8 - Primary nameserver:- Your sendy subdomain will be displayed, verify and click next
Page 9 - choose how your password should be stored:- select only store hashed passwords.
click next

On the All done page, click Next.

Now Virtualmin will have to be refreshed. Click Re-check and refresh configuration.

Some installation messages will appear and once all the steps are complete, you will see: your system is ready for use by virtualmin.

Within the left-side. click Virtualmin menu, select system settings and then features and plugin.

Make sure the appropriate features and plugins are selected.

NOTE: In case you ever wish to re-run the post-installation wizard.

Click on Virtualmin - System Settings - Re-Run Install Wizard.

Creating a server on Virtualmin

On the left-hand side of the webpage. select Virtualmin then click on create virtualmin server

Enter the following details:

Domain name: sendy.
Administration password: Input a new password.
Administration username: Leave as Automatic, it will be the username created with the subdomain of your Sendy URL. In this case, sendy.

Under Enabled features, I had the following checked,
DNS domain enabled
Nginx website enabled
MySQL database enabled
Webmin login enabled

Leave others as default then

Click Create Server.

Various messages should now appear and your virtual server should be created

Once created at the left-hand side of the web page under Virtualmin. You will see your new virtual server listed.

Click on virtual server summary and verify that both Administration username and Administration group is your subdomain username e.g sendy. while, the Home directory will be set to /home/yourselectedsubdomain e.g /home/sendy

Installing An SSL Certificate Within Virtualmin

Ón the left-hand side of the webpage, under virtualmin. Click on Server Configuration and then Manage SSL Certificate

Make sure the SSL website is enabled on features and plugins..

On the Manage SSL Certificate page. Click on Let's Encrypt

click the box on Domain names listed here. Enter your sendy installation url inside the Request Certificate for box.
Click Request Certificate

There shouldn't be an error if done correctly and using an amazon ec2 instance.

Click on Current Certificate in the top menu.

Click copy to Virtualmin then Log out and close the tab or browser.

Clear cookies and cache and also flushdns if necessary.

Open a new tab or re-launch a new browser and input the virtualmin URI: sendymail

It might ask you to accept encryption. Once you login it will be encrypted.

Configuring Nginx on AWS web server

Login into the server via ssh or putty at the command line or terminal. enter: sudo nano /etc/nginx/sites-available/yoursendyURL.conf
Image description

Use the downward arrow key to scroll through the file until you reach the following line:

fastcgi_param HTTPS $https;

Below, give you line space by pressing ENTER twice, Then paste the following script within the extra blank lines:

location = / {
index index.php;
}
location / {
if (!-f $request_filename){
rewrite ^/([a-zA-Z0-9-]+)$ /$1.php last;
}
}
location /l/ {
rewrite ^/l/([a-zA-Z0-9/]+)$ /l.php?i=$1 last;
}
location /t/ {
rewrite ^/t/([a-zA-Z0-9/]+)$ /t.php?i=$1 last;
}
location /w/ {
rewrite ^/w/([a-zA-Z0-9/]+)$ /w.php?i=$1 last;
}
location /unsubscribe/ {
rewrite ^/unsubscribe/(.)$ /unsubscribe.php?i=$1 last;
}
location /subscribe/ {
rewrite ^/subscribe/(.
)$ /subscribe.php?i=$1 last;
}
location ~* .(ico|css|js|gif|jpe?g|png)(\?[0-9]+)?$ {
expires max;
log_not_found off;
}

Image description

Save and restart nginx - sudo systemctl restart nginx.service
Image description

configuring and securing MYSQL database on AWS Web Server

SSH into your ubuntu server and launch the terminal.

Enter: mysql_secure_installation
Image description

You would be asked some questions, configuring and securing your MySQL Server.

Validate Password plugin: n
Root password: The one you set in the post-installation wizard of Virtualmin(Incase you misplaced it, you can login to virtualmin to generate a new one).
Remove anonymous users: y
Disallow root login remotely: y
Remove test database and access: y
Reload privilege tables: y

You need to make some changes on the mysql config file for sendy to work properly.

on Terminal, Enter: sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf
Image description

scroll to the end of the file and paste this script: sql_mode = "STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
Image description
Save and exit the file

Restart MYSQL service

sudo service mysql restart
Image description

Retrieving Sendy Database Password

log in into Virtualmin by navigating to Your Virtualmin URL within your browser:

On the left-hand side, click on virtualmin then click on Edit Databases.

On Edit Databases - click on passwords and select your sendy username.

You will see a message - change database password and a key symbol.

Hover over the key symbol to get your username and password. Kindly save them in the notepad as you will make use of it.

Installing Sendy On Your AWS Server

Download Sendy with the link received in your email at the time of license purchase.
You can download the sendy compressed file directly to your local computer or AWS web server..

I downloaded the sendy file to my local computer, unzip it and transfer to AWS Ubuntu server using (Filezilla client, winScp or cyberduck)

NOTE: you will have to edit one of the files. You can edit the file on your local computer before using FTP to transfer files to Web Server or You transfer the file to web server and navigate to the file to make changes.
NOTE: Make sure you the files to this directory on the EC2 server at /home/sendy/public_html

navigate to includes/config.php file on local computer or aws ubuntu server and make these changes.

sudo nano /home/sendy/public_html/includes/config.php
Image description

The URL within the (APP_PATH) line should be replaced with Your Sendy Installation URL (including the https:// prefix ).

$dbHost should be set to localhost
$dbUser should be set to Your sendy subdomain retrieved from virtualmin
$dbPass should be set to the Sendy database password retrieved from virtualmin
$dbName should be set to Your subdomain
Image description

Make sure you save the config.php file.

<> Make sure all files are copied. especially the .htaccess file. ON the terminal change the permission for uploads file to 777.

sudo su sendy
cd ~
cd public_html
ls -la | grep uploads
chmod 777 uploads
exit

Install some libraries that are needed by Sendy:

sudo apt-get install libcurl4-openssl-dev php7.2-curl
Image description

sudo reboot

Once the web server is back on and running. navigate to your browser and enter the sendy url e.g sendy.

You should see an install page now..

Fill in all information. then, click Install now.

Creating AWS SES Access Keys For Sendy

Login to your AWS console.
Go to the search box and type IAM using the Services menu.
Make sure that your region is set correctly in the top-right.
Click on Users on the lefy-hand side od the page. Image description
Click Add User.
Enter the following details:
User name: sendy
Access type: Programmatic access
Image description

Click Next: Permissions.
Select Attach existing policies.
In the search type sesf . Select AmazonSESFullAccess.
In the search type snsf . Select AmazonSNSFullAccess.
Image description
Click Next: Tags. This section can be skipped.
Click Next: Review. Review your settings.
Image description
Click Create User

Your access keys should now be created.. click show to see your secret access key or you download the csv.

Image description

Switch back to Sendy — sendy. at the top right corner drop down menu, select Settings.
Under Amazon Web Services Credentials. Paste your AWS Access Key ID and AWS Secret Access Key. Select the AWS region under Amazon SES region, Make sure it's the same region selected on AWS console.

Click Save.

Verifying Domain Within AWS SES (Simple Email Service)

By default, You will be placed in a sandbox environment with 200 quota, and can only send limited emails to and from verified emails or domain in AWS ses and Sendy. you will have to request for more quota by contacting AWS.

Sign in your AWS console and on the search box, type ses and click on it.

Image description
On AWS SES webpage. You have 2 options:
on the right, you can click on create identities.
Image description
while, on the left you can click on verified identities under configuration.
Image description
Let's create an identity

Identity details:
Select Domain and enter your domain( you can also enter your domain, but its much better verifying your domain)
Verifying your domain:
Click the drop down option on Advanced DKIM settings
choose identity type: Easy DKIM

DKIM signing key length: Select RSA_2048_BIT

Publish DNS records to Route53: Click Enabled if you are using route53 else do not enable.

DKIM signatures: Select Enabled

Image description
Image description

Once that's done. Create Identity
Image description

To verify ownership of this identity, DKIM must be configured in the domain DNS settings using the CNAME records provided.

Download the CSV records and add them to your DNS.

Navigate to your domain control panel and create a CNAME and TXT record.

Check your email for verification and confirmation of the domain being verified.

Troubleshooting and fixing 413 error

So while my client was tryin to upload a csv file above 2MB, he encounter an error 413.

Here is how to solve it. Navigate to your Ubuntu webserver using nano editor.

we are going to increase the "client body size" on Nginx and also increase the upload maximum filesize on php.

For Nginx - sudo nano /etc/nginx/nginx.conf
Within the http {
client_body_max_size 500m;

Image description

increase the "client body size"

Image description

Restart nginx
sudo systemctl reload nginx.service

For php - sudo nano /etc/php/7.2/fpm/php.ini
Under File Uploads and Data Handling

;The maximum size of an uploaded file.
upload_max_filesize = 500M

;Sets max size of post data allowed. This setting also affects file upload. To upload large files, this value must be larger than upload_max_filesize
post_max_size = 500M

Under File Uploads
Image description

Under Data Handling
Image description

Restart php7.2

sudo systemctl restart php-fpm

Thank you so much.. I will be updating this blog from time to time with more information and pictures.. I've got a cloud support task and alsoa new project on Microsoft virtual desktop infrastructure (VDI). i hope to write more blog soon.


For reference and incase you need more guidelines:-

Maynard patton: https://colcol.co.uk/introduction-to-installing-sendy-and-virtualmin-on-aws/

Neha M: https://levelup.gitconnected.com/setting-up-your-own-bulk-mail-server-using-sendy-and-aws-20d05ee01362

I'm can't thank this authors enough.


You can see my other blog:
AWS SAA-C02
https://medium.com/@ossniper/aws-solution-architect-saa-c02-passed-resources-used-thoughts-73016f2459e2

ebgp peering
https://medium.com/@ossniper/ive-decided-to-write-about-bgp-ip-sla-pbr-and-ospf-as-i-will-be-updating-this-blog-from-time-to-7ce3244a10ff

Top comments (0)