Rails 6 released with many awesome features and action mailbox was one of them that has come to make the life easier. From Official Action Mailbox Guide:
Action Mailbox routes incoming emails to controller-like mailboxes for processing in Rails. It ships with ingresses for Mailgun, Mandrill, Postmark, and SendGrid. You can also handle inbound mails directly via the built-in Exim, Postfix, and Qmail ingresses.
Basically, Action Mailbox can be used to forward all incoming emails to your Rails app and process it further as required like storing attachments, creating records from the email body in you database and many more.
And today, we will be implementing Action Mailbox with SendGrid.
Requirements
- Setup Action Mailbox with SendGrid using the official Rails documentation
- Update DNS records to forward emails received in the mailbox towards our Rails app
- Test integration in development with built in UI provided by Rails
- Test integration in development with NGROK to ensure seamless production release
Tested and working in
- Ruby 3.0.0
- Rails 7.0.2.4
- Action Mailbox 7.0.2.4
You should have
- Existing app built with Rails 7 or higher
Let's start integrating Action Mailbox with SendGrid in our Rails app now.
Step 1: Setup action mailbox
We will be following instructions from the Official Rails Guide for Action Mailbox.
- Install migrations needed for InboundEmail and ensure Active Storage is set up:
$ rails action_mailbox:install
$ rails db:migrate
Step 2: Add Ingress Configurations
Tell Action Mailbox to accept emails from SendGrid by adding the following to both "development.rb" and "production.rb"
# config/environments/development.rb & config/environments/production.rb
config.action_mailbox.ingress = :sendgrid
Step 3: Generate Password for authenticating requests
First of all, we should generate a strong password that Action Mailbox can use to authenticate requests to the SendGrid ingress.
You can add any strong password or let Rails generate it for you. You can log into Rails console and generate a password for you:
> rails c
irb > SecureRandom.alphanumeric
# => "Kk9YGvzdPN69bfiu"
After that you can use rails credentials:edit
in the command line to add the password to your application's encrypted credentials under action_mailbox.ingress_password
, where Action Mailbox will automatically find it:
action_mailbox:
ingress_password: YOUR_STRONG_PASSWORD
If you are using nano editor you can edit credentials with following command:
$ EDITOR="nano" rails credentials:edit
Alternatively, you can also provide the password in the RAILS_INBOUND_EMAIL_PASSWORD
environment variable.
If you are using figaro
gem you can add the following to your "config/application.yml":
# config/application.yml
RAILS_INBOUND_EMAIL_PASSWORD: 'YOUR_STRONG_PASSWORD'
Step 4: Setup a Mailbox
Now we should setup a mailbox that will process all incoming emails through our Rails app.
You can generate a new mailbox with:
$ bin/rails generate mailbox forwards
This will create forwards_mailbox
inside app/mailboxes
# app/mailboxes/forwards_mailbox.rb
class ForwardsMailbox < ApplicationMailbox
def process
end
end
Step 5: Whitelist email domains
We can configure our application_mailbox
to accept all incoming emails to our Rails app and forward it to our forwards_mailbox
for further processing.
Action Mailbox also accepts regex to whitelist domains or match certain emails. Let's look at how we can configure all these alternatives:
- Accept all incoming emails
# app/mailboxes/application_mailbox.rb
class ApplicationMailbox < ActionMailbox::Base
routing :all => :forwards
end
- Accept all emails from single domain
# app/mailboxes/application_mailbox.rb
class ApplicationMailbox < ActionMailbox::Base
routing /.*@email-domain.com/i => :forwards
end
- Accept email from multiple domains
# app/mailboxes/application_mailbox.rb
class ApplicationMailbox < ActionMailbox::Base
routing /.*@primary-email-domain.com|.*@secondary-email-domain.com/i => :forwards
end
This regex matching is telling application mailbox to forward all emails coming from @email-domain.com
to our forwards_mailbox
. For e.g. if we configure it to be /.*@gmail.com/i
and our Rails app receives email to john-doe@gmail.com
then it will be forwarded to our forwards_mailbox
where we can further process it since this email matches with the pattern @gmail.com
.
NOTE: Your mailbox name should match the name you've given it in the routing params i.e. forwards
will route to forwards_mailbox
.
Step 6: Test in development
Action Mailbox provides it's own set of UIs to test inbound emails in the development environment. To access this, let's fire up the Rails server first:
$ rails s
Visit Action Mailbox Inbound Emails Localhost URL and click on New inbound email by form
. Fill in all required details like From, To, Subject and Body. You can leave other fields blank.
Before clicking on Deliver inbound email
, let's add byebug
(or any other debugging breakpoint e.g. binding.pry) to our process
method so we know action mailbox is actually forwarding our emails to the right place.
# app/mailboxes/forwards_mailbox.rb
class ForwardsMailbox < ApplicationMailbox
def process
byebug
end
end
You should make sure that email in From input box matches the email domain configured. Now when you click Deliver inbound email
, the execution of the server process should stop at the process
method since we have a breakpoint there. This means action mailbox is correctly forwarding incoming emails and our configurations are correct. You can perform further process as required in your app now.
But wait. Dang, there is an error from Rails while testing inbound emails in development!
Let's dig into what is happening.
Issue with Inbound Action Mailbox Testing in Development
Error reads: "undefined method 'original_filename' for "":String" and "NoMethodError in Rails::Conductor::ActionMailbox::InboundEmailsController#create"
Looking at the code in Action Mailbox of core Rails, I found out that this error is occurring because controller is trying to process the empty attachment further. But finding out why Rails was submitting empty attachment when we haven't chosen any attachment was hard. Note that, this also happens even if you choose one or multiple attachments.
In params, we get this "attachments"=>[""]
and controller is trying to process it further with the following code:
private
def new_mail
Mail.new(mail_params.except(:attachments).to_h).tap do |mail|
mail[:bcc]&.include_in_headers = true
mail_params[:attachments].to_a.each do |attachment|
mail.add_file(filename: attachment.original_filename, content: attachment.read)
end
end
end
Here, we are getting error in the line mail.add_file(filename: attachment.original_filename, content: attachment.read)
because "attachment" is empty string i.e. "" and not an object which has properties like "original_filename". Hence the error.
After looking into controller, my next stop for debugging the error was to look into the view because it shouldn't have sent the empty attachment in the first place.
View was just using a normal file_upload tag:
<div>
<%= form.label :attachments, "Attachments" %><br>
<%= form.file_field :attachments, multiple: true %>
</div>
There couldn't be any issue here, so I looked into the rendered HTML in the webpage and found out that there was a hidden tag for attachment:
<input name="mail[attachments][]" type="hidden" value="" autocomplete="off">
<input multiple="multiple" type="file" name="mail[attachments][]" id="mail_attachments">
Hence, the form is submitting empty attachment to the controller.
This problem could be solved in controller by filtering out attachments that are empty and I was near to submitting a PR to Rails. But then I thought, if I am getting this issue, there are obviously other developers who have been into this since this is an issue in Rails core and not in the code I have written.
Searching further, I found this issue titled Action Mailbox Conductor throws NoMethodError when creating inbound email submitted to Rails Core Github.
And if there is an issue, there must also be a PR. YES, there was one already titled Cannot deliver new inbound email via form but it hadn't been merged yet.
But for this tutorial and until PR is merged, we need this to work in our app. So, I was looking into how I can resolve it the best and searching for the solution that would work for all of us and not just me.
Scrolling further into the issue, I found a monkey patching very suitable for our use case.
Monkey Patching the issue
Add the following to your config/application.rb
# monkey patching to resolve the issue of action mailbox inbound email sending empty attachment
config.to_prepare do
Rails::Conductor::ActionMailbox::InboundEmailsController.class_eval do
private
def new_mail
Mail.new(mail_params.except(:attachments).to_h).tap do |mail|
mail[:bcc]&.include_in_headers = true
mail_params[:attachments].to_a.compact_blank.each do |attachment|
mail.add_file(filename: attachment.original_filename, content: attachment.read)
end
end
end
end
end
Don't forget to restart the server and reload the page. After that you can submit the form again.
Voilà!! It works 🥳.
Now, your server should have stuck in the debugging breakpoint.
That's it, we have now successfully setup action mailbox and tested in development.
Now let's test using NGROK so we know that our configuration will work seamlessly (pretty much) in our production environment.
Step 7: Setup NGROK
Let's setup NGROK in our local machine:
-
Download the application
You can download the application from this download link.
If you are on MacOS, I highly suggest downloading NGROK using homebrew with the command
brew install ngrok/ngrok/ngrok
. It's easier than manual download and also don't normally give off any issue. -
Serve your app using NGROK URL
While keeping the rails server running as it is, open a new tab in your command line.
You can then run the command
ngrok http 3000
, which will give you an URL connecting your local Rails app running on port 3000 to the internet. You should look at the URL besides the "Forwarding" option, it will be something similar toForwarding https://e73a-27-34-12-7.in.ngrok.io -> http://localhost:3000
When running the NGROK, you should see a screen similar to the screenshot below:
-
Access the Rails app with NGROK URL
Open the URL you got before from NGROK e.g.
https://e73a-27-34-12-7.in.ngrok.io
in your browser and you should be able to see the Rails welcome screen or whatever your default page for the app is.But, but, there is an error again 😭
Hah, don't worry. I have got you covered.
You should be seeing the Error UI similar to what is in the screenshot below:
This happens because of missing "auth token" which you can get after signing up to NGROK for free.
-
Sign up to NGROK
You can sign up to NGROK using this signup link.
-
Add NGROK auth-token to local configuration file
After signing up, you are presented with a dashboard and you can copy the auth-token from setup-and-installation step number 2 called "Connect you account"
Or you can follow this link to your auth token page.
Copy the token given and run the following in your command line:
$ ngrok config add-authtoken <your-authtoken>
Now restart your NGROK server and go to the new URL provided.
NOTE: URL changes each time you restart the NGROK server unless you use pro version and pay for the static URL.
What, Error? Again!!! 🤕
-
Resolving blocked host in Rails app
After accessing the NGROK URL, you should see an error page similar to the one below:
This is because Rails blocks https access in development and unauthorized URLs overall.
Let's add the NGROK URL to "config/environments/development.rb"
# config/environments/development.rb config.hosts << "add-your-ngrok-url"
Restart the rails server and reload the page in the browser. Now, you should be able to see the default page of your Rails app. This is what I see in mine since it's a new application just for this blog:
-
Authorize all NGROK URLs in development
Above, we only allowed current URL provided by NGROK and as I have already said this URL changes each time we restart the server.
Changing it every time we restart the server is a hassle so we will add regex which will allow matching NGROK URLs to connect to our Rails app in development.
Let's replace previous configuration with the following:
config.hosts << /.+\.ngrok\.io/
Now, if you restart the rails server you should still be able to access the default page in your app. Also, try with restarting the NGROK server and accessing the page again which you should still be able to without getting blocked host error.
Step 8: Authenticate domain in SendGrid
You can follow the official SendGrid tutorial to authenticate your domain in SendGrid.
Part of the tutorial also goes through setting up MX records which we will go into detail here.
You can authenticate your domain by following this link to the Domain Authentication page
You will receive list of CNAME and Values similar to what is listed below in the process where prabinpoudel.com.np
and em1181
will be different:
- em1181.prabinpoudel.com.np
- s1._domainkey.prabinpoudel.com.np
- s2._domainkey.prabinpoudel.com.np
We will come back to this page after next step again so don't close the page yet.
Let's go to our DNS provider's dashboard and configure these records first.
Setup DNS Records
We need to add DNS records from SendGrid to our DNS provider so our email is actually being processed by SendGrid and routed to our Rails app with Inbound Parse Hook.
I use CloudFlare, so I will be showing you process to setup MX record using the settings from CloudFlare as an example.
-
Go to DNS tab from the left menu
-
Click on "Add Record" and choose MX from the dropdown then add the following values to each field
a. Name: "@"
b. Mail Server: "mx.sendgrid.net"
c. TTL: "auto"
d. Priority: "10"You can also find the instruction for adding MX record in the tutorial to setup Inbound Parse Hook from SendGrid.
-
Click on "Add Record" again and add all three CNAME records we got previously while authenticating the domain one by one
Copy values from authenticating the domain page add them to CloudFlare:
a. Type: CNAME
b. Name: value from CNAME
c. Target: value from VALUE
d. Proxy Status: Turn the toggle button off (it will be on by default)
e. TTL: Auto -
Go back to domain authentication page and click on "I've added these records" and click on "Verify" button
If everything was copied over correctly, you will see a page with the information "It worked! Your authenticated domain for prabinpoudel.com.np was verified."
Else you will get errors and you will have to fix those before moving forward.
Step 9: Configure Inbound Parse in SendGrid
We will be following the official SendGrid doc for configuring inbound parse hook to forward inbound emails to /rails/action_mailbox/sendgrid/inbound_emails
with the username "actionmailbox" and the password we generated just before this.
- From your SendGrid Dashboard click Settings, and then click Inbound Parse. You are now on the Inbound Parse page. Or you can click on this Inbound Parse Link to go there directly.
- Click "Add Host & URL"
- You can add/leave the subdomain part as required. I have left it blank because I don't have any subdomain just for receiving emails
Under "Domain", choose your domain name that you just verified
-
Under the URL we will have to construct one and add it
The format for the URL should be:
https://actionmailbox:<your_action_mailbox_ingress_password>@<rails_app_nginx_url>/rails/action_mailbox/sendgrid/inbound_emails
For e.g. it will be
https://actionmailbox:my_strong_password@5829-2400-1a00-b050-3fb6-b0ce-5946-b9be.in.ngrok.io/rails/action_mailbox/sendgrid/inbound_emails
for my Rails app.In production it can be a different URL so you should replace
rails_app_nginx_url
with the URL from where your Rails application is accessible to the internet. Check "POST the raw, full MIME message" and click on Add
Now we are ready to test our integration with live email using SendGrid and Ngrok.
Step 10: Test if MX records are recognized by the internet
Before we test our integration with live email, we need to make sure that MX records are recognized by the Internet.
It may take some time for DNS records to be recognized throughout the world so email forwarding may yet not work for you. The maximum time period until this happens is 24 hours.
You can test if DNS records for your domain is working correctly and recognized from the website MX Toolbox
- Add your domain name e.g. prabinpoudel.com.np
- Click on "MX Lookup"
You should see "DNS Record Published" status in the test result table
Step 11: Test incoming email with SendGrid and NGROK
Finally, we are now at the last step. We will now send email to our mail server and we should receive it in our local Rails app and server should stop in our debugging breakpoint.
From your favorite email provider e.g. Gmail, send a test email to your domain e.g. for me I will test it via sendgrid-test@prabinpoudel.com.np.
It takes some time to process the email by SendGrid and receive in our Rails app, maximum ~1 minute.
You can check if the email is being received by SendGrid or not from Parse Webhook Statistics
Tada!! 🎉
You should have received the email and rails server must have stopped in the debugging breakpoint.
Conclusion
Congratulations!!! You have come a long way and gone through a lot of process to integrate Action Mailbox with SendGrid.
You can find a working app for this blog at Action Mailbox with SendGrid. You can view all changes I made for configuring SendGrid with Action Mailbox in the PR: Setting up Action Mailbox with SendGrid
Next, you can deploy the app to staging or production and add new Inbound Parse URL in SendGrid to point to the URL of those applications.
If you have any confusions, suggestions or issues while implementing any steps in this email, please let me know in comment section below and I will do my best to help you.
Thanks for reading. Happy coding and tinkering!
References:
- Action Mailbox (Official Documentation)
- Using Action Mailbox in Rails 6 to Receive Mail
- Action Mailbox Conductor throws NoMethodError when creating inbound email
- Cannot deliver new inbound email via form #44008
- Setting Up The Inbound Parse Webhook
- How to set up domain authentication for Twilio SendGrid
Image Credits: Cover Image by erica steeves from Unsplash
Top comments (0)