DEV Community

Discussion on: SNS vs SQS? AWS Messaging Services - Know the difference

kp profile image
KP

@andrewbrown thank you for the response :)
I've tried this.

I am using SES to send email and am trying to get notified of bounces, complaints and successful deliveries by:

  1. Email forwarding
  2. SNS HTTP(S) POST notification (using this package to receive the notifications but I never get any: github.com/jdavidbakr/mail-tracker)

Help with either (but especially the HTTP SNS POSTs) would be great. Note that the ngrok url / tunnel in the examples below was functional, pasting the url for discussion sake...I can resurrect it if you need me to.

My settings:

  1. From us-west-2.console.aws.amazon.com/s...
    Screenshot: prnt.sc/pquysj

  2. SNS Topics and subscriptions:
    Topic: us-west-2.console.aws.amazon.com/s...
    Confirmed Subscription: us-west-2.console.aws.amazon.com/s...

  3. Send email
    From: support@keenbrain.com To: bounce@simulator.amazonses.com
    FAIL: No bounce email notification received (at either support@keenbrain.com or kunal@keenbrain.com )
    FAIL: No SNS HTTP(S) POST notification received at af84a2ab.ngrok.io/email/sns

  4. Send email
    From: support@keenbrain.com To: complaint@simulator.amazonses.com
    PASS: Complaint email notification received (at support@keenbrain.com )
    FAIL: No SNS HTTP(S) POST notification received at af84a2ab.ngrok.io/email/sns

  5. Send email
    From: support@keenbrain.com To: success@simulator.amazonses.com
    FAIL: No "delivered" email notification received (at support@keenbrain.com or kunal@keenbrain.com )
    FAIL: No SNS HTTP(S) POST notification received at af84a2ab.ngrok.io/email/sns

So, I'm trying to get all the failed scenarios to pass, but especially the ones involving the SNS HTTP(S) POST notifications.

Thread Thread
andrewbrown profile image
Andrew Brown 🇨🇦 Author

Thank you for preparing that in good detail.

✔️So you have the SNS topics set correctly in SES
✔️So my next question was did you confirm your subscription which you did.
? The next thing I wasn't sure is what kind of subscription but since you are using ngrok and you mention HTTP(S) I am guessing you are trying to create an endpoint so that the SNS notification goes back into your laravel app so you can maybe store the bounces in your application's database.

So the question is where is the problem?
Is it SES, SNS or your Application?

So we need rule each one out, so now I ask if you have logging turned on about delivery status on the notification.

When this is turned on it will deliver that information to CloudWatch logs. If there are logs in there than we know it's your application, if you don't see anything in there then we know the problem is with SES.

If you already have this turned on its going to a bit tricky to debug and we'll have to think outside the box.

Thread Thread
kp profile image
KP • Edited on

@andrewbrown AWS has to be the most complex, non-user-friendly interface I have seen, thank you for your help. I truly appreciate it :)

You understood my situation perfectly...I am trying to get the SNS notification to my Laravel endpoint and log in to my database, or a file...
where end to end HTTP endpoint connectivity is established. Are you also using Laravel (or this package github.com/jdavidbakr/mail-tracker, by chance?)

Coming back to the problem...I did not have logging enabled previously, so I went ahead and did that (your screenshots really helped)

Went to the SNS console, clicked on the TOpic
Selected "Delivery Status Logging". Screenshot: res.cloudinary.com/practicaldev/im...
Clicked "Edit" and turn on Delivery Status Logging with HTTP(S): Screenshot: res.cloudinary.com/practicaldev/im...
Screenshot: prnt.sc/prdk3v

I then enabled Cloudwatch Logs for SMS messages:
docs.aws.amazon.com/sns/latest/dg/...

Also, before kicking off a test again, since my ngrok process (tunnel to my localhost so I can test SNS pings to my Laravel app) was dead, I restarted ngrok from the command line:

    ngrok http 8001

Btw, you can find my tentative ngrok tunnel (to my localhost:8001) at 1133c091.ngrok.io

Thread Thread
kp profile image
KP

I got the new subscription confirmed
(was missing the full URL (the /email/sns part): 1133c091.ngrok.io/email/sns

Screenshot: prnt.sc/prdro5

and I now have logging enabled per your instructions...

Thread Thread
kp profile image
KP • Edited on

@andrewbrown

Another update: I found documentation pointing me to cloudwatch, so I see where the logs will flow into.

This is what I did next:
From: support@keenbrain.com To: complaint@simulator.amazonses.com
PASS: Complaint email notification received (at support@keenbrain.com )

This is what the cloudwatch logs show:
Screenshots:
prnt.sc/prdufv
prnt.sc/prdukf

{
    "notification": {
        "messageMD5Sum": "8f10e089c532ef6f078e9860cd028d24",
        "messageId": "70c867fc-4d68-5dc4-adae-31cabe42a76f",
        "topicArn": "arn:aws:sns:us-west-2:753513409872:KeenBrain_Amazon_SNS_for_SES_Emails_Topic",
        "timestamp": "2019-11-02 02:58:50.382"
    },
    "delivery": {
        "deliveryId": "88181b7b-6a95-5980-8fa6-4d4d990f3bab",
        "destination": "https://1133c091.ngrok.io/email/sns",
        "providerResponse": "Internal Server Error",
        "dwellTimeMs": 112754,
        "attempts": 3,
        "statusCode": 500
    },
    "status": "FAILURE"
}

Does this mean there's some kind of internal server error in my application?
This is weird because I'm using an out-of-the-box package: github.com/jdavidbakr/mail-tracker

which supposedly is working for other people.

As you can see, 1133c091.ngrok.io/email/sns is a POST endpoint. Any idea how I can debug further?

Thread Thread
andrewbrown profile image
Andrew Brown 🇨🇦 Author

You are right on both.

SNS was sending the logs to CloudWatch and you found them.

The error being reported is from your Laravel app. So it's not SNS or SES and it's not ngrok.
It's your Laravel application.

I rarely use Laravel buts its very similar to Ruby on Rails.
You could be getting an error for multiple reasons in your application:

  • Expects authentication for an endpoint
  • Routes are incorrectly set
  • The error is occurring within your controller action due to misconfiguration

The problem doesn't appear to be with mail-tracker. I think the bounced email is working, an SNS notification is being triggered, but when SNS delivers the response to your endpoint its fails there.

Thread Thread
kp profile image
KP • Edited on

@andrewbrown

Thank you for the response....getting closer, thanks to you.

So, it's an issue with the application / package combination. Are you able to access 1133c091.ngrok.io (you should see "Email Sent" and 1133c091.ngrok.io/email/sns (you should see an error like "The GET method is not supported for this route. Supported methods: POST.")

The Laravel application is simply using the mail-tracker package (I've followed all the install and config instructions in the readme to a tee. I'm not saying that you should read that (you've been a great help)...but couldn't it potentially be a problem with the package itself? The SNS notification receipt /endpoint (which the package is responsible for and provides) is what is erroring out at 1133c091.ngrok.io/email/sns

By the way, I get the same error in cloudwatch logs if I send email to :
success@simulator.amazonses.com
or bounce@simulator.amazonses.com

However, I get no email receipts to my sending email, even though I have SES email forwarding enabled:
prnt.sc/pre15a
Is this something you've seen? I only seem to be getting emails of complaints, but not of successful deliveries or bounces.