Amazon SES

Anymail integrates with the Amazon Simple Email Service (SES) using the Boto 3 AWS SDK for Python, and supports sending, tracking, and inbound receiving capabilities.

Changed in version 11.0: Anymail supports only the newer Amazon SES v2 API. (Anymail 10.x supported both SES v1 and v2, and used v2 by default. Anymail 9.x and earlier used SES v1.) See Migrating to the SES v2 API below if you are upgrading from an earlier Anymail version.

Installation

You must ensure the boto3 package is installed to use Anymail’s Amazon SES backend. Either include the amazon-ses option when you install Anymail:

$ pip install "django-anymail[amazon-ses]"

or separately run pip install boto3.

Changed in version 10.0: In earlier releases, the “extra name” could use an underscore (django-anymail[amazon_ses]). That now causes pip to warn that “django-anymail does not provide the extra ‘amazon_ses’,” and may result in a broken installation that is missing boto3.

To send mail with Anymail’s Amazon SES backend, set:

EMAIL_BACKEND = "anymail.backends.amazon_ses.EmailBackend"

in your settings.py.

In addition, you must make sure boto3 is configured with AWS credentials having the necessary IAM permissions. There are several ways to do this; see Credentials in the Boto docs for options. Usually, an IAM role for EC2 instances, standard Boto environment variables, or a shared AWS credentials file will be appropriate. For more complex cases, use Anymail’s AMAZON_SES_CLIENT_PARAMS setting to customize the Boto session.

Limitations and quirks

Changed in version 11.0: Anymail’s merge_metadata is now supported.

Hard throttling: Like most ESPs, Amazon SES throttles sending for new customers. But unlike most ESPs, SES does not queue and slowly release throttled messages. Instead, it hard-fails the send API call. A strategy for retrying errors is required with any ESP; you’re likely to run into it right away with Amazon SES.
Tags limitations: Amazon SES’s handling for tags is a bit different from other ESPs. Anymail tries to provide a useful, portable default behavior for its tags feature. See Tags and metadata below for more information and additional options.
Open and click tracking overrides: Anymail’s track_opens and track_clicks are not supported. Although Amazon SES does support open and click tracking, it doesn’t offer a simple mechanism to override the settings for individual messages. If you need this feature, provide a custom ConfigurationSetName in Anymail’s esp_extra.
No delayed sending: Amazon SES does not support send_at.
Merge features require template_id: Anymail’s merge_headers, merge_metadata, merge_data, and merge_global_data are only supported when sending templated messages (using Anymail’s template_id).
No global send defaults for non-Anymail options: With the Amazon SES backend, Anymail’s global send defaults are only supported for Anymail’s added message options (like metadata and esp_extra), not for standard EmailMessage attributes like bcc or from_email.
Arbitrary alternative parts allowed: Amazon SES is one of the few ESPs that does support sending arbitrary alternative message parts (beyond just a single text/plain and text/html part).
AMP for Email: Amazon SES supports sending AMPHTML email content. To include it, use message.attach_alternative("...AMPHTML content...", "text/x-amp-html") (and be sure to also include regular HTML and text bodies, too).
Envelope-sender is forwarded: Anymail’s envelope_sender becomes Amazon SES’s FeedbackForwardingEmailAddress. That address will receive bounce and other delivery notifications, but will not appear in the message sent to the recipient. SES always generates its own anonymized envelope sender (mailfrom) for each outgoing message, and then forwards that address to your envelope-sender. See Email feedback forwarding destination in the SES docs.
Spoofed To header allowed: Amazon SES is one of the few ESPs that supports spoofing the To header (see Additional headers). (But be aware that most ISPs consider this a strong spam signal, and using it will likely prevent delivery of your email.)
Template limitations: Messages sent with templates have some additional limitations, such as not supporting attachments. See Batch sending/merge and ESP templates below.

Tags and metadata

Amazon SES provides two mechanisms for associating additional data with sent messages, which Anymail uses to implement its tags and metadata features:

SES Message Tags can be used for filtering or segmenting CloudWatch metrics and dashboards, and are available to Kinesis Firehose streams. (See “How do message tags work?” in the Amazon blog post Introducing Sending Metrics.)

By default, Anymail does not use SES Message Tags. They have strict limitations on characters allowed, and are not consistently available to tracking webhooks. (They may be included in SES Event Publishing but not SES Notifications.)
Custom Email Headers are available to all SNS notifications (webhooks), but not to CloudWatch or Kinesis.

These are ordinary extension headers included in the sent message (and visible to recipients who view the full headers). There are no restrictions on characters allowed.

By default, Anymail uses only custom email headers. A message’s metadata is sent JSON-encoded in a custom X-Metadata header, and a message’s tags are sent in custom X-Tag headers. Both are available in Anymail’s tracking webhooks.

Because Anymail tags are often used for segmenting reports, Anymail has an option to easily send an Anymail tag as an SES Message Tag that can be used in CloudWatch. Set the Anymail setting AMAZON_SES_MESSAGE_TAG_NAME to the name of an SES Message Tag whose value will be the single Anymail tag on the message. For example, with this setting:

ANYMAIL = {
    ...
    "AMAZON_SES_MESSAGE_TAG_NAME": "Type",
}

this send will appear in CloudWatch with the SES Message Tag "Type": "Marketing":

message = EmailMessage(...)
message.tags = ["Marketing"]
message.send()

Anymail’s AMAZON_SES_MESSAGE_TAG_NAME setting is disabled by default. If you use it, then only a single tag is supported, and both the tag and the name must be limited to alphanumeric, hyphen, and underscore characters.

For more complex use cases, set the SES EmailTags parameter (or DefaultEmailTags for template sends) directly in Anymail’s esp_extra. See the example below.

esp_extra support

To use Amazon SES features not directly supported by Anymail, you can set a message’s esp_extra to a dict that will be shallow-merged into the params for the SendEmail or SendBulkEmail SES v2 API call.

Examples (for a non-template send):

message.esp_extra = {
    # Override AMAZON_SES_CONFIGURATION_SET_NAME for this message:
    'ConfigurationSetName': 'NoOpenOrClickTrackingConfigSet',
    # Authorize a custom sender:
    'FromEmailAddressIdentityArn': 'arn:aws:ses:us-east-1:123456789012:identity/example.com',
    # Set SES Message Tags (change to 'DefaultEmailTags' for template sends):
    'EmailTags': [
        # (Names and values must be A-Z a-z 0-9 - and _ only)
        {'Name': 'UserID', 'Value': str(user_id)},
        {'Name': 'TestVariation', 'Value': 'Subject-Emoji-Trial-A'},
    ],
    # Set options for unsubscribe links:
    'ListManagementOptions': {
        'ContactListName': 'RegisteredUsers',
        'TopicName': 'DailyUpdates',
    },
}

(You can also set "esp_extra" in Anymail’s global send defaults to apply it to all messages.)

Batch sending/merge and ESP templates

Amazon SES offers ESP stored templates and batch sending with per-recipient merge data. See Amazon’s Sending personalized email guide for more information.

When you set a message’s template_id to the name of one of your SES templates, Anymail will use the SES v2 SendBulkEmail call to send template messages personalized with data from Anymail’s normalized merge_data, merge_global_data, merge_metadata, and merge_headers message attributes.

message = EmailMessage(
    from_email="[email protected]",
    # you must omit subject and body (or set to None) with Amazon SES templates
    to=["[email protected]", "Bob <[email protected]>"]
)
message.template_id = "MyTemplateName"  # Amazon SES TemplateName
message.merge_data = {
    '[email protected]': {'name': "Alice", 'order_no': "12345"},
    '[email protected]': {'name': "Bob", 'order_no': "54321"},
}
message.merge_global_data = {
    'ship_date': "May 15",
}

Amazon’s templated email APIs don’t support a few features available for regular email. When template_id is used:

Attachments and inline images are not supported
Alternative parts (including AMPHTML) are not supported
Overriding the template’s subject or body is not supported

Changed in version 11.0: Extra headers, metadata, merge_metadata, and tags are now fully supported when using template_id. (This requires boto3 v1.34.98 or later, which enables the ReplacementHeaders parameter for SendBulkEmail.)

Status tracking webhooks

Anymail can provide normalized status tracking notifications for messages sent through Amazon SES. SES offers two (confusingly) similar kinds of tracking, and Anymail supports both:

SES Notifications include delivered, bounced, and complained (spam) Anymail event_types. (Enabling these notifications may allow you to disable SES “email feedback forwarding.”)
SES Event Publishing also includes delivered, bounced and complained events, as well as sent, rejected, opened, clicked, and (template rendering) failed.

Both types of tracking events are delivered to Anymail’s webhook URL through Amazon Simple Notification Service (SNS) subscriptions.

Amazon’s naming here can be really confusing. We’ll try to be clear about “SES Notifications” vs. “SES Event Publishing” as the two different kinds of SES tracking events. And then distinguish all of that from “SNS”—the publish/subscribe service used to notify Anymail’s tracking webhooks about both kinds of SES tracking event.

To use Anymail’s status tracking webhooks with Amazon SES:

First, configure Anymail webhooks and deploy your Django project. (Deploying allows Anymail to confirm the SNS subscription for you in step 3.)

Then in Amazon’s Simple Notification Service console:

Create an SNS Topic to receive Amazon SES tracking events. The exact topic name is up to you; choose something meaningful like SES_Tracking_Events.
Subscribe Anymail’s tracking webhook to the SNS Topic you just created. In the SNS console, click into the topic from step 2, then click the “Create subscription” button. For protocol choose HTTPS. For endpoint enter:

https://random:random@yoursite.example.com/anymail/amazon_ses/tracking/
- random:random is an ANYMAIL_WEBHOOK_SECRET shared secret
- yoursite.example.com is your Django site
Anymail will automatically confirm the SNS subscription. (For other options, see Confirming SNS subscriptions below.)

Finally, switch to Amazon’s Simple Email Service console:

If you want to use SES Notifications: Follow Amazon’s guide to configure SES notifications through SNS, using the SNS Topic you created above. Choose any event types you want to receive. Be sure to choose “Include original headers” if you need access to Anymail’s metadata or tags in your webhook handlers.
If you want to use SES Event Publishing:
1. Follow Amazon’s guide to create an SES “Configuration Set”. Name it something meaningful, like TrackingConfigSet.
2. Follow Amazon’s guide to add an SNS event destination for SES event publishing, using the SNS Topic you created above. Choose any event types you want to receive.
3. Update your Anymail settings to send using this Configuration Set by default:
  ANYMAIL = { # ... other settings ... # Use the name from step 5a above: "AMAZON_SES_CONFIGURATION_SET_NAME": "TrackingConfigSet", }

Caution

The delivery, bounce, and complaint event types are available in both SES Notifications and SES Event Publishing. If you’re using both, don’t enable the same events in both places, or you’ll receive duplicate notifications with different event_ids.

Note that Amazon SES’s open and click tracking does not distinguish individual recipients. If you send a single message to multiple recipients, Anymail will call your tracking handler with the “opened” or “clicked” event for every original recipient of the message, including all to, cc and bcc addresses. (Amazon recommends avoiding multiple recipients with SES.)

In your tracking signal receiver, the normalized AnymailTrackingEvent’s esp_event will be set to the the parsed, top-level JSON event object from SES: either SES Notification contents or SES Event Publishing contents. (The two formats are nearly identical.) You can use this to obtain SES Message Tags (see Tags and metadata) from SES Event Publishing notifications:

from anymail.signals import tracking
from django.dispatch import receiver

@receiver(tracking)  # add weak=False if inside some other function/class
def handle_tracking(sender, event, esp_name, **kwargs):
    if esp_name == "Amazon SES":
        try:
            message_tags = {
                name: values[0]
                for name, values in event.esp_event["mail"]["tags"].items()}
        except KeyError:
            message_tags = None  # SES Notification (not Event Publishing) event
        print("Message %s to %s event %s: Message Tags %r" % (
              event.message_id, event.recipient, event.event_type, message_tags))

Anymail does not currently check SNS signature verification, because Amazon has not released a standard way to do that in Python. Instead, Anymail relies on your WEBHOOK_SECRET to verify SNS notifications are from an authorized source.

Note

Amazon SNS’s default policy for handling HTTPS notification failures is to retry three times, 20 seconds apart, and then drop the notification. That means if your webhook is ever offline for more than one minute, you may miss events.

For most uses, it probably makes sense to configure an SNS retry policy with more attempts over a longer period. E.g., 20 retries ranging from 5 seconds minimum to 600 seconds (5 minutes) maximum delay between attempts, with geometric backoff.

Also, SNS does not guarantee notifications will be delivered to HTTPS subscribers like Anymail webhooks. The longest SNS will ever keep retrying is one hour total. If you need retries longer than that, or guaranteed delivery, you may need to implement your own queuing mechanism with something like Celery or directly on Amazon Simple Queue Service (SQS).

Inbound webhook

You can receive email through Amazon SES with Anymail’s normalized inbound handling. See Receiving email with Amazon SES for background.

Configuring Anymail’s inbound webhook for Amazon SES is similar to installing the tracking webhook. You must use a different SNS Topic for inbound.

To use Anymail’s inbound webhook with Amazon SES:

First, if you haven’t already, configure Anymail webhooks and deploy your Django project. (Deploying allows Anymail to confirm the SNS subscription for you in step 3.)
Create an SNS Topic to receive Amazon SES inbound events. The exact topic name is up to you; choose something meaningful like SES_Inbound_Events. (If you are also using Anymail’s tracking events, this must be a different SNS Topic.)
Subscribe Anymail’s inbound webhook to the SNS Topic you just created. In the SNS console, click into the topic from step 2, then click the “Create subscription” button. For protocol choose HTTPS. For endpoint enter:

https://random:random@yoursite.example.com/anymail/amazon_ses/inbound/
- random:random is an ANYMAIL_WEBHOOK_SECRET shared secret
- yoursite.example.com is your Django site
Anymail will automatically confirm the SNS subscription. (For other options, see Confirming SNS subscriptions below.)
Next, follow Amazon’s guide to Setting up Amazon SES email receiving. There are several steps. Come back here when you get to “Action Options” in the last step, “Creating Receipt Rules.”
Anymail supports two SES receipt actions: S3 and SNS. (Both actually use SNS.) You can choose either one: the SNS action is easier to set up, but the S3 action allows you to receive larger messages and can be more robust. (You can change at any time, but don’t use both simultaneously.)
- For the SNS action: choose the SNS Topic you created in step 2. Anymail will handle either Base64 or UTF-8 encoding; use Base64 if you’re not sure.
- For the S3 action: choose or create any S3 bucket that Boto will be able to read. (See IAM permissions; don’t use a world-readable bucket!) “Object key prefix” is optional. Anymail does not currently support the “Encrypt message” option. Finally, choose the SNS Topic you created in step 2.

Amazon SES will likely deliver a test message to your Anymail inbound handler immediately after you complete the last step.

If you are using the S3 receipt action, note that Anymail does not delete the S3 object. You can delete it from your code after successful processing, or set up S3 bucket policies to automatically delete older messages. In your inbound handler, you can retrieve the S3 object key by prepending the “object key prefix” (if any) from your receipt rule to Anymail’s event.event_id.

Amazon SNS imposes a 15 second limit on all notifications. This includes time to download the message (if you are using the S3 receipt action) and any processing in your signal receiver. If the total takes longer, SNS will consider the notification failed and will make several repeat attempts. To avoid problems, it’s essential any lengthy operations are offloaded to a background task.

Amazon SNS’s default retry policy times out after one minute of failed notifications. If your webhook is ever unreachable for more than a minute, you may miss inbound mail. You’ll probably want to adjust your SNS topic settings to reduce the chances of that. See the note about retry policies in the tracking webhooks discussion above.

In your inbound signal receiver, the normalized AnymailTrackingEvent’s esp_event will be set to the the parsed, top-level JSON object described in SES Email Receiving contents.

Settings

Additional Anymail settings for use with Amazon SES:

AMAZON_SES_CLIENT_PARAMS

Optional. Additional client parameters Anymail should use to create the boto3 session client. Example:

ANYMAIL = {
    ...
    "AMAZON_SES_CLIENT_PARAMS": {
        # example: override normal Boto credentials specifically for Anymail
        "aws_access_key_id": os.getenv("AWS_ACCESS_KEY_FOR_ANYMAIL_SES"),
        "aws_secret_access_key": os.getenv("AWS_SECRET_KEY_FOR_ANYMAIL_SES"),
        "region_name": "us-west-2",
        # override other default options
        "config": {
            "connect_timeout": 30,
            "read_timeout": 30,
        }
    },
}

In most cases, it’s better to let Boto obtain its own credentials through one of its other mechanisms: an IAM role for EC2 instances, standard AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN environment variables, or a shared AWS credentials file.

AMAZON_SES_SESSION_PARAMS

Optional. Additional session parameters Anymail should use to create the boto3 Session. Example:

ANYMAIL = {
    ...
    "AMAZON_SES_SESSION_PARAMS": {
        "profile_name": "anymail-testing",
    },
}

AMAZON_SES_CONFIGURATION_SET_NAME

Optional. The name of an Amazon SES Configuration Set Anymail should use when sending messages. The default is to send without any Configuration Set. Note that a Configuration Set is required to receive SES Event Publishing tracking events. See Status tracking webhooks above.

You can override this for individual messages with esp_extra.

AMAZON_SES_MESSAGE_TAG_NAME

Optional, default None. The name of an Amazon SES “Message Tag” whose value is set from a message’s Anymail tags. See Tags and metadata above.

AMAZON_SES_AUTO_CONFIRM_SNS_SUBSCRIPTIONS

Optional boolean, default True. Set to False to prevent Anymail webhooks from automatically accepting Amazon SNS subscription confirmation requests. See Confirming SNS subscriptions above.

IAM permissions

Anymail requires IAM permissions that will allow it to use these actions:

To send mail:
- Ordinary (non-templated) sends: ses:SendEmail and ses:SendRawEmail
- Template/merge sends: ses:SendBulkEmail and ses:SendBulkTemplatedEmail
To automatically confirm webhook SNS subscriptions: sns:ConfirmSubscription
For status tracking webhooks: no special permissions
To receive inbound mail:
- With an “SNS action” receipt rule: no special permissions
- With an “S3 action” receipt rule: s3:GetObject on the S3 bucket and prefix used (or S3 Access Control List read access for inbound messages in that bucket)

This IAM policy covers all of those:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "ses:SendEmail",
      "ses:SendRawEmail",
      "ses:SendBulkEmail",
      "ses:SendBulkTemplatedEmail"
    ],
    "Resource": "*"
  }, {
    "Effect": "Allow",
    "Action": ["sns:ConfirmSubscription"],
    "Resource": ["arn:aws:sns:*:*:*"]
  }, {
    "Effect": "Allow",
    "Action": ["s3:GetObject"],
    "Resource": ["arn:aws:s3:::MY-PRIVATE-BUCKET-NAME/MY-INBOUND-PREFIX/*"]
  }]
}

Note

Confusing IAM error messages

Permissions errors for the SES v2 API refer to both the v2 API “operation” and the underlying action whose permission is being checked. This can be confusing. For example, this error (emphasis added):

An error occurred (AccessDeniedException) when calling the SendEmail operation:
User 'arn:...' is not authorized to perform 'ses:SendRawEmail' on resource 'arn:...'

actually indicates problems with IAM policies for the ses:SendRawEmail action, not the ses:SendEmail action. (Even though Anymail calls the SES v2 SendEmail API, not SendRawEmail.)

Following the principle of least privilege, you should omit permissions for any features you aren’t using, and you may want to add additional restrictions:

For Amazon SES sending, you can add conditions to restrict senders, recipients, times, or other properties. See Amazon’s Controlling access to Amazon SES guide. But be aware that:
- The v2 ses:SendBulkEmail action does not support condition keys that restrict email addresses, and using them can cause misleading error messages. To restrict template sends, apply condition keys to ses:SendBulkTemplatedEmail and then add a separate statement to allow ses:SendBulkEmail without conditions.
- The v2 ses:SendRawEmail and ses:SendEmail actions used for non-template sends do support conditions to restrict addresses.
- Technically, the v2 ses:SendEmail action does not seem to be required for the SES v2 SendEmail API operation as Anymail uses it (with the Content.Raw param), but including it seems prudent given Amazon’s confusing error messages and incomplete documentation on the subject.
For auto-confirming webhooks, you might limit the resource to SNS topics owned by your AWS account, and/or specific topic names or patterns. E.g., "arn:aws:sns:*:0000000000000000:SES_*_Events" (replacing the zeroes with your numeric AWS account id). See Amazon’s guide to Amazon SNS ARNs.
For inbound S3 delivery, there are multiple ways to control S3 access and data retention. See Amazon’s Managing access permissions to your Amazon S3 resources. (And obviously, you should never store incoming emails to a public bucket!)

Also, you may need to grant Amazon SES (but not Anymail) permission to write to your inbound bucket. See Amazon’s Giving permissions to Amazon SES for email receiving.
For all operations, you can limit source IP, allowable times, user agent, and more. (Requests from Anymail will include “django-anymail/version” along with Boto’s user-agent.) See Amazon’s guide to IAM condition context keys.

Migrating to the SES v2 API

Changed in version 10.0.

Anymail 10.0 and later use Amazon’s updated SES v2 API to send email. Earlier Anymail releases used the original Amazon SES API (v1) by default. Although the capabilities of the two SES versions are virtually identical, Amazon is implementing improvements (such as increased maximum message size) only in the v2 API.

(The upgrade for SES v2 affects only sending email. There are no changes required for status tracking webhooks or receiving inbound email.)

Migrating to SES v2 requires minimal code changes:

Update your IAM permissions to grant Anymail access to the SES v2 sending actions: ses:SendEmail and ses:SendRawEmail for ordinary sends, and/or ses:SendBulkEmail and ses:SendBulkTemplatedEmail to send using SES templates. (The IAM action prefix is just ses for both the v1 and v2 APIs.)

If you run into unexpected IAM authorization failures, see the note about misleading IAM permissions errors above.
If your code uses Anymail’s esp_extra to pass additional SES API parameters, or examines the raw esp_response after sending a message, you may need to update it for the v2 API. Many parameters have different names in the v2 API compared to the equivalent v1 calls, and the response formats are slightly different.

Among v1 parameters commonly used, ConfigurationSetName is unchanged in v2, but v1’s Tags and most *Arn parameters have been renamed in v2. See AWS’s docs for SES v1 SendRawEmail vs. v2 SendEmail, or if you are sending with SES templates, compare v1 SendBulkTemplatedEmail to v2 SendBulkEmail.

(If you do not use esp_extra or esp_response, you can safely ignore this.)
If your settings.py EMAIL_BACKEND setting refers to amazon_sesv1 or amazon_sesv2, change that to just amazon_ses:
```
EMAIL_BACKEND = "anymail.backends.amazon_ses.EmailBackend"
```