Tracking Messages

Table of Contents

Introduction

Tracking Opens

Tracking Clicks

Open and Click Bot Detection

Tracking Unsubscribes

Tracking Spam Complaints

Tracking Failures

Tracking Deliveries

Webhooks

Tags

Stats

Introduction to Tracking Messages

Mailgun provides comprehensive email tracking tools that offer insights through Events, Stats, and Tagging in order to help you optimize your email strategy by providing detailed insights into the performance and engagement of your emails. You can monitor various aspects of your emails, including:

• Tagging: Categorize and filter your emails for better management.
• Tracking Open Messages: See when recipients open your emails.
• Tracking Clicks: Monitor links clicked within your emails.
• Tracking Unsubscribes: Track when recipients opt-out from your mailing list.
• Tracking Spam Complaints: Identify when your emails are marked as spam.
• Tracking Failures: Understand when and why email deliveries fail.
• Tracking Deliveries: Confirm successful delivery of your emails.

Mailgun provides a variety of methods to access this data:

• To see every event that happened to every message, view and search Events through the Logs tab in the Control Panel. You can search by fields, like recipient, subject line, and even fields that don't use up in the Logs, such as message-id. Data is stored for at least 30 days for paid accounts, and at least 2 days for free accounts.
• Access data on Events programmability through the Events API. Data is stored for at least 30 days for paid accounts and at least 2 days for free accounts.
• View, search, and edit tables for Bounces, Unsubscribes, and Spam Complaints in the Suppressions table of the Control Panel of their retrospective APIs (Bounces API, Unsubscribes API, Complaints API). Data is stored indefinitely.
• Access statistics aggregated by tags in the Analytics tab of the Control Panel or the Stats API. Data is stored for at least six months.
• Receive notifications of events through he Webhook each time an Event happens and store the data on your side.

In addition to tracking messages, Mailgun permanently stores them when they cannot be delivered due to a hard bounce (permanent failure), or when a recipient unsubscribes or marks the message as spam. In these cases, Mailgun will not attempt to deliver messages to those recipients in the future. This is to protect your sending reputation.

Enable Tracking

Event tracking is automatically enabled, with the exception for Unsubscribes , Opens , and Clicks. You can enable these for your domain's via the Domains tab of the Control Panel.

Opens and Clicks tracking can be enabled on two levels – per sending domain (as mentioned above) and per message. Please see Open Tracking and Click Tracking for more details

There are two critical components to get tracking working properly

• You will need to point CNAME records to mailgun.org for Mailgun to rewrite links and track opens. These records can be found in your domains DNS records Tracking records section.
• There needs to be an HTML part of the message for Mailgun to track opens. (See Tracking Opens and Tracking Clicks for more details.)

Tracking Protocol (HTTP vs HTTPS)

By default, tracking URLs use the non-secure HTTP protocol, however Mailgun does support using the secure HTTPS protocol for tracking URLs. Mailgun utilizes Let’s Encrypt with HTTP-01 challenges via your existing tracking CNAME record to issue a TLS certificate. This configuration also supports HTTP Strict Transport Security (HSTS). Follow the instructions below to utilize HTTPS for your tracking URLs:

1. You must first ensure that you have a tracking CNAME in place pointing to mailgun.org (or eu.mailgun.org if your domain is in our EU infrastructure). By default the DNS record is email.domain.com, however, the tracking hostname is configurable, so just be sure the hostname in your CNAME record matches the tracking hostname set.

2. Once your CNAME record is in place, you’ll need to initiate the generation of an x509 TLS certificate , which will kick off a background task within our system to generate the certificate using Let’s Encrypt. The response body will include a status endpoint in the location field that you can poll to see when this process is complete.
3. Once you have verified that the certificate is generated, you will need to update the web_scheme from http to https .
4. After these steps are complete, send a test message to ensure your URLs are utilizing the HTTPS protocol and are working as intended.

Tracking Opens

Mailgun uses tracking pixels and URL redirects to track every time a recipient opens a message. These events can be viewed in the Control Panel in the Logs tab. You can also see counters of opens aggregated by tags by visiting the Analytics tab on the Control Panel. In addition, you can be notified through a webhook, or get the data programmatically through the Events API.

Open tracking can be enabled two ways

• on a per-domain basis: toggled in the Tracking Settings which can be found on your domain's settings page
• on a per-message basis using the parameters, o:tracking , or o:tracking-opens when sending an email. This will override the domain setting. See the API documentation for more details

By default, the open tracking pixel is added at the bottom of your email to mitigate possible impacts to email design. If you send long emails that experience truncation or other rendering issues at the recipient, you can ensure opens are being tracked accurately with placement of the tracking pixel at the top of your emails. This can be done on a domain level or a per-message level.

• on a per-domain basis: Place open tracking pixel at top of message can be toggled in the Tracking Settings which can be found on your domain's settings page (Same as where you enable open tracking)
• on a per-message basis using the parameters, o:X-Mailgun-Track-Pixel-Location-Top when sending an email. This will override the domain setting. See the API documentation for more details

As mentioned earlier on in this article, you will have to add the appropriate CNAME records to your DNS as specified in the Domain Verification & DNS section in order for this feature to work properly.

Tracking Clicks

Mailgun can track every time a recipient clicks on a link included in your email. When you enable Click tracking, links will be overwritten and point to Mailgun's servers, giving us the ability to track. These events can be viewed in the Control Panel in the Logs tab. You can also see counters of opens aggregated by tags by visiting the Analytics tab on the Control Panel. In addition, you can be notified through a webhook, or get the data programmatically through the Events API.

Click tracking can be enabled two ways

• on a per-domain basis: toggled in the Tracking Settings which can be found on your domain's settings page
• on a per-message basis using the parameters, o:tracking , or o:tracking-clicks when sending an email. This will override the domain setting. You can specify that you only want links rewritten in the HTML part of the message with the parameter o:tracking-clicks and passing htmlonly . See the API documentation for more details.

You can also disable click tracking for a specific link by including the HTML attribute disable-tracking=true in the HTML tag of the link. With this HTML attribute in the link's HTML tag, Mailgun will not rewrite the URL. Example : <a href="http://mailgun.com" disable-tracking=true\>Mailgun\</a\>

Open Webhook

You can specify webhook URLs programmatically using the Webhooks API. When a user opens an email that you have sent, your opened URLs will be called with the following webhooks payload.

Clicks Webhook

You can specify webhook URLs programmatically using the Webhooks API. Every time a user clicks on a link inside of your messages, your clicked URLs will be called with the following webhooks payload.

Open and Click Bot Detection

Mailgun uses tracking pixels and URL redirects to track when a user opens the message and clicks links in the email. However, there are various third-party automated systems that will automatically open and message and follow the links for virus scanning and user activity obfuscation, such as Apple Mail Privacy Protection.

Because automated systems can affect the accuracy of open and click tracking, Mailgun will attempt to detect when one of the systems retrieves the tracking pixel or clicks a link. When a bot is detected opening or clicking a link in the email, Mailgun will indicate this via the client-info.bot field in the open/click events.

{
    "client-info": {
      "client-name": "unknown",
      "client-type": "unknown",
      "user-agent": "Mozilla/5.0",
      "device-type": "unknown",
      "client-os": "unknown",
      "bot": "apple"
    },
    "tags": [],
    "timestamp": 1652883435.279025,
    "recipient": "bot@apple.com",
    "geolocation": {
      "region": "Unknown",
      "country": "US",
      "city": "Unknown"
    },
    "event": "opened",
}

The bot field can have one of the possible values:

Tracking Unsubscribes

Every time a recipient requests to unsubscribe from mailings, Mailgun can keep track of it. When you enable unsubscribe tracking, Mailgun will insert unsubscribe links and remove those recipients from your mailings automatically for you.

To see unsubscribes, go to the Logs tab, or see counters of unsubscribes aggregated by tags found on the Analytics tab of the Control Panel. You can also get notifications through a webhook, or get data programmatically through the Events or Bounces API

Mailgun supports three types of unsubscribe levels: domain, tag, Mailing Lists

Auto-Handling

Mailgun will automatically prevent future emails from being sent to recipients who have unsubscribed when you enable the Unsubscribe functionality by going to the settings in your domain area on the Control Panel. You can also edit the unsubscribed address list from the Control Panel or through the API.

Unsubscribe Variables

Mailgun provides you with several unsubscribe variables:

When these variables are included in your emails, any recipient who clicks on the URL will be automatically unsubscribed, and those email addresses will be blocked from receiving future emails from that domain or message tag.

Mailgun can automatically provide a customizable unsubscribe footer in each email you send. You can edit the unsubscribe footer by editing the settings in your control panel.

To enable/disable unsubscribes programmatically, you can do the following:

• Enable the Unsubscription feature for your domain.
• Remove text in the HTML and text footer so they won't be appended automatically.
• Insert a variable in the HTML and text bodies of your email when you need unsubscribe links.
  • This variable will be replaced by the corresponding unsubscribe link.

When you go to the Suppressions tab of the Control Panel or through the API, you can also:

• View/get a list of unsubscribed addresses
• Remove an unsubscribed address from the list
• Add a new unsubscribed address

Learn how to programmatically manage lists of unsubscribed users by visiting the Unsubscribes section of the API reference.

Unsubscribe Webhook

You can specify webhook URLs programmatically using Webhooks API. When a recipient unsubscribes, Mailgun will invoke the unsubscribed webhook with the following webhooks payload.

Tracking Spam Complaints

Email service providers (ESP) are very sensitive to users clicking on spam complaint buttons. It's important to monitor that activity to maintain a good sending reputation. While not all ESP supports Feedback Loop (FBL) notifications, Mailgun makes sure that you get data on all the ones that you do, will not remove recipients from future messages if the complaint has been filed by the recipient, Mailgun automatically tracks all messages recipients report as spam.

You can view spam complaints on the control panel by clicking on the logs tab or by clicking on the Analytics tab to see counters of complaints aggregated by tags. You can also be notified through a webhook. When a recipient report one of your emails as spam, Mailgun will invoke the complained webhook with the following payload. Another way to track complaints is to get the data programmatically through the Events API or the Complaints API. You can Mailgun provides the Spam Complaint API to programmatically manage lists of users who have complained.

Tracking Failures

Mailgun tracks all delivery failures, which consists of both hard bounces (permanent failures), and soft bounces (temporary failures).

When an email message is said to "bounce", this means that it was rejected by the recipient SMTP server. Bounced addresses are found on the "Bounces" table, which is found on the Suppressions tab on the Control Panel. With respect to failure persistence, Mailgun classifies bounces into two categories:

• Hard bounces (permanent failure): This based on these criteria:
  • The recipient is not found
  • The recipient email server specifies that the recipient does not exist
• Soft bounces (temporary failure):
  • Email is not delivered because the mailbox is full or for other reasons.

Permanent and Temporary Failure Webhooks There may be a few reasons why Mailgun needs to stop trying to deliver messages. The most common reason being that Mailgun has received a hard bounce or repeatedly received soft bounces. Continually attempting to deliver the message may affect the sender's reputation with the receiving ESP. When the address is on one of the 'Do Not Send" lists because the recipient has previously bounced, unsubscribed, or reported spam, Mailgun will drop the message and stop trying to send it. If one of these events occurs, Mailgun will POST the following webhooks payload to your permanent_fail URLs. You can specify webhook URLs programmatically using the Webhook API.

With respect to when the recipient SMTP server has rejected incoming messages, Mailgun will classify bounces into the following categories:

• Immediate bounce :
  • An email message is rejected by the recipient SMTP server during the SMTP session.
• Delayed (asynchronous) bounce:
  • The recipient SMTP server accepts an email message during the SMTP session. After some time, it will send a Non-Delivery Report email message to the message sender.

Tracking Deliveries

Mailgun tracks all successful deliveries that occur when the recipient email server responds that it has accepted the message.

You can see when a message has been successfully sent by clicking on the Logs tab found on the Control Panel. You can also be notified when a message has been delivered through a webhook or get the data programmatically through the Events API.

Do note that, for messages that get Routed to a HTTP endpoint, we do not send a Delivered webhook to your configured URL(s). We do emit the delivered event for the message itself so that the Routed delivery shows up in the Events API.

Webhooks

Webhooks allows you to programmatically handle events that happen with your messages sent through Mailgun. By configuring URL(s) in the Webhooks tab of the Control Panel, Mailgun can send HTTP/HTTPS POST requests to specified endpoints when certain events occur. Webhooks are configured at the domain level, enabling you to set unique endpoints for each domain via the domain drop-down selector.

Supported Webhook types:

• Accepted, Delivered, Clicks, Spam Complaints, Unsubscribes, Permanent Failures, Temporary Failures, Spam Complaints

| Note: If you want to include an HTTPS endpoint, it must be configured with a trusted CA (Certificate Authority) signed SSL certificate, not a self-signed certificate.

You can read more about the data that is posted in the appropriate section below (Tracking Opens, Tracking Clicks, Tracking Unsubscribes, Tracking Spam Complaints, Tracking Failures, Tracking Deliveries). We recommend using http://bin.mailgun.net/ for creating temporary URLs to test and debug your webhooks. | | --- |

For Webhook POSTs, Mailgun listens for the following codes from your server and reacts accordingly:

• If Mailgun receives a 200 (Success) code, it will determine the webhook POST is successful and not retried.
• If Mailgun receives a 406 (Not Acceptable) code, Mailgun will determine the POST is rejected and not retry.
• For any other code, Mailgun will retry POSTing according to the schedule below for Webhooks other than the delivery notification.

If your application is unable to process the webhook request, but you do not return a 406 error code, Mailgun will retry (other than for delivery notification) during 8 hours at the following intervals before stopping to try: 5 minutes, 10 minutes, 15 minutes, 1 hour, 2 hours, and 4 hours.

The Webhooks API endpoint allows you to programmatically manipulate the webhook URLs defines for a specific domain. See the Webhooks API for more details.

Payload

When something happens to your email, your URL will be called with application/JSON payload with the following data:

{
  “signature”:
  {
    "timestamp": "1529006854",
    "token": "a8ce0edb2dd8301dee6c2405235584e45aa91d1e9f979f3de0",
    "signature": "d2271d12299f6592d9d44cd9d250f0704e4674c30d79d07c47a66f95ce71cf55"
  }
  “event-data”:
  {
    "event": "opened",
    "timestamp": 1529006854.329574,
    "id": "DACSsAdVSeGpLid7TN03WA",
    // ...
  }
}

The 'signature' parameters are described in Securing Webhooks and the 'event-data' parameters are the same as described in Event Structure.

Securing Webhooks

To ensure the authenticity of event requests, Mailgun signs them and posts the signature alongside the webhook's event-data.

A signature takes the following form:

To verify the webhook originated from Mailgun, you will need to:

• Concatenate timestamp and token values together with no separator.
• Encode the resulting string with the HMAC algorithm, using your Webhook Signing Key as a key and SHA256 digest mode.
• Compare the resulting hexdigest to the signature. If they do not match then the webhook is not from Mailgun!
• Optionally, you can cache the token value locally and not honor any subsequent request with the same token. This will prevent replay attacks.
• Optionally, you can check if the timestamp is not too far from the current time.
  • There can be delays in webhook processing that are outside of Mailgun's control so you don't want to be too aggressive with this check.

To visualize, here's a sample NodeJS snippet that checks the Webhook signature:

const crypto = require('crypto')

const verify = ({ signingKey, timestamp, token, signature }) => {
    const encodedToken = crypto
        .createHmac('sha256', signingKey)
        .update(timestamp.concat(token))
        .digest('hex')

    return (encodedToken === signature)
}

Tags

Mailgun allows you to tag your email with unique identifiers. They can help segment your email into relevant categories for later analysis and optimization. Tags are visible via event data and will be included in webhook payloads.

Examples of when to use a tag:

• Identifying mail type, like “password reset” or “welcome”
• Identifying campaign or audience, like “Black Friday” or “new signup”

Tags are further enhanced when using Mailgun for open and click tracking. You can understand the full performance of your tag via Analytics in the Mailgun UI, including comparison to other tags.

Tagging Code Sample

curl -s --user 'api:YOUR_API_KEY' \
    https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages \
    -F from='Excited User <postmaster@YOUR_DOMAIN_NAME>' \
    -F to=recipient@example.com \
    -F subject="Hello there!" \
    -F text='Testing some Mailgun awesomeness!' \
    -F o:tag='September newsletter' \
    -F o:tag='newsletters'

Stats

You can view stats aggregated by tag within the Analytics portion of the Mailgun UI. Additionally the Stats API can provide this detail in your own application. You can also filter by tags in the Reporting Dashboard.