EMAIL HANDLING FOR SERVICE AND MAILINGS IN SUPEROFFICE ONLINE
The different options
Each CRM Online Customer will be handed their own email account for SuperOffice Service, in the form of "email@example.com.XX" (.eu or .com). More email accounts can be added in the SuperOffice Service admin pages. SuperOffice Service uses Mailgun for sending and receiving emails to those accounts, and SuperOffice Mailings uses Mailgun for sending emails.
Customers have several alternatives to handle emails in SuperOffice Online with these services. The options are listed below. The default service for new customers is suocrm.eu, some of the existing customers use suocrm.com.
ALTERNATIVE 1: CRM Online email service only
Use the email address generated from SuperOffice Online and use this to directly communicate with your customers. Example: "firstname.lastname@example.org"
Mail sent from SuperOffice Service / SuperOffice Mailings to your customer:
Click here to see a large illustration.
Mail sent from your customer to your SuperOffice Service:
Click here to see a large illustration.
- We handle all your shared email accounts (support@.., sales@.., …)
- We handle the volumes of sending your emails
- We handle the email reputation management
- Easy to get started, ready to use
- Separate domain name for emails from these 'shared' accounts (email@example.com vs. firstname.lastname@example.org)
- Other users of Mailgun (incl. CRM Online customer) can spoof this email address
ALTERNATIVE 2: CRM Online email service with own domain name and 'mail forward'
Use this alternative if you want to keep your old email addresses (your company domain address)
When creating a new mailbox in Service you automatically receive a "Forwarding address"
Important! You will have to set up forwarding from address 1 >> address 2 on your local email account
When using Mailgun as the sending email service, and sending it as someone else (your own domain), it is recommended to set up both an SPF record and a DKIM record as a best practice, to avoid outgoing emails from Service to your customers ending up as spam. Read more on SPF and DKIM in section "Security and best practices" below.
Out (no SPF/DKIM):
Mail sent from SuperOffice Service / SuperOffice Mailings to your customer, using your own domain as sender address:
Click here to see a large illustration.
Out (with SPF):
Mail sent from SuperOffice Service / SuperOffice Mailings to your customer, using your own domain as the sender address, and have set up an SPF record:
Click here to see a large illustration.
Out (with DKIM):
Mail sent from SuperOffice Service / SuperOffice Mailings to your customer, using your own domain as the sender address, and have set up a DKIM record:
Click here to see a large illustration.
Mail sent from your customer to your SuperOffice Service:
- Other users of Mailgun (incl. CRM Online customer) can spoof this email address, even with SPF
- See details in section "Security and best practices" below.
- Require to administer and setup of mail forwarding on your local mail account for incoming emails
- Require DNS entry for SPF (cumbersome)
- Requires registration, (administration) and DNS entry for DKIM (cumbersome)
- Not setting up SPF and DKIM correctly will affect the service reputation
- Some do not want to have a mail forwarding service (?)
ALTERNATIVE 3: Customizing our Mail Service - Own SMTP email service
Use this alternative if you want to use your own email service (your company-maintained email service) for sending emails from SuperOffice Service and Marketing.
You continue to use our Mailgun service (Option 1 or 2 above) for incoming emails.
Out - for both Service and Marketing:
Use your own email server for all outgoing emails. Mail sent from SuperOffice Service / SuperOffice Mailings to your customers, using your own email server.
Out - for Marketing only:
Use your own email server for outgoing SuperOffice Mailings to your customers. You continue to use our Mailgun service (Alternative 1 or 2 above) for mail sent from SuperOffice Service.
Out - for Service only:
Use your own email server for outgoing SuperOffice Service to your customers. You continue to use our Mailgun service (Alternative 1 or 2 above) for mail sent from SuperOffice Mailings.
Feedback from customers that leads to the need for customization is related mainly to these issues:
- Some customers question the fact that data (emails) are stored temporarily in the US.
Most of our customers are located in europe, and we use mailservers located in Europe when sending from the domain smtp.eu.mailgun.org.
The default Mailgun service for customers is localted in EU.
Note: If you checked the mails that are sent and noticed the ip address in the header, ie. 22.214.171.124.
Using https://www.iplocation.net/ip-lookup to lookup the IP it reports it to be located in US. They expected it to be located in Germany, what is the reason for this?
Our EU infrastructure is located at the AWS datacenter in Frankfurt, Germany (including the dedicated IPs). However, AWS advertises all their IPs from the same two ASNs. BYOIP process does all the registration of those IPs in the US, but the IPs themselves are advertised on border devices in Frankfurt. As a result, many lookups will only show this US registration data, leading to the false assumption that the IPs themselves are located in the EU. So, while the DNS may show the registration within the US, the actual turrets are housed within the EU.
Customers who still use Mailgun service in the US:
For a few older customers who already are set up to use Mailgun service in the US, and has not been moved automatically due to their technical setup, can change to EU by contacting support.
Data is processed and stored outside EU and even if Mailgun Inc is Privacy Shield compliant and have signed a DPA with SuperOffice, and they object on a principal basis.
In 2020 the European Court of Justice made a ruling that says the EU-US Privacy Shield (the agreement between the EU and the US that should ensure safe transfer of personal data from the EU to the US), is now illegal. This is now known as the Schrems II ruling.
The SuperOffice CRM mailing feature is using an American cloud service provider; Mailgun Inc. for distributing mass-emails and for email-correspondence related to customer service tickets. This service has been protected by the Privacy Shield agreement which no longer is valid (after 2020).
Mailgun Inc. however also provides a Datacenter located in EU and we have move all our customers to this location. This was done as automatic as possible and without any cost on your part. By doing this, the SuperOffice CRM Online solution is fully GDPR compliant and satisfies all requirements for privacy data protection set out by this regulation.
If you need more information about the Schrems II ruling, we suggest you search your national Data Protection authority website for additional information.
For those customers who have set up a DKIM towards Mailguns datacenter located in US, to be GDPR complient you need to set up a new DKIM towards the datacenter located in EU as soon as possible.
- Some customers are not happy with some security risk involved in using this kind of mailing service (SPF, DKIM).
Customers do not like this because:
SPF does not provide 100% security for the authenticity of outgoing emails. It does not prevent another crooked Mailgun customer to manipulate an email-senders address. The way mailing services (in general, not only Mailgun) is designed to perform and provide scalability and economies of scale, makes it currently not possible to close this vulnerability. It might not be a big issue, but some of our professional customers are aware of this and do not accept the risk. So we have to find another solution (more about this below).
What can we do to mitigate this? See a) & b) below.
Let’s look at the possible ways we can change the setup to meet the arguments.
a) Outgoing emails created by the mailings module in SuperOffice can be sent to another email-server than Mailgun.
In our standard configuration, the mail is sent to Mailgun using an URL registered as a setting in CRM Online Operation Center. This URL can manually be changed (by Operations) and route all outgoing email to another email-service (i.e. customers standard or another mass mailing service). See below for more information on how to proceed.
This change will affect where the emails are processed and stored (1) and …
… if the new email-service in controlled by the customer, it removes the SPF challenge (2).
b) The best solution for using email together with SuperOffice Online is to use the email addresses we supply as part of the mailing-service. In our product, the customer can create addresses such as "email@example.com", "firstname.lastname@example.org", etc. If the customer accepts that replies to customer inquiries or mailings originate from these addresses, then there is no need to configure SPF and little risk of being caught by spam filters. You can still publish your own email@example.com address, but set it up to be forwarded to the address we supply. The end customer will experience that the reply is originating from another address than the one they sent to, but this should be of little concern.
See the option "ALTERNATIVE 1: CRM Online email service only" above
Using any mailing service always represent some kind of trade-off. Changing to a private email-server does not necessary makes things perfect …
So, please be aware:
Why do we use a mass mailing service like Mailgun? What features and security does it provide? What benefits will disappear if a customer starts using his normal mail-service for mass mailing? It is important to take these qualities into considerations when discussing the issues with a customer. This is not a black and white situation with a silver bullet solution that solves everything. Both solutions have pro’s and con’s – both related to cost, capacity, features and security.
Let’s remind ourselves and our customers why the Mailgun – service does a good job:
- Open to the internet;
The customer do have to make their mail-server available for SuperOffice Online. This will imply making the server open to the internet. This will inevitably represent a security risk.
Sending out large amounts of email from a company email server may lead to blacklisting and/or poor reputation. This again may increase the risk of other emails from the company caught as spam.
- Mass mailing capacity and cost;
A company mail server may have limitations related to no. of emails outgoing (typically mass emails). In addition, it might affect performance, stability, administration and both direct and operational cost.
- Support and maintenance;
Mailgun is an integrated part of the SuperOffice CRM Online concept. In terms of operation, upgrades, support, and knowledge. It is 100% managed by SuperOffice. Customers taking responsibility for running their own mail-server will represent a custom integrated solution that will have an effect on the mentioned areas.
- Do not require a separate DNS entry for SPF
- Do not require a separate DNS entry for DKIM
- registration, (administration)
- Full control of email domain and emails going out
- You handle all your shared email accounts (locally), setup and configuration settings will (for now) be handled by Online Operations.
- You handle the volumes of sending your emails
- You handle the blacklisting of your IP address (email reputation management)
- Do require you to administer and setup of mail forwarding to Mailgun on your local mail account for incoming emails (bounce etc.) and/or maintenance of "reply to" / "from" in your outgoing emails need to match your setup of how you plan to get your customers response emails back into SuperOffice Service via Mailgun.
How to configure your CRM Online site to use your own SMTP (outgoing) mail server?
Get in contact with our support and they will help you to get it configured.
Access to your SMTP server from Online servers
It is a good idea to allow (whitelist) access to your own SMTP server from our Online environment based on IP address: 126.96.36.199, 188.8.131.52, 184.108.40.206
Mailgun sub-processor information (part of our DPA). Revised in January 2018.
Entity Company Name
Mailgun Technologies Inc. 535 Mission St. San Francisco, CA94105
Entity Type and description of Service
Email service provider. Mailgun is 1) sending mass emails generated from SuperOffice CRM and 2) receiving and sending replies related to service-tickets in SuperOffice Service. Emails are stored by Mailgun for max. 72 hours for resending purposes.
Individual emails sent from the customer’s own email service (i.e. exchange, Gmail) is not sent to Mailgun.
The Personal Data to be Processed concerns the following categories of Data Subjects:
Email recipients in e-marketing campaigns and customer service tickets.
Categories of Personal Data
Email address and free text content of emails.
Sensitive Personal Data (if relevant)
Personal Data will be subject to the following Processing activities.
Security and best practices
Spam mail messages have been a plague since the Internet became popular and they kept growing more and more as the number of devices and people connected grew. Despite the numerous attempts of creation of anti-spam tools, there is still a high number of unwanted messages sent every day.
Luckily, lately, it seems that something is changing with the adoption of three (relatively) new tools, which are starting to be widely used: SPF, DKIM, and DMARC.
Prevent address forgery (spoofing) spammers can forge, or "spoof," your domain's 'From address' to make their spam look like it came from someone in your domain. To help prevent this, we recommend authenticating mail sent from your domain in two ways: by creating SPF records, and by adding a digital signature to your messages that conform to the Domain Keys Identified Mail (DKIM) standard.
SPF (Sender Policy Framework) is a DNS text entry, which shows a list of servers that should be considered, allowed to send mail for a specific domain. Incidentally, the fact that SPF is a DNS entry can also be considered as a way to enforce the fact that the list is authoritative for the domain since the owners/administrators are the only people allowed to add/change that main domain zone.
Today, nearly all abusive e-mail messages carry fake sender addresses. Spammers send email from their mail servers but with your ‘domain’ as the sending email. The victims whose addresses are being abused often suffer from the consequences, because their reputation gets diminished and they have to disclaim liability for the abuse or waste their time sorting out misdirected bounce messages.
The purpose of an SPF record is to prevent spammers from sending messages with forged From addresses at your domain. Recipients can refer to the SPF record to determine whether a message purporting to be from your domain comes from an authorized mail server. If your domain does not have an SPF record, some recipient domains may reject messages from your users because they cannot validate that the messages come from an authorized mail server.
You should only have one SPF record for your domain. If you use more than one outgoing email provider, you need to combine their include directives together
The important thing to remember is that SPF records are just a whitelist and/or blacklist of IPs that can or can't send on behalf of your domain.
Read more about SPF in How to set up SPF?
DKIM (Domain Keys Identified Mail) should be considered a method to verify that the messages' content is trustworthy, meaning that they were not changed from the moment the message left the initial mail server. This additional layer of trustability is achieved by an implementation of the standard public/private key signing process. The owners of the domain add a DNS entry with the public DKIM key, which will be used by receivers to verify that the message DKIM signature is correct, while on the sender side the server will sign the entitled mail messages with the corresponding private key. Receiving email servers look up your public key and verify that nothing has changed in the email.
Not all receiving mail servers support the DKIM standard.
Read more about DKIM in How to order a DKIM key from SuperOffice, and how to set up DKIM?
Domain-based Message Authentication, Reporting & Conformance is an email authentication protocol. it builds on the widely deployed SPF and DKIM protocols, adding a reporting function that allows senders and receivers to improve and monitor the protection of the domain from fraudulent email. DMARC acts as a policy statement that declares what to do with emails that fail on SPF, DKIM, or both.
There are a few different modes that you can use with DMARC, but the most basic one is to receive reports from receiving email servers on 'pass' or 'fail' status, and receivers are given a simple way to check the legitimacy of the email.
Monitoring on DMARC:
Mailgun is not able to act upon the DMARC reports out of the box (?) but the reports can be used with a 3.party DMARC aggregation service:
Postmark runs a free DMARC aggregation service, which will aggregate all of the reports from DMARC-supporting services and send you a report every Monday morning with details. The first step in implementing DMARC is to sign up with Postmark's service, set up the DMARC record that they give you in your DNS, and wait a week. You'll probably have to go through this cycle every week for at least a few weeks, in order to catch all of the services that send email as you
Upon reception, the receiving mail server checks if there is any existing DMARC policy published in the domain used by the SPF and/or DKIM checks. If one or both the SPF and DKIM checks succeed while still being aligned with the policy set by DMARC, then the check is considered successful, otherwise, it's set as failed. If the check fails, based on the action published by the DMARC policy, different actions are taken:
Read more about DMARC
- Fix Your Email Deliverability with DMARC
- DMARC for Email Service Providers
Sender Reputation is used by email providers and filters to determine whether to accept or reject email by knowing whether a sender is a good sender or a spammer. Sender reputation is based on factors like email volume, complaint rates, user unknown rates, blacklistings, filtering rates, spam trap hits, and bounce rates.
Commonly called Realtime blacklist, DNSBL or RBL. An email blacklist is a real-time database that uses set criteria to determine if an IP is sending emails that could be considered spam. There are over 300 publicly available blacklists. Public blacklists are created by large, trustworthy companies, as well as small, independent networks. Since anyone can create a blacklist, they don’t all have the same impact on deliverability. Mailbox providers and filtering companies do not leverage inbox placement on every blacklist. They typically combine data from various public blacklists, as well as data from their own networks, to determine your credibility as a sender. This allows other mail servers to check if an email is from a server's IP address that might have possibly been flagged for sending spam in the past.
It’s important to note that blacklist providers are not the ones blocking your mail—it’s the mailbox provider leveraging your blacklist status that blocks your mail. If a blacklist resulted in a block, focus on the potential causes for the listing. Blacklisting is most often caused by poor list quality and end-user complaints, but they all have their own criteria for accepting inbound mail and all can have a negative effect on your delivery rate.
There are two types of blacklists: IP address-based and domain-based.
Mailgun will try their best to react to blacklisting by requesting removal immediately from any publicly available blacklist. Any non-publicly available blacklist is not easy to monitor, and must, therefore, be handled case by case.
In order not to face the blacklisting problem we recommend to follow suggestions in the articles below:
Read more about blacklisting at sendgrid.com
A 'Blocklist' is a list the mail server owners have. A blocklist is a custom database of email addresses and domains from which server owners never want to receive emails from. To create this list, they may use their own blacklist (non-public), a public blacklist, and/or several blacklists as a source. It’s important to note that blacklist providers are not the ones blocking your mail—it’s the mailbox provider leveraging your blacklist status that blocks your mail. All blocking of mail is done on some server somewhere, and the blacklist doesn't control those servers. It is mail server owners blocking your email after all, not the blacklist. (even though many uses the term 'blacklist' and 'blocklist' interchangeably)
Large freemail providers such as Gmail, Hotmail, and Yahoo maintain their own blocklists, and the process for getting delisted is not always clearly stated.
The distinction between spam and nonspam is to a large extent a matter of personal preference. For example, a bulk email message containing product offers may be actively wanted by some recipients, while others regard it as spam.
Typically this depends on whether the recipient has actively requested to be added to a distribution list or not, but the email server has no access to information about opt-ins and so cannot use it to determine if an email is spam.
Because there is no hard definition of "spam" it is also relatively easy to end up on a blocklist by mistake. A classic example is an email user who subscribes to a mailing list, forgets about it, and then hits the "spam" button instead of unsubscribing from the list. It is also easy to become listed if dumb autoresponders send out-of-office replies in response to spam messages with forged "from" addresses.
In many cases, this can be caused by the fact that the recipient’s mail server sees your email address as spam, because the Sender domain is different from the actual domain being received from.
Creating a mailing in SuperOffice Online, you define a sender address ('from:'). You can use any email address for this, as there are no restrictions. Let's say you are using a sender email address which has the domain (email ending)“@yourdomain.com”. When the mailing reaches the recipient mail server it sees that the mailing is sent from “customer.suocrm.com” domain. This can cause your newsletter being identified as Spam.
Read more about spam at Wikipedia
Spam traps are email addresses that may or may not exist and are used to judge your sender reputation. The term “trap” refers to how these types of addresses are scattered throughout the internet to catch people either not using proper list building practices, harvesting emails, purchasing lists from a third party, or marketers who have poor list hygiene (whether they know it or not). “Honeypot” or “Planted” Traps email addresses have been intentionally created to trap spammers searching the millions of websites on the Internet for any address they can find. These traps are never published and do not belong to a real person thus could never “opt-in” to any list since it is impossible for the address to initiate, respond or give consent to having received an email of any kind. They are used by anti-spam groups to catch spammers, monitor and collect spam. If you send an email to one of these traps, you will get exposed for using illegal marketing practices and you will get blacklisted which will seriously harm both your delivery and your reputation.
Read more about spam traps at Wikipedia
Spam complaints are reports made by email recipients against emails they consider to be unsolicited. Most ISPs offer to report facilities that allow their customers to mark emails they do not remember opting in for, or did not opt-in for, as spam. When an email recipient clicks this button, ISP lets the senders' email server (in our scenario, Mailgun) know that one of our customers is potentially sending spam. This kind of tool enables email recipients to report anything they consider to be unsolicited "junk mail". Through a direct feedback loop with major ISPs, as well as email providers like Gmail and Outlook.com (Hotmail), Mailgun can automatically detect if a significant number of a customer's recipients flag the customer's email campaign as spam. Some spam complaints may be false. Perhaps the recipient was lazy or just forgot about giving their permission in the first place.
If one of the customer's email campaigns gets more than a typical number of complaints, which is calculated as a percentage of the sending size, could result in their Mailgun account being suspended. You can minimize the chances of your campaigns being reported as spam by following best practice email marketing guidelines.
Most of the spam complaints coming from Gmail and Outlook.com (Hotmail) because they are both the world's biggest email providers, and also the main providers using feedback loops.
If an email service or ISP does not have a feedback loop set up, then any complaints by your subscribers are not passed back to Mailgun. ISPs still consider every spam complaint as an official complaint from their customers. This means that if enough recipients mark a campaign as spam, the ISP will punish the offending sender by adding them to their blacklist or negatively adjusting their sender reputation score.
Mail classifications depend heavily on reports from users. Most users of the mail hosters, can mark and unmark messages as spam, and can move non-spam messages between inbox tabs. In both cases, mail vendors learn from user corrections and overtime automatically adjusts the classification to match users’ preferences.
Your bounce rate is the percentage of email addresses that your campaign could not be delivered to. Email providers and anti-spam networks monitor bounce rates for every campaign you send and use that information to decide if they'll accept mail from you in the future. Besides affecting an individual sender's reputation, high bounce rates can land our sending IPs on blacklists, and potentially affect others who use Maligun service. While Mailgun do a lot of work behind the scenes to protect the customer from potential problems, there are certain best practices you can follow to keep things running as smoothly as possible. As a guideline, a well maintained, a permission-based list should typically see bounce rates of 2% or less for each email campaign (mailing) sent.
Spammers can forge, or "spoof," your domain's From address to make their spam look like it came from someone in your domain.
There isn't a way to stop email address from being spoofed. Anyone can spoof an email address. The trick is to identify spoofed emails when they are received, adjusting your spam filters and security on the incoming mail server. It also depends on how email is being spoofed.
To help prevent this, we recommend authenticating mail sent from your domain in two ways: by adding a digital signature to your messages that conform to the DomainKeys Identified Mail (DKIM) standard, and by creating SPF records.
Read more about spoofing as Wikipedia
Phishing attacks depend on sending an email that pretends to be from a trusted brand or individual. For the first decade or more of their existence, these attacks took a mass-mailing approach, hitting large numbers of email accounts in hopes of extracting a relatively small amount of value from each victim duped into giving up information.
Spear Phishing: Spear phishing is a specific type of phishing attack targeted at a single individual inside an organization where the payoff for a successful breach is particularly high.
Read more about Phishing attacks on Valimail.com
DDOS (denial-of-service) attacks are in the category of an email bomb, which is a form of net abuse consisting of sending huge volumes of email to an address in an attempt to overflow the mailbox or overwhelm the server where the email address is hosted in a denial-of-service attack
Read more about DDOS in Wikipedia
The bad news: limits and best practices
Unfortunately, even by having a perfectly functional mail system with all the above tools enforced you will not be 100% safe from the bad guys out there. Not all servers are using all three tools shown above. It is enough to look at the table shown in Wikipedia to see how that is possible: http://en.wikipedia.org/wiki/Comparison_of_mail_servers
Furthermore, there are some limits that you should always consider when dealing with SPF and DKIM:
As already said above DKIM alone does not grant in any way that the sender server is allowed to send outgoing mail for the specific domain
SPF is powerless with messages forged in a shared hosting scenario, as all the mail will appear as the same coming IP.
FOR CRM ONLINE:
Adding an SPF record to use CRM Online’s email service (com: mailgun.org, eu: eu.mailgun.org), will the policy only to all non-Mailgun email servers since there is at this point not possible to distinguish between users of this email service on IP address (they all users the same one). Do note, combining SPF with DKIM will be adequate to distinguish your domain, due to the fact each customer uses its own domain key.
Again, this technology relies upon the receiver's email server to use the tools…
How to set up SPF and DKIM?