No, I’m not talking about sunscreen. The SPF I’ll be discussing here is Sender Policy Framework. It’s an email security measure that can prevent unauthorized use of your email domain name. I’ll be going over a few things in this post, but if you just want to know how to craft an SPF record, you can scroll to the “The SPF Puzzle” section below. For those of you who enjoy learning deeper on the subject, enjoy!
What is SPF (Sender Policy Framework)?
SPF is a technique that uses Domain Naming Service (DNS) records to publish a list of email servers that are authorized to send email for a specific email address domain (the part of the email address after the “@” symbol). Email security systems use SPF checks to determine if an incoming email is coming from an authorized source. An SPF check will return one of 4 results: PASS, HARDFAIL, SOFTFAIL, and NEUTRAL. Only emails that generate a PASS result will be allowed to go through without issues. The other results either result in a rejection (HARDFAIL) or a spam score increase (SOFTFAIL). NEUTRAL results are handled differently by each security system, but often result in a slight increase to spam score.
The Problem – Spoofing
The biggest problem with email over SMTP is the lack of security. There is no SMTP-based solution for making sure a message came from the person whose name is in the From: field. That limitation allows all kinds of mischief. If you have your own Email server software set up on a computer anywhere on the Internet, you can send as any email address. From the time that SMTP was created until about a decade ago, there was absolutely nothing to stop you from pretending you are anyone else. This is called “Spoofing.” And it’s a huge problem.
Spoofing is at the core of every email-based malicious activity. This shows up in broad phishing attacks that use a randomized source address as well as targeted spear phishing attacks that fake the source address of a company’s known associates. Malicious emails with infected attachments or links will use the contact list of a compromised computer to craft emails that spoof the from: address. The ways that email spoofing is used to attack individuals and companies are numerous.
SPF – A Tool to Stop Spoofing
To stop spoofing, there needs to be a way to verify that a message was sent by an authorized server. Depending on the size of a business and how many different services or applications they use to send email, defining what counts as an authorized email server can be complicated in most situations. Most mail security platforms these days keep databases of information about where email for various domains comes from. This information is then used as part of an IP address’s “Sender Reputation” for a given email domain. This technique is useful, but communication between mail security developers isn’t always great and all of them tend to apply their own version of Sender Reputation techniques. This means that both authorized and unauthorized delivery can occur inconsistently, depending on whose security services you use.
SPF, however, allows an organization to define which servers can send email using their domain name(s). It does so through the use of a DNS TXT record (usually called a “text” record). TXT records can contain any information you want, but are regularly used by applications to perform certain actions. For instance, setting a domain up for use with Office 365 involves setting a TXT record with a specific value. Microsoft’s domain validation software will generate a code that an admin for that domain with permission adds to their public DNS data. Once the TXT record is added, domain validation will look for the record. If it is found, that domain is validated and can be used in Office 365.
How SPF Works
For various reasons, the major flaws in SMTP that allow spoofing have not been directly addressed. The responsibility for securing email and stopping malicious attacks through email falls to an organization’s Email Security system. This system can be a cloud-based spam filtering service, an application installed on a mail server to scan incoming messages, or a dedicated “appliance” or VM whose sole purpose is providing mail security services. As such, SPF policies are detected and applied by an email security solution, rather than by the mail server itself.
When an email is received by a mail security system, a reputation check occurs. The reputation check has a few different parts (Not necessarily done in this order):
1. A reverse DNS lookup of the sending IP address
2. A check against the security software’s database of known sending IPs for that domain
3. A lookup of any TXT records in the domain, looking for SPF information.
Once SPF info is found, the security software will check the detected sending IP address against the information contained in the SPF record. If the IP address is listed as valid, the SPF lookup passes and the email moves on to other filtering processes. In many situations, SPF records will completely disable the use of Reverse DNS lookups and checks for known sending IPs. If the SPF lookup fails, the security software will usually reject the message or, at the very least, mark it as likely fraudulent and continue processing.
Working with SPF
SPF rules look very confusing at first glance, but with a little explanation, they become much easier to understand. For reference, the SPF record of my email domain, acbrown-it.com, is this: “v=spf1 include:spf.protection.outlook.com -all”
An SPF record works from left to right. If my domain had more than a single mail system sending for it, you’d see additional entries aside from “include:spf.protection.outlook.com” and “-all.” Yes, those are separate entries in the record. In this situation, though, you can see that my email is provided by Office 365, as that is the default SPF record for that service. I’ll explain how the record is built.
First, you need to have the section that states, in essence, “This is an SPF record” and which version of SPF is being used. SPF 1 is pretty much the only version in use at the moment, so a valid SPF record needs to have “v=spf1” at the start. Everything after that is an entry, with each entry separated by a space. Each entry uses a “type” followed by the authorized value. That value depends on the type. The record Types are shown below:
| Entry Type | Purpose | Usage |
| A | Entry is a DNS A record | A=mail.domain.com |
| MX | The MX records for the domain are valid | MX |
| IP4 | Entry is an IPv4 IP address | ip4:12.34.56.78 |
| IP6 | Entry is an IPv6 IP address | ip6:::1 |
| Include | Use this domain's SPF record | Include:domain2.com |
| All | Use everything! | all |
A, IP4, and IP6 are fairly self explanatory in that table, but the others are a little odd. The MX record value tells the SPF processor that whatever the MX record of the domain shows, that is a valid source of email for the domain. Include tells the system that it should check the SPF records of the domain that you define. That’s why my email domain shows “include:spf.protection.outlook.com”. That domain name contains an SPF record with all of the possible IPs used by Office 365 when sending email. So my SPF record says “Whatever info is included in that SPF record, well that works for this domain as well.”
Include entries are generally used when you have a cloud-based mail service that sends email on behalf of your domain. Companies like Constant Contact, Mail Monkey, Google, and Microsoft publish TXT records at a specific host name that defines all of their IPs. If you were to run a TXT lookup on spf.protection.outlook.com, for instance, you’d see the following SPF record:
“v=spf1 ip4:207.46.100.0/24 ip4:207.46.163.0/24 ip4:65.55.169.0/24 ip4:157.56.110.0/23 ip4:157.55.234.0/24 ip4:213.199.154.0/24 ip4:213.199.180.128/26 ip4:52.100.0.0/14 include:spfa.protection.outlook.com -all”
Note that this is mostly IPv4 address ranges (in CIDR notation). But there is another include shown there as well. You can run a lookup against that host name to get more valid IP addresses and another include, and so on. Why does Microsoft do it like this? Mostly for readability, but also because there is a general rule that each SPF record should contain no more than 10 entries. There is another rule that says there should be no more than 10 DNS-based lookups total. Each A, MX, and INCLUDE entry counts as a single DNS lookup, because the SPF processor will resolve those records to get the actual IP addresses associated with them. MS has a lot of IPs that can send email, so they use nested INCLUDE entries to meet both requirements.
The last type in the table is “ALL.” This is a special entry type that should only be placed at the end of an SPF record because it represents every other possible IP address. The character you place in front of the ALL depends on how you want the SPF processor to treat every other IP address in the world. The processing of SPF entries occurs based on special characters that denote what the processor should do if a match occurs. The next table explains how this works:
| Character | Result | Meaning |
| + | Pass | A match causes a PASS |
| ? | Neutral | This is kind of useless |
| ~ | SoftFail | A match is noted, but not rejected |
| - | Fail | A match is rejected outright |
Each of these symbols represents an action that the SPF processor will take if a match occurs. Unless otherwise specified, every entry has an implied + action associated with it, so my domain’s SPF record actually processes like this – “v=spf1 +include:spf.protection.outlook.com -all” – and will result in a PASS result if the sending IP is listed in Office 365’s SPF record(s). The “-all” at the end tells the processor to reject messages from any other IP because the “-” character tells the processor to drop everything that matches the entry. Since the entry the “-” is in front of is an ALL, everything outside of spf.protection.outlook.com gets dropped.
The other two symbols are special cases. A “?” character in front of an SPF entry is essentially useless because it tells the SPF processor that you have no opinion about whether the IP can send or not. Using “?all” is like saying, “I really couldn’t care less about this whole SPF thing and have no opinion about any other IPs that aren’t listed.” You may as well have no ALL entry. Or an SPF record if you use that character for all entries.
The “~” character is, however, a little more useful. It stands for SOFTFAIL, which is somewhere between a pass and a fail. In many situations, SPF records are configured with ~all at the end instead of “-all.” If they are not certain they know the IPs of all servers that will send valid emails for their domain, they may not want to immediately drop messages that fall under the ALL entry. This is a good transitional tool for moving from no SPF to enforcing SPF, or for just using SPF as a recommendation, but not enforcing it. It’s a lower security way to implement SPF without causing partners that use your domain to get rejected outright if the mail admin doesn’t know about them. Strategically, it’s better to use “-all,” but the choice is up to you.
As for what happens when the SPF processor matches a rule that results in a SOFTFAIL, that’s kind of complicated. The results depend heavily on the email security system that is running the SPF processor. Different systems have different rules for handling FAIL and SOFTFAIL results. For instance, some will simply increase the “spam score” associated with that email, increasing the likelihood the email will get dropped. Others will treat FAIL and SOFTFAIL results the same, rejecting both outright. Some systems have highly granular controls that allow the organizations that use them to determine what happens on their own.
Wrapping it Up
So hopefully you know a little more about SPF, how it works, and how to use it. Understand, however, that SPF is just one technique for preventing domain spoofing in email. In future articles, I’ll cover the complementary tool DKIM and the DMARC system, which uses DKIM and SPF to virtually guarantee that an email domain cannot be spoofed. As always, if you have questions or find anything in this post to be confusing, leave a comment and let me know! I’m always happy to help my fellow admins out and clear up confusion.
