SPF is a very broken anti-forgery technique which you should not implement. You should neither publish SPF records, nor check SPF on incoming email.
If you use SPF to reject mail, you will be causing genuine email to be rejected. There are much better alternatives which address the forgery problem without throwing the baby out with the bathwater.
SPF is Sender Policy Framework, formerly known as Sender Permitted From. It makes the assumption that mail from a given email address will only ever be seen when sent directly from certain IP addresses which are associated with that address, and it gives the domain owner a way to specify which IP addresses are 'acceptable'.
So, for example, I might state that mail from the infradead.org domain will only be seen from hosts with hostnames ending in 'infradead.org', from the IPv4 subnets 81.187.226.96/29 or 81.2.98.173/30, or from the IPv6 subnet 2001:b80:10b::/48. The idea would be that if you ever see a mail claiming to be from an address at infradead.org which is coming from a different host, then you should know that it's a forgery.
(That's about as far as the pretty graphical but mostly content-free official explanation goes, but if you take a moment to look closer or think about it, there's a little bit more to it than that...)
Unfortunately, this assumption is false. You do see perfectly genuine mail from my domain, from machines other than mine. This happens due to mail forwarding. People tend to change their ISP quite often, but don't want to have to tell everyone that they've changed their email address. So they have an account elsewhere, at a vanity domain or on another computer, and they forward mail from that address to whichever is their current ISP, or their employer.
There is a proposed solution to this -- the SPF advocates say that the whole world should 'upgrade' to make the initial flawed assumptions on which SPF is based come true.
Their idea is that when a server forwards a mail, it shouldn't just use the original sender's address as has been done for the last couple of decades. Instead, it should munge the sender's address as it resends the mail, faking an address at the domain which is actually responsible for the forwarding. Then, if a bounce is generated, that faked address receives the bounce and the bounce needs to be forwarded on to the original sender of the original mail.
This forwarding of bounces could be easily abused if done naïvely, so the idea is that we also include a timestamp to time-limit it, and a cryptographic hash which can only be generated locally. And of course you have to include the original sender's address; both the domain and the username. So when Bill Gates sends me an email to my vanity domain, for example, and my computers forward it to my current ISP, they'd actually end up rewriting the sender address to: SRS0=hh=tt=microsoft.com=bgates@infradead.org instead of leaving it intact.
This rewriting scheme they call 'SRS', and they need it to be implemented everywhere before they can use SPF without throwing away genuine mail.
You may have managed to spot one or two potential problems with this scheme all by yourself, but some people do seem to have trouble so let's spell them out explicitly:
SRS is not common. If you publish SPF records, you are going to be asking people to throw away genuine email which you did actually send.
If you check SPF records strictly for anything larger than a toy domain with a handful of users, then you're likely to be rejecting genuine email which is sent to you or your users.
There is little incentive for people to deploy SRS. It's a cumbersome and convoluted workaround for a broken assumption.
Those who believe SPF to be an insanely stupid idea will strongly resist, and those who are simply uninterested in SPF have little reason to want to use it. Yet SPF requires these these uninterested third parties to upgrade too -- it's not just limited to those who are participating.
People are very slow to deploy new ideas on the Internet, and especially with respect to email, even when those ideas are good. Many people haven't even managed to deploy Extended SMTP (ESMTP) yet, even.
There will be a porcine implementation of RFC1149 before SRS is ubiquitous.
Some people claim that SPF directly combats spam. It doesn't. SPF attempts to address forgery. In fact, a large amount of spam rates an SPF 'pass' result, because spammers have rapidly adopted SPF for themselves.
You still need a blacklist or other kind of trust database, to tell you which domains are trustworthy and which are not. But we already have lots of blacklists; it's just that we list the IP address instead of the domain name, to tell you which hosts are trustworthy and which are not. We don't need to break email and force everyone to upgrade to some bizarre new scheme just for that.
Although the uptake of SRS on forwarding sites will be low, it can still be a common spammer trick. Any spamming host can do the SRS trick to 'take responsibility' for forwarded mail. If your ISP receives mail from the address I used in the above example -- 'SRS0=hh=tt=microsoft.com=bgates@infradead.org' -- then they have no real way of telling whether it really did come from Bill Gates via my servers, or whether it's a fake. SPF would just accept the mail, depending on my domain's reputation in the trust database. It's all about how much you trust the one server which is offering the mail -- it's not an end-to-end authentication.
So SPF is really not any better than blacklisting by IP address or HELO name. Again, the breakage it imposes is not worth it.
The original sender address is useful information, and can be lost if an intermediate host mangles the mail by using SRS.
A sender address may be present in a blacklist, or may fail SMTP callouts. Yet an intermediate host 'takes responsibility' for the mail in question, causing it to be accepted anyway.
Sender addresses are also used for mail filtering. The sender address is the most reliable method of filtering mailing list traffic into its own folder; using Cc: and/or To: headers has both false positives and false negatives, and using the List-Id: header has false positives too. Again, SRS mangles this information.
How good are your spam filters? By rewriting the address of mail you're forwarding so that it appears to come from your own domain, you put your own reputation on the line. You could be blacklisted for mail which you claimed even though you didn't send it and you have no real knowledge of the original sender.
In the simple case without forwarding, SPF does manage to give a clear indication that a mail is valid. But that's not really very useful in practice. What we need is a blacklist but SPF gives us only a whitelist. We can't safely use that to reject mail -- we can only reject mail if it's clearly invalid, but SPF can only honestly say 'valid' or 'unknown'.
In summary, SPF doesn't really achieve what it sets out to achieve, and it breaks far more than it's worth. The costs far outweigh the benefits.
Some SPF advocates would claim that you can't make an omelette without breaking eggs; that the spam problem has got so bad that we have to break things and switch to a new email system.
That's silly for two reasons. Firstly, if you're going to break the world and make people upgrade, you might as well just switch wholesale to something like Internet Mail 2000 rather than pretending it's still compatible with SMTP. SMTP has enough other problems (like the way bounces are done, the difficulty of implementing variable preferences for different users, etc.) that we might as well fix it all at once, if we find it necessary to make such an incompatible change.
Secondly, and more relevantly, there are alternative schemes which give a measure of protection against address forgery but without the flawed assumptions, the loss of genuine mail, and the need for the rest of the world to 'upgrade'. You would have to be mad to choose all the problems of SPF, when other schemes are far saner can and offer the same benefits:
These schemes (and some others) are all fairly similar in nature. They offer a cryptographic solution to the forgery problem, where a sender adds headers to the mail with a signature, and that's verified by the recipient by checking against a published key for the domain or user in question.
Google, Yahoo and Earthlink, amongst others, are already using DomainKeys on outgoing email. AOL are reportedly going to be doing so in the early part of this year (2005).
These solutions don't suffer from the forwarding problem; they're compatible with email as we know it today, and offer the same protection as SPF does but without the false rejections.
There is an effort under way to merge these into a single proposal for approval by the IETF. See the MASS web page for more details.
These proposals are based on the same idea, originally inspired by the concept of using SRS on your own outgoing email instead of on forwarded mail.
For example, I never send email with MAIL FROM:<dwmw2@infradead.org>; it's always rewritten to appear to be from a time-limited SRS-encoded address.
This means that I can instantly start rejecting bounces targeted at the raw email address, because I know they're bounces to mail I did not actually send. And this, in turn, means that anyone doing SMTP sender verification callouts will reject faked mail claiming to be from that address, because a bounce cannot be delivered.
Since SMTP callouts are unpopular in some quarters, SES takes the concept further and provides alternative methods for validation of reverse-paths, and also allows for a message digest to be included in the reverse-path, to prevent replay attacks in the event of an encoded reverse-path being discovered and abused by a spammer.
By implementing SES or BATV, you can instantly stop accepting bounces to mail which you didn't send.
CSV is a very simple method for identifying servers by their HELO greeting instead of just by their IP address. It uses SRV records in DNS to specify which IP addresses may use a given hostname in their HELO greeting. It also offers a method of storing the reputation database in DNS.
Since SPF can only really be used for evaluating the trust level of the host which is actually sending the mail, CSV is sufficient to use as a viable alternative.
"When banks start DomainKeys or S/MIME signing all outbound mail, I promise to give up SPF and Sender ID."
-- Meng Weng Wong, inventor of SPF.