I've received a lot of questions about SMTP/email servers and just how the heck they work. I too was very confused as to how mail servers do their thing, but supporting mail server software quickly builds confidence in that area.
I'm writing a few articles to help shed some light on some of the lesser-known facets of mail servers.
The Basics
Firstly, "SMTP" is just a "simple mail transfer protocol". It just describes a very simple protocol to transfer mail from server A to server B. The rest of the complexity of email comes from the other standards and protocols and the mail servers themselves.
Here is what a basic SMTP connection looks like at the raw packet level. It's just a simple text protocol.
First your server identifies itself with this command:
>> EHLO mydomain.com
The "EHLO domain" identifies your own server, which might not be the email's FROM domain. Ideally, your server's IP address should have a PTR record that points to the domain specified here. Some mail servers will reject your mail if that is not the case.
The server will respond with capabilities; for example, from Google's SMTP server:
<< 250-mx.google.com at your service, [2600:380:ec80:24c9:e4c4:c972:3df3:fbe2]
<< 250-SIZE 157286400
<< 250-8BITMIME
<< 250-STARTTLS
<< 250-ENHANCEDSTATUSCODES
<< 250-PIPELINING
<< 250-CHUNKING
<< 250 SMTPUTF8
I love that "at your service" bit. So polite. :)
Next we send where the email is coming from.
>> MAIL FROM:<test@example.com>
<< 250 2.1.0 OK l9si4700839qkp.380 - gsmtp
This is the "envelope" address being set. Please note this is NOT the "From:" address in the email content. This is invisible to the user (visible in headers only), and usually contains a "bounce" address in order to return bounced email messages to the host rather than unsuspecting users.
The "From:" address in the email content is not part of the SMTP protocol.
Afterwards we set the recipient - the "envelope" recipient.
>> RCPT TO:<mukunda@mukunda.com>
<< 250 2.1.5 OK l9si4700839qkp.380 - gsmtp
Servers can set multiple recipients this way. Note that this is entirely what selects who is receiving the mail. The headers in the email content like the "To:" field are not used for delivery. Think of the "From:" and "To:" field as being hidden inside the envelope. The courier has no idea about them.
Another important facet of the RCPT TO field is that you are typically only allowed to send mail TO the domain you are connecting to. You would need to authorize your connection to send anywhere else, if that is allowed ("relaying").
"Open relays" are improperly configured mail servers that allow any sender to send anywhere withou authentication. You might hear the term "open relay blacklist" which are IP address lists that block those servers.
Anyway, next step is sending the email data.
>> DATA
<< 354 Go ahead l9si4700839qkp.380 - gsmtp
We're in the clear. What follows next is not part of the SMTP protocol - it's email content. Basically we just send a chunk of data that contains the email (very readable, with MIME formatted fields), and then terminate it with a dot (and yes there are some obscure semantics which allow you to escape the dot if necessary).
>> asoifiajo
>> asdf
>> asdfasdfa
>> sd
>> f
>> .
<< 550-5.7.1 [2600:380:ec80:24c9:e4c4:c972:3df3:fbe2 11] Our system has
<< 550-5.7.1 detected that this message is not RFC 5322 compliant:
<< 550-5.7.1 'From' header is missing.
<< 550-5.7.1 To reduce the amount of spam sent to Gmail, this message has been
<< 550-5.7.1 blocked. Please visit
<< 550-5.7.1 https://support.google.com/mail/?p=RfcMessageNonCompliant
<< 550 5.7.1 and review RFC 5322 specifications for more information.
l9si4700839qkp.380 - gsmtp
Google's server was smart enough to figure me out. This is a good way to get your server temporarily banned.
See my SMTP submission template for a well-formed MIME email example.
Once the email is "submitted" to the server, it is either relayed or stored in a mailbox database.
From the mailbox database, it can be read by users through various web-based mail programs or standard protocols, such as POP3, IMAP, ActiveSync, EWS, etc.
Domains and Mail
Now that we maybe have an idea how the protocol works, let's look at domains and mail.
Domains and DNS are a critical part of mail flow. Poor configuration of a domain can result in your mail being blocked.
When a server is relaying mail (after it has been submitted via SMTP), it checks the domain of the recipients.
MX records are used to point out mail servers. MX records should point to a FQDN that handles your mail. Here is an example of looking up the server for my address mukunda@mukunda.com
root@marctest:~/snap# dig mx mukunda.com
; <<>> DiG 9.16.1-Ubuntu <<>> mx mukunda.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60451
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;mukunda.com. IN MX
;; ANSWER SECTION:
mukunda.com. 299 IN MX 0 aspmx.l.google.com.
;; Query time: 116 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Sun Jun 06 23:37:00 UTC 2021
;; MSG SIZE rcvd: 71
We see the "mail exchanger" as "aspmx.l.google.com". This is the SMTP server's address.
If there is no MX record found, mail servers will typically fall back to a direct A record; so MX records are not entirely necessary.
Email doesn't need to be addressed to a domain, either. You can address it to a plain hostname or IP address.
user@192.168.1.32
user@127.0.0.1
user@localhost
I think this is most useful when sending email to a machine rather than a user. For example, Google Groups don't (at the time of writing) have a way to send webhook notifications, so what I did to get around that was subscribe to the group with an email such as "parser@ipaddress" to send email notifications to my web server. From there, I parsed the email payload and generated the notifications manually.
"Address literals" are well-defined in RFC 5321 (4.1.2):
Path = "<" [ A-d-l ":" ] Mailbox ">"
Mailbox = Local-part "@" ( Domain / address-literal )
address-literal = "[" ( IPv4-address-literal /
IPv6-address-literal /
General-address-literal ) "]"
; See Section 4.1.3
Fun stuff, right? If email really does it for you, I recommend giving the RFC a read-through.
Local Testing
"How do we test mail flow in a local network?"
This is tricky at first but not too bad once you have a grasp on a local DNS setup.
The first thing you want to do is set up the different mail servers in your network. My VM Primer has some handy tips for setting up virtual machines.
For example, we could set up a configuration like so for a Windows-based mail environment:
VM1: Domain Controller and DNS provider
This could easily be part of the Exchange server as well, but we can make a dedicated machine in order to split up our services for better management. It is also simply recommended to keep DCs separate.
This server will also be our DNS server to link different mail servers together.
VM2: Exchange 2016 mail server
Exchange is a beast of a mail server, but it's not too hard to set up--the installer will guide you, and you can typically google any errors. Download one of the Cumulative Updates to install a trial version.
I set up my Exchange server a while ago, so I can't quite remember the initial configuration, but the main important thing is to check over the connectors.
Connectors are what control inbound and oubound mail flow. If they aren't set up right, your mail server might not be able to receive mail, or it might not be able to send mail.
One other thing to take note of with Exchange is that it is highly integrated with Active Directory. If you want to set up two Exchanges with different domains, I recommend using separate domain controllers; otherwise you'll probably have a bad time. :)
In my lab, I typically test with just one Exchange server and mix it with other servers or methods like IIS SMTP, Kerio Connect, PowerShell, or plain telnet.
VM3: IIS SMTP mail server
IIS SMTP is accessible from the Roles and Features. Just pop open Server Manager and install it. It's managed by the IIS6 interface.
Remember, if something seems confusing, just try it out--can always start over if you irreversibly break something.
Here is a default domain for the server "jenny" I just installed this on.
This means that the server can accept email addresses such as "user@jenny".
You can add other domains to be "accepted" by this server. "Accepted" is a key word, because we would reject any mail if the envelope (RCPT TO) doesn't specify one of our accepted addresses.
Mail that is received by the server may end up in
C:\inetpub\mailroot\Drop
Putting it all Together
So, once we have our servers set up, the next step is simply connecting them together through the local DNS server. That is, each system is configured to use the local DNS server in their network settings.
For example, my DNS server is at 192.168.1.21. If something -can't- be resolved by it, then it does forward to my router, and then my ISP's DNS server takes over.
These are two domains in my testing network that I use to bounce mail around:
- arch.local
- nemesis.local
(In production environments it's frowned upon to use "fake" domains like this, and you are encouraged to purchase real domains. For testing it's absolutely fine.)
UPDATE: 6/13/21
.local works fine in Windows environments, but after having quite the struggle in a Linux environment, I would not recommend using .local for local testing either. You will have a bad time! ".local" is not a "fake" TLD. It has special use.
See .local on Wikipedia:
RFC 6762 reserves the use of the domain name label "local" as a pseudo-top-level domain for hostnames in local area networks that can be resolved via the Multicast DNS name resolution protocol. Any DNS query for a name ending with the label "local" MUST be sent to the mDNS IPv4 link-local multicast address 224.0.0.251, or its IPv6 equivalent FF02::FB. A domain name ending in local, may be resolved concurrently via other mechanisms, for example, unicast DNS.
Here are a few screenshots from the DNS manager.
Here is my main domain for my test environment. It's a bit cluttered because it is also the domain controller, but you can see some of the important mailer entries.
- MX record for my Exchange server (presto.arch.local) user@arch.local will mail to that server.
- Autodiscover record, useful for Exchange testing.
- mail.arch.local, not really important, but an alias for presto, though note that you are not supposed to point MX records to CNAMEs. I'm not sure why.
- Sub MX record for my IIS SMTP server (mex.arch.local) user@mex.arch.local will mail to that server.
My other nemesis.local domain contains another mail server.
- This configuration less common, as the base domain usually points to a web server (typically something like mx.nemesis.local would be used for the mail server).
- But in this case yes, the MX points back to nemesis.local and then the A record there is used to find the IP.
- MX records should not point directly to IP addresses.
When someone in the network sends to user@nemesis.local, it will look up the MX record under nemesis.local and let that server handle the message.
This gets pretty confusing when there are extra relays involved. For example, for some test scenarios, I would point arch.local's MX record to my IIS server (mex), and then the IIS server would relay to Exchange afterwards.
Testing
"Mail submission" is an important facet of mail servers.
This is very similar to the protocol bits we described above, except instead of "delivering mail to a server", we are "submitting a mail for delivery". It's basically the same thing underneath, but the implications are very different. The server will be relaying our mail message to the proper location.
As described previously, this requires authentication for sanely configured servers.
Luckily, sending mail locally is one form of valid "authentication". If it is coming from a local IP address, or better yet, the mail server itself (localhost), that is usually good enough to allow relaying.
It varies from server to server where the configuration for this is, but pretty much any mail server should let you allow mail through from local addresses on the internal LAN network.
You should be able to do things like Send-MailMessage from PowerShell to test your servers, for example:
Send-MailMessage -SmtpServer Presto -from test@example.com -to test@nemesis.local
This is pretty confusing to understand at first, but this is how it works:
The FROM address can be anything.
- There are spam prevention measures to prevent
"spoofing" emails in most production environments.
- You should disable SPF checks for testing.
- We don't really need to worry about those for test environments.
- There are spam prevention measures to prevent
"spoofing" emails in most production environments.
The SmtpServer is where the mail is being "submitted".
- In this case, we're submitting a mail to my Presto server, which is an Exchange server.
- From there, the server will relay it to nemesis.local.
The -to address here is the "envelope" address, so this mail will be relayed to the MX for nemesis.local, and end up in the mailbox test@nemesis.local.
You could also submit directly to the destination, e.g:
Send-MailMessage -SmtpServer mail.nemesis.local ` -from test@arch.local -to test@nemesis.local
This would send a mail on behalf of arch.local, and is much the same as what the Exchange server would do.
The big difference is the originating address, which has a big part in email authentication. SPF and DKIM are the main standards that control email authentication, and are quite outside the scope of this article.
When submitting to the Exchange server instead, that allows it to be the authoritative source for the email, and thereby highly increase the chance of deliverability.
This isn't important when testing, but it is critical in production.
You should also be able to send authoritative mail from the relevant web panels.
A large part of the SMTP protocol standard emphasizes deliverability.
- If your message can't be delivered upfront, you'll get an error during the SMTP transmission. Otherwise, you
- If an error happens later, it will bounce back and you should get an NDR (if enabled).
- A lot of the time, such NDRs are delayed, as the mail
servers will retry for some time before giving up.
- This happens if the error is not known to be permanent from the get go.
You can see it's easy to get lost in the details of email, but it is pretty basic at a high level if you only focus on delivery.
Things like mailbox storage, client protocols, and security add a whole lot more complexity. I'll probably write more in additional articles. Let me know via comment if you are interested in anything in particular.