Changes

[[Category:OPS335]][[Category:OPS335 Labs]] = Email Servers =OVERVIEW=={{Admon/important|Warning|Your lab 4a must be complete before you can start this lab.}} In Lab 4a, you configured and ran the '''Postfix''' application for our MTA (a.k.a. SMTP server) on your '''vm2''' and '''vm3''' machines.

You may not be aware of it as That setup has some major drawbacks::* It required an user, but email is a very <u>complex</u> system SMTP server ('''MTA''') to administerbe configured on each machine. In fact, the more modern e-mail systems :* The Message Store (eg. web-based mail applications, etc'''MS''') are more technically involved than the other archaic, hard-would also be unique toeach machine -configure, and sometimes inter-operable systems what a user received on one server would not exist on any other.

We're going to spend three weeks working on it In this lab you will centralize some of this information, so that by the end of the course you will have a general understanding of what services are involved user can send email from any machine in sending, filteringthe network, and reading email. You will also have the skills incoming mail sent to configure a basic setup using the default services on CentOS 7centralized messages store.

You will begin by modifying the existing '''Postfix''' ('''MTA''') servers to make mail they send come from your domain (instead of each machine). Then you will add a record to your DNS server to allow mail to be sent to the domain itself, instead of the individual machines. Next, you will add a Local Delivery Agent ('''LDA''') to your '''vm3''' by installing '''dovecot-lda''', configure it, and test it to make sure that is is working correctly. Finally, you will set up an '''IMAP''' server called '''Dovecot''' on your '''vm3''' machine, so you can read your email from an MUA such as ''Thunderbird'' or a ''Webmail'' application. You will set up a webmail application called '''Roundcube''' in a later lab). === Learning About the Services Involved in an Email Delivery === In reality, the terms '''MTA''', '''MDA''', '''MUA''', '''LDA''' can actually be considered misleading since some of those services can be combined together to form a single entity (application), while other applications may operate as separate entities. There may be overlap, so if you don't find those acronyms helpful, don't worry too much about them. On the other hand, when referred to in diagrams, they can help to visualize those processes when trying to understand how an e-mail system works. [http://wiki.dovecot.org/MailServerOverview Here is an overview] of those terms (from the Dovecot wiki). It is worth viewing this link. In the diagram displayed above, the elements include: * '''User Account'''. The individual who wants to send or receive mail messages.* '''MUA''' (email client). This is the application that the individual uses to send or receive mail messages. It can be a '''native application''' or a '''web application'''. You will learn how to setup and use both types of these applications throughout the remainder of this course.* Two '''MTA''' servers. These are the servers responsible for getting your emails to the <u>destination</u> server.** They are similar to routers (which route packets) but work on the <u>application</u> layer rather than the <u>network</u> layer.** In our example, there are only two MTAs - but there can be several.** You connect to your MTA over a <u>secure</u> connection, so your emails can't be read by the operators of the network you're connected to.** The mail message then travels the rest of the way to the destination MTA <u>unencrypted</u>, so anyone with access to the routers in-between can read all your emails. That is why many organizations will refuse to send you confidential information over email.* '''LDA/MDA''' Server. This server will receive the email from the MTA, and will store it on disk in some format. '''MailDir''' and '''MBOX''' are the most popular mailbox formats.* '''IMAP/POP3''' server(s). When sending an email, you send it to the destination using your MTA, but you also want to save it in your '''"Sent"''' folder for yourself. This is accomplished by a separate connection to either your '''IMAP''' or '''POP3''' server.** Thus, a situation can occur that although you sent your email successfully, it may never make it to your "Sent" folder - the <u>second</u> connection to your IMAP server is quite unrelated to the first connection to the '''SMTP''' server.* '''DNS''' Server. A DNS server is also involved - it is needed to retrieve the address of the email server responsible for email for a particular domain. This is done with '''MX''' records. ===Online References=== * [Categoryhttps:OPS335//help.ubuntu.com/community/Dovecot Dovecot Community Documentation]* [http://wiki.dovecot.org/LDA Dovecot-lda]* [[Categoryhttp:OPS335 Labs]//wiki.dovecot.org/LDA/Postfix Configuring dovecot-lda with postfix] == INVESTIGATION 1: INSTALL THUNDERBIRD (MUA) and SETUP A REFERENCE CLIENT==

The diagram above doesnUnlike the '''mailx''t even include reading email, but it's as simple as it gets (if MUA) application you want to run an email server). You installed and used in Lab 4a, this lab will need to learn to administer basics of all be using the systems in the diagram. We won't go in too much depth, only the minimum you need. But we will not ''Thunderbird''' (for exampleMUA) go over every detail of Postfix, application instead which on its own has is a graphical application that uses a rather [https://en.wikipedia.org/wiki/Postfix_%28software%29#Architecture complex] [https://www.credativ.de/blog/postfix-architecture-overview diagram]'''centralized Message Store''' (MS) to retrieve and read mail messages.

=== Services involved in Although we will be eventually setting up the Thunderbird application to perform all the mail operations discussed above, you need to learn to '''"walk before you can run"'''. Eventually, you are going to set up all those mail services, but to begin with, you will set up an email delivery ===client to connect to an already working server which is the '''Seneca email server'''. Once we learn how to do this for our Seneca email account, then we can use it for our mail servers for our VM2 and VM3.

The terms MTA, MDA, MUA, LDA are not 100% well defined, because few of '''Perform the related services are simple and do exactly one thing. There is overlap, so if you donfollowing steps:''t find the acronyms helpful - don't worry about them. But they can help organize your thoughts when trying to keep all this in your head.

[http://wiki.dovecot.org/MailServerOverview Here#Switch to your '''host'''s an overview] of from machine, and install the Dovecot wiki, it's worth reading''Thunderbird''' email application.#When you first launch the Thunderbird application, a configuration dialog should appear as shown in the diagram below:

In our diagram we have<br>::[[Image:Seneca-student-thunderbird-email-setup.png|600px]]<br><ol><li value="3">Use the data in the table below to configure the Thunderbird settings dialog box for YOUR Seneca e-mail account:</li></ol>

* A user. That{| class="wikitable" border="1" style="margin-left:40px;"! Setting !! '''Incoming: IMAP''' !! '''Outgoing: SMTP'''s the person who wants to send an email.* An MUA (email client)|-| '''Username'''|| yoursenecauserid@myseneca. This is the application the user uses to send an emailca || yoursenecauserid@myseneca. It can be a native application or a web application. Weca|-| '''servername'''ll set up both types|| outlook.* Two MTAsoffice365. These are the servers responsible for getting your emails to the destination servercom || outlook.** They are similar to routers (whcih route packets) but work on the application layer rather than the network layeroffice365.com** In our example there are only two MTAs |- but there can be several.** You connect to your MTA over a secure connection, so your emails can| 't be read by the operators of the network you're connected to.'port''' || 993 || 587** The rest of the way to the destination MTA the emails travel completely unencrypted, so anyone with access to the routers in between can read all your emails. This is why many organizations will refuse to send you confidential information over email.|-* The LDA| '''security''' || SSL/MDA will receive the email from the MTA, and will store it on disk in some format. MailDir and MBOX are the most popular mailbox formats.TLS || STARTTLS|-| '''References''' * When sending an email you send it to the destination using your MTA, but you also want to save it in your | colspan="Sent2" folder for yourself| [1] [https://employees. This is accomplished by a separate connection to your IMAP or POP3 serversenecacollege.** This is why ca/spaces/77/it can happen that you sent your -services/wiki/view/2394/other-email successfully but it never makes it to your "Sent" folder - the second connection to your IMAP server is quite unrelated to the first connection to the SMTP server.clients ITS - Configuring other Email Clients]* Note that a DNS server is also involved - it's needed to retrieve the address of the email server responsible for email for a particular domain. This is done with the MX records we looked at in the DNS labs.|}

Install Thunderbird {{Admon/important |Unencrypted Options|Notice that there are <u>unencrypted</u> options available to connect to your SMTP/IMAP servers but those are rarely used these days - the potential for abuse is too great. On a free wifi network, the operator would be able to not only read your email, but also obtain your password without any password/encryption cracking tools. In fact, even on a private wired network, it is not uncommon for an employer to use a packet sniffer utility to monitor all the traffic going over their network (Packet Sniffing applications were actually found to be legally acceptable practice if used by the management of organizations)}}<br><ol><li value="3">After you create your host'''Thunderbird''' account, you should be able to read your existing email and send new email within the Thunderbird application.</li><li>Take time to view your ''Account Settings'' and configure it like ''Preferences'' to get a feel for what settings exist. For example:<ul><li>How often will Thunderbird check for new messages?</li><li>Will the messages you write be in HTML or plain text?</li><li>How do you change your SMTP server settings? Why are they in a different section?</li></ul></li><li>The main objective of thissection was to learn how to setup your Thunderbird application to read your Seneca email, obviously using so in the next section you can use the exact type of setup for your own information:email server.</li></ol>

[[Image:Seneca'''Record steps, commands, and your observations in INVESTIGATION 1 in your OPS335 lab log-student-thunderbird-email-setup.png|600px]]book'''

After you create In Lab 4a, both of your account - you should email servers were sending mail messages addressed from users of the actual machines themselves. This would be able confusing for the receiver who might get emails from the same user @vm1, vm2, and vm3. Which would they respond to? To avoid this problem from occurring, we can make all servers make the sent mail appear to read your existing email come from a central location (usually the '''domain'''), and send new make incoming email in Thunderbirdsent to that address to be accessible from machines within our network.

Look through '''Perform the Account Settings and Preferences to get a feel for what settings exist. For examplefollowing steps:* How often will Thunderbird check for new messages?* Will the messages you write be in HTML or plain text?* How do you change your SMTP server settings? Why are they in a different section?'''

If everything is working - that# Issue the '''mail''s good, now you know what you'll try command to build in view the email labsmessages you sent between your '''vm2''' and '''vm3''' in your lab 4a. The goal Notice that each is addressed from root on whichever machine sent it.# On both machines (vm2 and vm3), edit the '''/etc/postfix/main.cf''' file to have change the exact same setup using '''myorigin''' parameter from '''$myhostname''' to '''$mydomain'''. Restart the '''postfix''' service.# Now, send emails messages (via the '''mail''' command) between both of your servers instead of Seneca serversvm2 and vm3 machines, and view the mail messages by issuing '''mail''' in each vm. The sender address should now read that the received mail messages came from '''root@yourdomain.ops'''.

We::The next step is to configure what addresses that the server will receive email for. This is done using postfix by setting the ''ll use Postfix as the MTA, and we'll mydestination''' parameter (configuration variable) to include '''$mydomain''' (this is assuming you've set it up on '''vm2mydomain''', '''myorigin''. This will be the email server for your internal network. You'll be able to send email out of your network, and receive email from within your network, but not receive from outside your network because:# Outsiders will never find the MX recors for your domain, because there are no .org servers pointing to your DNS server (you haven't paid for it''inet_interfaces''' properly).# Even if they did - your local network is using IP addresses on a private subnet, which is not routeable on the internet, so it cannot be reached from the outside.

Postfix will work with <ol><li value="4">Edit the '''/etc/postfix/main.cf''' file for '''vm3 ONLY''', scroll down to the line containing: '''mydestination''' and change line to the default configurationtext shown below:<br><source>mydestination = $mydomain, so enable it$myhostname, start itlocalhost.$mydomain, and check that itlocalhost</source>'''Note:''' Even though your machine's runningname is ''vm3.yoursenecaid. Also look ops'', your postfix MTA will also receive emails addressed to the domain called: yoursenecaid.ops</li></ol><br>::In order for it in this to work, we need to add a DNS record that will point mail sent to the list domain towards one of listening ports:the SMTP servers configured to accept it.

<source langol><li value="bash5">netstat -atnpAdd an '''MX''' record to the forward lookup zone on '''host''' so that all incoming mail addressed to the domain is sent to your vm3.</li><li>Restart the service and use the '''dig''' command to confirm that it works.</li><li>Send an email from your '''vm2''' to '''root@yourdomain.ops'''</li><li>Confirm that it arrives on your '''vm3''' machine</li></sourceol>

== Testing When email is sent from either vm, it is addressed from the connection domain, but receiving MTAs might query why mail sent from vm2 doesn't match the address of the MX record for the domain. This would be a red-flag for potential spam. To avoid this, we can relay all mail sent from vm2 (or any other machine in our network) through vm3 so that it properly appears to Postfix ==come from the mail server that matches the MX record for the domain.

Connect from your server to your server using telnet'''Perform the following steps:'''

# Move to your vm2 machine.# Direct your '''vm2''' MTA to relay mail through vm3, by making the following editing change for the '''/etc/postfix/main.cf''' file:<br><source lang>relayhost ="bash"vm3.<yourdomain>telnet localhost 25.ops</source># Restart the '''postfix''' service.# Next, you must instruct your '''vm3''' machine to allow your vm2 machine to pass email through it by making the following editing change to the '''/etc/postfix/main.cf''' file:<br><source>mynetworks = 192.168.X.0/24</source>NOTE: Substitute in your '''own network''' for X<br><br># Restart the '''postfix''' service.

Note that it will tell you once your connect that '''Escape character All mail is '^]''', which means your can hold control and press the square bracket key now being delivered to end the session a centralized location (and then exit the telnet appalso appears to be coming from that same location), but a user would still have to access that server to retrieve it.

If this worked - that means the service is running and listening === Install and responding to connections. Let's see if it works from other machines. Telnet to vm2 from Configure the host Local Delivery Agent (connect to the SMTP portLDA/MDA) and see if it works. If your firewall is set up properly - it shouldn't, you'll need to allow incomming connections to TCP port 25.===

Once you open Postfix is capable of performing the port in the firewall - try the telnet commandfunction of an LDA, but its LDA capabilities are limited, thus postfix is generally not used for that purpose. You should get a different error this time. This time Currently, the problem most popular LDA is that your service isn't listening on the outside interface'LMTP'', but we will be installing, configuring, and using an LDA called '''Dovecot''' since itis also popular and we will setting up Dovecot as an '''IMAP'''s currently configured to listen only on server later in this lab. Using both Postfix and Dovecot will actually increase the loopback (lo) interfaceperformance of our IMAP server.

The first change #Move to your '''vm3''' machine.#Dovecot is not installed when you installed your Virtual machines in previous labs.<br>Install the Dovecot application by issuing the Postfix configuration will be following command:<br><source>yum install dovecot</source>#Edit your '''/etc/postfix/main.cf''' file and scroll down to make it listen (or search for incoming connections on the extrnal intercace, that) '''mailbox_command'''s eth0 from . Add the VMs point of view.following line:<br><source>mailbox_command = /usr/libexec/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"</source>

The configuration file is ::NOTE: Do <u>'''not'''</etcu> replace any variables, those are set automatically by Postfix when it runs the LDA. If you are interested in learning more about the Dovecot application, you can read about dovecot-lda [http:/postfix/mainwiki.cf''' Edit it dovecot.org/LDA/Postfix here] and change [http://wiki.dovecot.org/LDA here].<br><ol><li value="4">Finally, edit the '''inet_interfaces/etc/dovecot/conf.d/10-mail.conf''' file and indicate where you want your mail delivered by including the following line:<source>mail_location = maildir:~/Maildir</source></li><li>Restart your postfix service.</li><li>While the emails are still stored only on VM3, they will now be easier for other machines/services to '''all'''access.</li><li>Due to permissions on the directories where mail will now be stored, root will no longer receive mail. Check the logs for an indication as to why.</li></ol>

At this point we should also set the string that will end up in the '''From:''' header Record steps, commands, and your observations in INVESTIGATION 2 in messages sent by this server. Change '''mydomain''' to your domain name and and '''myorigin''' to '''$mydomainOPS335 lab log-book'''.

Restart postfix, confirm ==INVESTIGATION 3: USING THUNDERBIRD (with netstatMUA) that it's now listening on all interfaces, not just loopback, FOR VM2 and test connecting to it from the host.VM3 MACHINES ==

== DNS Server used by the host = Accessing Received Mail Messages on VM3 VIA IMAP ===

So far you're probably not using vm1 as First, we will set up the DNS IMAP server so we can read email. The current way we have configured our mail server on our VM3 machine should allow all the email for your hostanyaccount@yoursenecaid. Weops should be delivered to our ''ll need to change that for email to work correctly. Normally you don't need an entire DNS server for running an email server, you just add an MX record in your registrarvm3's web interface, but we haven't paid for our domains so we don't have a registrar machine. We will set up Dovecot with a web interfaceIMAP to get easy access to that email.

Modify your /etc/resolv.conf to use vm1 as '''Perform the name server. This will not be a permanent change but thatfollowing steps:''s ok, we only need it for the email labs. You'll have to remember to do this every time you reboot your host.

If you regular #The configuration file for the Dovecot service (internetwhich is not the same thing as dovecot-lda) DNS lookups slow down significantly - is: '''/etc/dovecot/dovecot.conf'''. Modify the '''protocols''' option so that Dovecot will work with IMAP connections, no POP3 or LMTP.# Start the dovecot service, and ensure it will always start automatically when the machine boots.# Use the '''ss''' command to confirm the service is listening, and use '''nc''' on the '''host''' to confirm you can connect to it.# You'll probably fail, so using the information gathered from '''ss''', modify the firewall on vm3 to allow IMAP connections from your local network and try '''nc''' again. Once it works, do not forget to update save this change so it will still be there the DNS next time you reboot.#If you can connect - it's now time to do something wrong, that is allow connections to our IMAP server configuration on vm1 to forward requests over an unencrypted connection.# Edit the '''/etc/dovecot/conf.d/10-auth.conf''' file and set '''disable_plaintext_auth''' to '''no'''.# Then edit the Google nameserver (8'''/etc/dovecot/conf.8d/10-ssl.8conf''' file and set '''ssl''' to '''yes'''.8) instead <br><br>'''Note:''' This combination of parameters will allow your hostusername and password to be sent over the internet in plain text, for anyone interested to look at. In a later lab we'll set up secure SMTP and IMAP connections, for now this is all we have time for.<br><br># Restart dovecot so the changes take effect.

== Setup = Connecting to IMAP Servers Using Thunderbird ===

The process is a little challenging because Thunderbird tries really hard #On your '''host''' machine, return to prevent you from connecting to a server that doesn't work the Mail Account Setup dialog box (and ours mostly doesn't work at this pointeg. near top of lab). # Set it up a '''new email account'''. You will be using account settings to connect to your '''vm2''' for '''SMTP ''' and '''vm3''' for '''IMAP'''. Use <u>no </u> encryption, and use normal password authentication for IMAP (we don't have an IMAP server running yet, but that's ok).Refer to the diagram below for reference:

Thunderbird won't let you proceed with the <ol><li value="Done3" button because you will fail >Try to connect to your IMAPserver with Thunderbird by clicking on your '''Inbox'''.</li><li>If nothing happens, then check the Thunderbird Activity Manager for any errors. If the connection is successful, you should see the '''Trash''' box <u>appear</u> below Inbox. </li><li>Use the "Advanced config" button Thunderbird application to send an email to bypass your myseneca address. If you've done everything right, it will send the message successfully</li><li>Verify that checkyour message has been sent. Check your myseneca email and look at '''/var/log/maillog''' on vm2 (your email server).</li></ol> === Sending a Mail Message from VM2 (Using Thunderbird)=== '''Perform the following steps:'''

Now use Thunderbird #Use the '''ss''' and '''nc''' commands (like you did in lab 4a) to confirm your service is listening on the correct ports/interfaces. You will probably have to open the appropriate firewall port on '''vm3''' to allow incoming '''SMTP''' connections.<br><br>'''Note:''' You should be able to send an email to your myseneca any regular user <u>on</u> '''vm3''' using the email address'''yourusername@yoursenecaid. If youops'''ve done everything right - it will send using the message successfully but it will fail Thunderbird application on your host machine (which is configured to save it in use the Sent folder, because that's done with IMAP and you don't have an IMAP server yetaccount on your vm2).<br><br>

Still <ol><li value="2">Create a new account on your '''vm3''' machine using only your <u>first</u> name. We will use this account as a one- verify that your time "test" if the mail message has been sentreceived on your VM3 machine (from your VM2 machine). Check your myseneca email and look at <br><br>'''Note:''' It is <u>'''important'''</varu> that you '''<u>don't</log/maillogu>''' create this same account name on your vm2 (your email server)machine, since you want to easily identify the difference between the sending and receiving SMTP servers.<br /><br /></li></ol>

<ol><li value= Completing "3">Use the Lab =new account in Thunderbird to send an email to '''firstname@yoursenecaid.ops''' and then check the contents of '''/home/firstname/Maildir/new/''' on your '''vm3''' machine. There should be a file there with the contents of your email.</li><li>If there is no file, then check the log file '''/var/log/maillog''' to see what went wrong.</li><li>If you can see a file in the '''/home/firstname/Maildir/new/''' directory, then review the procedures on how you got the email server working (since you have performed many steps and set up many services).</li><li>Refer to the diagram at the top of this lab. Which services have you currently set up? Record your findings in your lab Logbook.</li></ol>

Students should be prepared with {{Admon/important |Encountering error messages when sending email|If you cannot properly receive sent e-mail messages, check the '''/var/log/syslog''' file for errors.<br><br> If you locate an error message in that file such as: '''Fatal: Error reading configuration: Invalid settings...'''all required commands (system information) displayed , then add the following <u>parameter</u> in a terminal '''/etc/dovecot/dovecot.conf''':<br />'''postmaster_address <nowiki>=</nowiki> DOMAIN''' (or multiple terminalswhere DOMAIN is actually <u>your</u> domain) prior to calling the instructor for signoff.<br /><br />After you have saved those changes, then '''restart'''your dovecot service. This problem can also be resolved by properly setting the hostname of your machine to include the domain.}}

{{Admon/important |Backup your VMs!|You MUST perform a '''Arrange evidence (command output) for each full backup''' of ALL of these items your VMs whenever you complete your '''OPS335 labs''' or when working on your screen'''OPS335 assignments'''. You should be using the dump or rsync command, then ask and you should use the Bash shell script that you were adviced to create in order to backup all of your instructor to review them VMs.}} '''Record steps, commands, and sign off on your observations in INVESTIGATION 3 in your OPS335 lab log-book''' == COMPLETING THE LAB == ===Online Submission===Follow the instructions for lab4b on blackboard.<!--===Andrew's completionsections=== You may choose to:* Submit screenshots of your work on Blackboard, in which case you don't need to come to the lab.* Or come to the lab, show me your work, and talk to me about it. I want to hear what you've learned and answer any questions you have. You'll get the same grade regardless of how you choose to submit your work.