Changes

[[Category:OPS335]][[Category:OPS335 Labs]] = Email Servers =OVERVIEW=={{Admon/important|Warning|Your lab 4a must be complete before you can start this lab.}} In Lab 4a, you configured and ran the '''Postfix''' application for our MTA (a.k.a. SMTP server) on your '''vm2''' and '''vm3''' machines.

You may not That setup has some major drawbacks::* It required an SMTP server ('''MTA''') to be configured on each machine.:* The Message Store ('''MS''') would also be aware of it as unique to each machine - what a user but email is a very complex system to administer, in fact it's a very complex system of received on one server would not exist on any other archaic, hard-to-configure, sometimes-interoperable complex systems.

We're going to spend three weeks working on it In this lab you will centralize some of this information, so that by the end of the course you will have a general understanding of what services are involved user can send email from any machine in sending, filteringthe network, and reading email. You will also have the skills incoming mail sent to configure a basic setup using the default services on CentOS 7centralized messages store.

You will begin by modifying the existing '''Postfix''' ('''MTA''') servers to make mail they send come from your domain (instead of each machine). Then you will add a record to your DNS server to allow mail to be sent to the domain itself, instead of the individual machines. Next, you will add a Local Delivery Agent ('''LDA''') to your '''vm3''' by installing '''dovecot-lda''', configure it, and test it to make sure that is is working correctly. Finally, you will set up an '''IMAP''' server called '''Dovecot''' on your '''vm3''' machine, so you can read your email from an MUA such as ''Thunderbird'' or a ''Webmail'' application. You will set up a webmail application called '''Roundcube''' in a later lab). === Learning About the Services Involved in an Email Delivery === In reality, the terms '''MTA''', '''MDA''', '''MUA''', '''LDA''' can actually be considered misleading since some of those services can be combined together to form a single entity (application), while other applications may operate as separate entities. There may be overlap, so if you don't find those acronyms helpful, don't worry too much about them. On the other hand, when referred to in diagrams, they can help to visualize those processes when trying to understand how an e-mail system works. [http://wiki.dovecot.org/MailServerOverview Here is an overview] of those terms (from the Dovecot wiki). It is worth viewing this link. In the diagram displayed above, the elements include: * '''User Account'''. The individual who wants to send or receive mail messages.* '''MUA''' (email client). This is the application that the individual uses to send or receive mail messages. It can be a '''native application''' or a '''web application'''. You will learn how to setup and use both types of these applications throughout the remainder of this course.* Two '''MTA''' servers. These are the servers responsible for getting your emails to the <u>destination</u> server.** They are similar to routers (which route packets) but work on the <u>application</u> layer rather than the <u>network</u> layer.** In our example, there are only two MTAs - but there can be several.** You connect to your MTA over a <u>secure</u> connection, so your emails can't be read by the operators of the network you're connected to.** The mail message then travels the rest of the way to the destination MTA <u>unencrypted</u>, so anyone with access to the routers in-between can read all your emails. That is why many organizations will refuse to send you confidential information over email.* '''LDA/MDA''' Server. This server will receive the email from the MTA, and will store it on disk in some format. '''MailDir''' and '''MBOX''' are the most popular mailbox formats.* '''IMAP/POP3''' server(s). When sending an email, you send it to the destination using your MTA, but you also want to save it in your '''"Sent"''' folder for yourself. This is accomplished by a separate connection to either your '''IMAP''' or '''POP3''' server.** Thus, a situation can occur that although you sent your email successfully, it may never make it to your "Sent" folder - the <u>second</u> connection to your IMAP server is quite unrelated to the first connection to the '''SMTP''' server.* '''DNS''' Server. A DNS server is also involved - it is needed to retrieve the address of the email server responsible for email for a particular domain. This is done with '''MX''' records. ===Online References=== * [https://help.ubuntu.com/community/Dovecot Dovecot Community Documentation]* [http://wiki.dovecot.org/LDA Dovecot-lda]* [Categoryhttp://wiki.dovecot.org/LDA/Postfix Configuring dovecot-lda with postfix] == INVESTIGATION 1:OPS335INSTALL THUNDERBIRD (MUA) and SETUP A REFERENCE CLIENT== Unlike the '''mailx''' (MUA) application you installed and used in Lab 4a, this lab will be using the '''Thunderbird''' (MUA) application instead which is a graphical application that uses a '''centralized Message Store''' (MS) to retrieve and read mail messages.  Although we will be eventually setting up the Thunderbird application to perform all the mail operations discussed above, you need to learn to '''"walk before you can run"'''. Eventually, you are going to set up all those mail services, but to begin with, you will set up an email client to connect to an already working server which is the '''Seneca email server'''. Once we learn how to do this for our Seneca email account, then we can use it for our mail servers for our VM2 and VM3. '''Perform the following steps:''' #Switch to your '''host''' machine, and install the '''Thunderbird''' email application.#When you first launch the Thunderbird application, a configuration dialog should appear as shown in the diagram below: <br>::[[Image:Seneca-student-thunderbird-email-setup.png|600px]]<br><ol><li value="3">Use the data in the table below to configure the Thunderbird settings dialog box for YOUR Seneca e-mail account:</li></ol> {| class="wikitable" border="1" style="margin-left:40px;"! Setting !! '''Incoming: IMAP''' !! '''Outgoing: SMTP'''|-| '''Username'''|| yoursenecauserid@myseneca.ca || yoursenecauserid@myseneca.ca|-| '''servername''' || outlook.office365.com || outlook.office365.com|-| '''port''' || 993 || 587|-| '''security''' || SSL/TLS || STARTTLS|-| '''References''' | colspan="2" | [1] [Categoryhttps://employees.senecacollege.ca/spaces/77/it-services/wiki/view/2394/other-email-clients ITS - Configuring other Email Clients]|} ::Note that your username is your full email address(<em>yourid@myseneca.ca</em>) and not just <em>yourid</em>.  {{Admon/important |Unencrypted Options|Notice that there are <u>unencrypted</u> options available to connect to your SMTP/IMAP servers but those are rarely used these days - the potential for abuse is too great. On a free wifi network, the operator would be able to not only read your email, but also obtain your password without any password/encryption cracking tools. In fact, even on a private wired network, it is not uncommon for an employer to use a packet sniffer utility to monitor all the traffic going over their network (Packet Sniffing applications were actually found to be legally acceptable practice if used by the management of organizations)}}<br><ol><li value="3">After you create your '''Thunderbird''' account, you should be able to read your existing email and send new email within the Thunderbird application.</li><li>Take time to view your ''Account Settings'' and ''Preferences'' to get a feel for what settings exist. For example:<ul><li>How often will Thunderbird check for new messages?</li><li>Will the messages you write be in HTML or plain text?</li><li>How do you change your SMTP server settings? Why are they in a different section?</li></ul></li><li>The main objective of this section was to learn how to setup your Thunderbird application to read your Seneca email, so in the next section you can use the exact type of setup for your own email server.</li></ol> '''Record steps, commands, and your observations in INVESTIGATION 1 in your OPS335 Labslab log-book''' == INVESTIGATION 2: SETUP A CENTRALIZED MESSAGE STORE == === Setup Your MTA to Use Correct Domain=== In Lab 4a, both of your email servers were sending mail messages addressed from users of the actual machines themselves. This would be confusing for the receiver who might get emails from the same user @vm1, vm2, and vm3. Which would they respond to? To avoid this problem from occurring, we can make all servers make the sent mail appear to come from a central location (usually the '''domain'''), and make incoming email sent to that address to be accessible from machines within our network. '''Perform the following steps:''' # Issue the '''mail''' command to view the email messages you sent between your '''vm2''' and '''vm3''' in your lab 4a. Notice that each is addressed from root on whichever machine sent it.# On both machines (vm2 and vm3), edit the '''/etc/postfix/main.cf''' file to change the '''myorigin''' parameter from '''$myhostname''' to '''$mydomain'''. Restart the '''postfix''' service.# Now, send emails messages (via the '''mail''' command) between both of your vm2 and vm3 machines, and view the mail messages by issuing '''mail''' in each vm. The sender address should now read that the received mail messages came from '''root@yourdomain.ops'''.  ::The next step is to configure what addresses that the server will receive email for. This is done using postfix by setting the '''mydestination''' parameter (configuration variable) to include '''$mydomain''' (this is assuming you've set up '''mydomain''', '''myorigin''' , and '''inet_interfaces''' properly).  <ol><li value="4">Edit the '''/etc/postfix/main.cf''' file for '''vm3 ONLY''', scroll down to the line containing: '''mydestination''' and change line to the text shown below:<br><source>mydestination = $mydomain, $myhostname, localhost.$mydomain, localhost</source>'''Note:''' Even though your machine's name is ''vm3.yoursenecaid.ops'', your postfix MTA will also receive emails addressed to the domain called: yoursenecaid.ops</li></ol><br>::In order for this to work, we need to add a DNS record that will point mail sent to the domain towards one of the SMTP servers configured to accept it. <ol><li value="5">Add an '''MX''' record to the forward lookup zone on '''host''' so that all incoming mail addressed to the domain is sent to your vm3.</li><li>Restart the service and use the '''dig''' command to confirm that it works.</li><li>Send an email from your '''vm2''' to '''root@yourdomain.ops'''</li><li>Confirm that it arrives on your '''vm3''' machine</li></ol> === Relay Email Through Another Server=== When email is sent from either vm, it is addressed from the domain, but receiving MTAs might query why mail sent from vm2 doesn't match the address of the MX record for the domain. This would be a red-flag for potential spam. To avoid this, we can relay all mail sent from vm2 (or any other machine in our network) through vm3 so that it properly appears to come from the mail server that matches the MX record for the domain. '''Perform the following steps:''' # Move to your vm2 machine.# Direct your '''vm2''' MTA to relay mail through vm3, by making the following editing change for the '''/etc/postfix/main.cf''' file:<br><source>relayhost = vm3.<yourdomain>.ops</source># Restart the '''postfix''' service.# Next, you must instruct your '''vm3''' machine to allow your vm2 machine to pass email through it by making the following editing change to the '''/etc/postfix/main.cf''' file:<br><source>mynetworks = 192.168.X.0/24</source>NOTE: Substitute in your '''own network''' for X<br><br># Restart the '''postfix''' service. All mail is now being delivered to a centralized location (and also appears to be coming from that same location), but a user would still have to access that server to retrieve it. === Install and Configure the Local Delivery Agent (LDA/MDA) === Postfix is capable of performing the function of an LDA, but its LDA capabilities are limited, thus postfix is generally not used for that purpose. Currently, the most popular LDA is ''LMTP'', but we will be installing, configuring, and using an LDA called '''Dovecot''' since it is also popular and we will setting up Dovecot as an '''IMAP''' server later in this lab. Using both Postfix and Dovecot will actually increase the performance of our IMAP server. '''Perform the following steps:''' #Move to your '''vm3''' machine.#Dovecot is not installed when you installed your Virtual machines in previous labs.<br>Install the Dovecot application by issuing the following command:<br><source>yum install dovecot</source>#Edit your '''/etc/postfix/main.cf''' file and scroll down to (or search for) '''mailbox_command'''. Add the following line:<br><source>mailbox_command = /usr/libexec/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"</source> ::NOTE: Do <u>'''not'''</u> replace any variables, those are set automatically by Postfix when it runs the LDA. If you are interested in learning more about the Dovecot application, you can read about dovecot-lda [http://wiki.dovecot.org/LDA/Postfix here]and [http://wiki.dovecot.org/LDA here].<br><ol><li value="4">Finally, edit the '''/etc/dovecot/conf.d/10-mail.conf''' file and indicate where you want your mail delivered by including the following line:<source>mail_location = maildir:~/Maildir</source></li><li>Restart your postfix service.</li><li>While the emails are still stored only on VM3, they will now be easier for other machines/services to access.</li><li>Due to permissions on the directories where mail will now be stored, root will no longer receive mail. Check the logs for an indication as to why.</li></ol>

The diagram above doesn't even include reading email, but it's as simple as it gets (if you want to run an email server). You will need to learn to administer basics of all the systems in the diagram. We won't go in too much depthRecord steps, only the minimum you need. But we will not (for example) go over every detail of Postfixcommands, which on its own has a rather [https://en.wikipedia.org/wiki/Postfix_%28software%29#Architecture complex] [https://www.credativ.de/blog/postfixand your observations in INVESTIGATION 2 in your OPS335 lab log-architecture-overview diagram].book'''

=== Services involved in email delivery =INVESTIGATION 3: USING THUNDERBIRD (MUA) FOR VM2 and VM3 MACHINES ==

[http://wikiFirst, we will set up the IMAP server so we can read email.dovecotThe current way we have configured our mail server on our VM3 machine should allow all the email for anyaccount@yoursenecaid.org/MailServerOverview Hereops should be delivered to our '''vm3''s an overview] of from the ' machine. We will set up Dovecot wiki, it's worth readingwith IMAP to get easy access to that email.

In our diagram we have'''Perform the following steps:'''

* A user. That's #The configuration file for the person who wants to send an email.* An MUA Dovecot service (email clientwhich is not the same thing as dovecot-lda). This is the application the user uses to send an email: '''/etc/dovecot/dovecot. It can be a native application or a web applicationconf'''. WeModify the 'll set up both types''protocols''' option so that Dovecot will work with IMAP connections, no POP3 or LMTP.* Two MTAs. These are # Start the servers responsible for getting your emails to dovecot service, and ensure it will always start automatically when the destination servermachine boots.** They are similar # Use the '''ss''' command to routers (whcih route packets) but work on confirm the application layer rather than the network layer.** In our example there are only two MTAs - but there can be several.** You connect to your MTA over a secure connectionservice is listening, so your emails canand use '''nc'''t be read by on the operators of the network '''host''' to confirm you're connected can connect toit.** The rest of the way to the destination MTA the emails travel completely unencrypted# You'll probably fail, so anyone with access to using the routers in between can read all your emails. This is why many organizations will refuse to send you confidential information over email.* The LDA/MDA will receive the email gathered from '''ss''', modify the MTA, and will store it firewall on disk in some format. MailDir vm3 to allow IMAP connections from your local network and MBOX are the most popular mailbox formatstry '''nc''' again.* When sending an email you send Once it to the destination using your MTAworks, but you also want do not forget to save this change so it in your "Sent" folder for yourself. This is accomplished by a separate connection to your IMAP or POP3 serverwill still be there the next time you reboot.** This is why it #If you can happen that you sent your email successfully but it never makes connect - it 's now time to your "Sent" folder - the second connection do something wrong, that is allow connections to your our IMAP server is quite unrelated to over an unencrypted connection.# Edit the first connection '''/etc/dovecot/conf.d/10-auth.conf''' file and set '''disable_plaintext_auth''' to '''no'''.# Then edit the SMTP server'''/etc/dovecot/conf.* Note that a DNS server is also involved d/10- itssl.conf''' file and set '''ssl'''s needed to retrieve the address '''yes'''.<br><br>'''Note:''' This combination of parameters will allow your username and password to be sent over the email server responsible internet in plain text, for email anyone interested to look at. In a later lab we'll set up secure SMTP and IMAP connections, for a particular domain. This now this is done with the MX records all we looked at in have time for.<br><br># Restart dovecot so the DNS labschanges take effect.

== Sample client setup = Connecting to IMAP Servers Using Thunderbird===

Eventually we're going to set up all those services, but to begin with we'll set up an email client to connect to a (hopefully) working server - 'Perform the Seneca email server. This will be a good exercise with an email client.following steps:'''

Install Thunderbird on #On your '''host''' machine, return to the Mail Account Setup dialog box (eg. near top of lab).# Set up a '''new email account'''. You will be using account settings to connect to your '''vm2''' for '''SMTP''' and configure it like this'''vm3''' for '''IMAP'''. Use <u>no</u> encryption, obviously using your own informationand use normal password authentication for IMAP. Refer to the diagram below for reference:

[[Image:Seneca-student-thunderbirdops335-email-setupstep1.png|600px]]

Notice that there are unencrypted options available <ol><li value="3">Try to connect to your SMTP/IMAP servers but those are rarely used these daysserver with Thunderbird by clicking on your '''Inbox'''. The potential </li><li>If nothing happens, then check the Thunderbird Activity Manager for abuse any errors. If the connection is too greatsuccessful, you should see the '''Trash''' box <u>appear</u> below Inbox. On a free wifi network </li><li>Use the operator would be able Thunderbird application to not only read your send an email - but also get to your passwordmyseneca address. If you've done everything right, without any passwordit will send the message successfully</encryption cracking toolsli><li>Verify that your message has been sent. And even Check your myseneca email and look at '''/var/log/maillog''' on a private network - it is not uncommon for an employer to sniff all the traffic going over their network vm2 (that was found in court to be a legally acceptable practiceyour email server).</li></ol>

Look through #Use the Account Settings '''ss''' and Preferences '''nc''' commands (like you did in lab 4a) to get a feel for what settings existconfirm your service is listening on the correct ports/interfaces. For exampleYou will probably have to open the appropriate firewall port on '''vm3''' to allow incoming '''SMTP''' connections.<br><br>'''Note:* How often will ''' You should be able to send email to any regular user <u>on</u> '''vm3''' using the email address '''yourusername@yoursenecaid.ops''' using the Thunderbird check for new messages?* Will application on your host machine (which is configured to use the messages you write be in HTML or plain text?* How do you change account on your SMTP server settings? Why are they in a different section?vm2).<br><br>

If everything <ol><li value="2">Create a new account on your '''vm3''' machine using only your <u>first</u> name. We will use this account as a one-time "test" if the mail message has been received on your VM3 machine (from your VM2 machine).<br><br>'''Note:''' It is working - <u>'''important'''</u> thatyou '''<u>don't</u>'s good'' create this same account name on your vm2 machine, now you know what since you'll try want to build in easily identify the email labs. The goal is to have difference between the exact same setup using your servers instead of Seneca sending and receiving SMTP servers.<br /><br /></li></ol>

<ol><li value= MTA for Sending "3">Use the new account in Thunderbird to send an email to '''firstname@yoursenecaid.ops''' and then check the contents of '''/home/firstname/Maildir/new/''' on your '''vm3''' machine. There should be a file there with the contents of your email.</li><li>If there is no encryption =file, then check the log file '''/var/log/maillog''' to see what went wrong.</li><li>If you can see a file in the '''/home/firstname/Maildir/new/''' directory, then review the procedures on how you got the email server working (since you have performed many steps and set up many services).</li><li>Refer to the diagram at the top of this lab. Which services have you currently set up? Record your findings in your lab Logbook.</li></ol>

We'll use Postfix as {{Admon/important |Encountering error messages when sending email|If you cannot properly receive sent e-mail messages, check the MTA, and we'll set it up on ''/var/log/syslog'vm2'' file for errors.<br><br> If you locate an error message in that file such as: '''Fatal: Error reading configuration: Invalid settings. This will be ..''', then add the email server for your internal networkfollowing <u>parameter</u> in '''/etc/dovecot/dovecot. Youconf'''ll be able to send email out of your network, and receive email from within your network, but not receive from outside your network because:# Outsiders will never find the MX recors for <br />'''postmaster_address <nowiki>=</nowiki> DOMAIN''' (where DOMAIN is actually <u>your </u> domain, because there are no ).org servers pointing to your DNS server (<br /><br />After you havenhave saved those changes, then '''restart'''t paid for it)your dovecot service.# Even if they did - This problem can also be resolved by properly setting the hostname of your local network is using IP addresses on a private subnet, which is not routeable on machine to include the internet, so it cannot be reached from the outsidedomain.}}

Postfix {{Admon/important |Backup your VMs!|You MUST perform a '''full backup''' of ALL of your VMs whenever you complete your '''OPS335 labs''' or when working on your '''OPS335 assignments'''. You should be installed by default. If it isn't - go ahead using the dump or rsync command, and install it. Install also you should use the netstat application (use yum search Bash shell script that you were adviced to create in order to find the package name) and the telnet commandbackup all of your VMs.}}

Postfix will work with the default configuration'''Record steps, so enable it, start itcommands, and check that ityour observations in INVESTIGATION 3 in your OPS335 lab log-book'''s running. Also look for it in the list of listening ports:

<source lang="bash">netstat -atnp</source>= COMPLETING THE LAB ==

Which one is it? Find ===Online Submission===Follow the port used by SMTP, and look instructions for connctions with the state LISTEN (i.e. currently listening)lab 4b on blackboard.<!--===Andrew's sections===

Connect You'll get the same grade regardless of how you choose to submit your server using telnet:work.

::<source langspan style="color:green;font-size:1.5em;">&#x2713;</span>Status and configuration of your Postfix service on vm2.::<span style="color:green;font-size:1.5em;">&#x2713;</span>Proof that you can connect to that service from the host.::<span style="color:green;font-size:1.5em;">&#x2713;</span>Your Thunderbird configuration.::<span style="color:green;font-size:1.5em;">&#x2713;</span>The email you sent to your myseneca account.::<span style="color:green;font-size:1.5em;">&#x2713;</span>Download and run '''wget https://ict.senecacollege.ca/~andrew.smith/ops335/labcheck4b.bash''' on your '''c7host''' machine. ::<span style="color:green;font-size:1.5em;">telnet localhost 25&#x2713;</sourcespan>Completed Lab4b log-book notes.-->

If this worked # What is the purpose of the Thunderbird application?# List the steps to configure your DNS to allow your Thunderbird application to connect to your mail server.# What is the purpose of the '''Dovecot''' package?# What is the purpose of the '''mydestination''' parameter contained in the '''/etc/postfix/main.cf''' file?# Why are '''IMAP''' and '''POP''' email servers placed on separate machines (vms)?# What is the purpose of the '''mail_location''' parameter contained in the '''/etc/dovecot/conf.d/10- that means yourmail.conf''' file?# Why is root not able to receive mail with the changed mail location? What could you change to allow mail to be sent to root again?