Changes

= Email Servers = You may not be aware of it as an user, but email is a very <u>complex</u> system to administer. In fact, the more modern e-mail systems (eg. web-based mail applications, etc) are more technically involved than the other archaic, hard-to-configure, and sometimes inter-operable mail systems. We are going to spread the remaining email labs over a few weeks, so that by the end of this topic, you will have a sufficient understanding of what services are involved in sending, filtering, and reading email. You will also have the skills to configure a basic mail setup using the default services provided for your Centos7 Linux distribution.[[Category:OPS335]][[Category:OPS335 Labs]]

==LAB RESOURCESOVERVIEW=={{Admon/important|Warning|Your lab 4a must be complete before you can start this lab.}}

That setup has some major drawbacks::* It required an SMTP server ('''MTA''') to be configured on each machine.:* The Message Store ('''MS''') would also be unique to each machine - what a user received on one server would not exist on any other.

In factFinally, the above diagram does not include reading mail messagesyou will set up an '''IMAP''' server called '''Dovecot''' on your '''vm3''' machine, but this acts so you can read your email from an MUA such as ''Thunderbird'' or a starting point in order to run a basic email server''Webmail'' application. Although You will be learning to administer the mail services set up a webmail application called '''Roundcube''' in the diagram above, we will not required you to go into tremendous depth (just the minimum requirementsa later lab). For example, we will not go over every aspect of the Postfix MTA service, but you should know what it represents and what is its main purpose, as opposed to the following: [https://en.wikipedia.org/wiki/Postfix_%28software%29#Architecture complex diagram 1] , [https://www.credativ.de/blog/postfix-architecture-overview complex diagram 2].

=== Learning About the Services involved Involved in email delivery an Email Delivery ===

In reality, the terms '''MTA''', '''MDA''', '''MUA''', '''LDA''' can actually be considered misleading since some of those services can be combined together to form a single entity (application), while other applications may operate as separate entities. There may be overlap, so if you don't find those acronyms helpful, don't worry too much about them. On the other hand, when referred to in diagrams, they can help to visualize those processes when try trying to understand how an e-mail system works.

[http://wiki.dovecot.org/MailServerOverview Here's is an overview] of those terms (from the Dovecot wiki). It is worth viewing this link.

* A '''userUser Account'''. That's the person The individual who wants to send an emailor receive mail messages.* An '''MUA''' (email client). This is the application that the user individual uses to send an emailor receive mail messages. It can be a '''native application ''' or a '''web application'''. We'll set up You will learn how to setup and use both typesof these applications throughout the remainder of this course.* Two '''MTAsMTA'''servers. These are the servers responsible for getting your emails to the <u>destination</u> server.

* The '''LDA/MDA''' Server. This server will receive the email from the MTA, and will store it on disk in some format. '''MailDir''' and '''MBOX''' are the most popular mailbox formats.* '''IMAP/POP3''' server(s). When sending an email, you send it to the destination using your MTA, but you also want to save it in your '''"Sent"''' folder for yourself. This is accomplished by a separate connection to either your '''IMAP''' or '''POP3''' server.

* Note that a '''DNS''' Server. A DNS server is also involved - it is needed to retrieve the address of the email server responsible for an email for a particular domain. This is done with the '''MX ''' records we looked at in the DNS labs.

=== Install Thunderbird Application and Setup a Reference Client Online References===

Eventually we're going to set up all those mail services, but to begin with, we will set up an email client to connect to a (hopefully) working server * [https://help.ubuntu.com/community/Dovecot Dovecot Community Documentation]* [http://wiki.dovecot.org/LDA Dovecot- the Seneca email serverlda]* [http://wiki.dovecot. This will be a good exercise org/LDA/Postfix Configuring dovecot-lda with an email client.postfix]

Install Thunderbird on your host, == INVESTIGATION 1: INSTALL THUNDERBIRD (MUA) and configure it like this, obviously using your own information:SETUP A REFERENCE CLIENT==

[[Image:Seneca-student-thunderbird-email-setupUnlike the '''mailx''' (MUA) application you installed and used in Lab 4a, this lab will be using the '''Thunderbird''' (MUA) application instead which is a graphical application that uses a '''centralized Message Store''' (MS) to retrieve and read mail messages.png|600px]]

Notice that there Although we will be eventually setting up the Thunderbird application to perform all the mail operations discussed above, you need to learn to '''"walk before you can run"'''. Eventually, you are unencrypted options available going to set up all those mail services, but to begin with, you will set up an email client to connect to your SMTP/IMAP servers but those are rarely used these days. The potential for abuse an already working server which is too greatthe '''Seneca email server'''. On a free wifi network the operator would be able Once we learn how to not only read your do this for our Seneca email - but also get your passwordaccount, without any password/encryption cracking tools. And even on a private network - then we can use it is not uncommon for an employer to sniff all the traffic going over their network (that was found in court to be a legally acceptable practice)our mail servers for our VM2 and VM3.

The specific security settings depend on how your servers were configured. The settings for '''Perform the seneca servers are [httpsfollowing steps://inside.senecacollege.ca/its/services/email/email_clients/imap.html published here].'''

After you create your account - you should be able #Switch to read your existing email '''host''' machine, and send new install the '''Thunderbird''' email application.#When you first launch the Thunderbird application, a configuration dialog should appear as shown in Thunderbird.the diagram below:

Look through the Account Settings and Preferences to get a feel for what settings exist<br>::[[Image:Seneca-student-thunderbird-email-setup. For example:png|600px]]* How often will Thunderbird check for new messages?<br>* Will <ol><li value="3">Use the messages you write be data in HTML or plain text?* How do you change your SMTP server the table below to configure the Thunderbird settings? Why are they in a different section?dialog box for YOUR Seneca e-mail account:</li></ol>

If everything is working {| class="wikitable" border="1" style="margin- thatleft:40px;"! Setting !! 's good, now you know what you'll try to build in the email labs'Incoming: IMAP''' !! '''Outgoing: SMTP'''|-| '''Username'''|| yoursenecauserid@myseneca.ca || yoursenecauserid@myseneca.ca|-| '''servername''' || outlook.office365.com || outlook.office365.com|-| '''port''' || 993 || 587|-| '''security''' || SSL/TLS || STARTTLS|-| '''References''' | colspan="2" | [1] [https://employees. The goal is to have the exact same setup using your servers instead of Seneca serverssenecacollege.ca/spaces/77/it-services/wiki/view/2394/other-email-clients ITS - Configuring other Email Clients]|}

== INVESTIGATION 1: SETUP MTA FOR SENDING :Note that your username is your full email address(NO ENCRYPTION<em>yourid@myseneca.ca</em>) ==and not just <em>yourid</em>.

{{Admon/important |Unencrypted Options|Notice that there are <u>unencrypted</u> options available to connect to your SMTP/IMAP servers but those are rarely used these days - the potential for abuse is too great. On a free wifi network, the operator would be able to not only read your email, but also obtain your password without any password/encryption cracking tools. In fact, even on a private wired network, it is not uncommon for an employer to use a packet sniffer utility to monitor all the traffic going over their network (Packet Sniffing applications were actually found to be legally acceptable practice if used by the management of organizations)}}<br><ol><li value=== Verify Postfix Service Status ==="3">After you create your '''Thunderbird''' account, you should be able to read your existing email and send new email within the Thunderbird application.</li><li>Take time to view your ''Account Settings'' and ''Preferences'' to get a feel for what settings exist. For example:<ul><li>How often will Thunderbird check for new messages?</li><li>Will the messages you write be in HTML or plain text?</li><li>How do you change your SMTP server settings? Why are they in a different section?</li></ul></li><li>The main objective of this section was to learn how to setup your Thunderbird application to read your Seneca email, so in the next section you can use the exact type of setup for your own email server.</li></ol>

Postfix should be installed by default. If it isn't ''Record steps, commands, and your observations in INVESTIGATION 1 in your OPS335 lab log- go ahead and install it. Install also the netstat application (use yum search to find the package name) and the telnet command.book'''

Postfix will work with the default configuration, so enable it, start it, and check that it's running. Also look for it in the list of listening ports== INVESTIGATION 2:SETUP A CENTRALIZED MESSAGE STORE ==

<source lang="bash">netstat -atnp</source>== Setup Your MTA to Use Correct Domain===

In Lab 4a, both of your email servers were sending mail messages addressed from users of the actual machines themselves. This would be confusing for the receiver who might get emails from the same user @vm1, vm2, and vm3. Which one is itwould they respond to? Find the port used by SMTP To avoid this problem from occurring, and look for connctions with we can make all servers make the state LISTEN sent mail appear to come from a central location (i.e. currently listeningusually the '''domain'''), and make incoming email sent to that address to be accessible from machines within our network.

=== Testing '''Perform the connection to Postfix ===following steps:'''

Connect # Issue the '''mail''' command to view the email messages you sent between your '''vm2''' and '''vm3''' in your lab 4a. Notice that each is addressed from your server root on whichever machine sent it.# On both machines (vm2 and vm3), edit the '''/etc/postfix/main.cf''' file to change the '''myorigin''' parameter from '''$myhostname''' to '''$mydomain'''. Restart the '''postfix''' service.# Now, send emails messages (via the '''mail''' command) between both of your server using telnet:vm2 and vm3 machines, and view the mail messages by issuing '''mail''' in each vm. The sender address should now read that the received mail messages came from '''root@yourdomain.ops'''.

Note ::The next step is to configure what addresses that it the server will tell receive email for. This is done using postfix by setting the '''mydestination''' parameter (configuration variable) to include '''$mydomain''' (this is assuming you once your connect that 've set up '''mydomain''', ''Escape character is '^]myorigin''', which means your can hold control and press the square bracket key to end the session (and then exit the telnet app'''inet_interfaces''' properly).

Once you open <ol><li value="4">Edit the port in '''/etc/postfix/main.cf''' file for '''vm3 ONLY''', scroll down to the firewall - try line containing: '''mydestination''' and change line to the telnet commandtext shown below:<br><source>mydestination = $mydomain, $myhostname, localhost.$mydomain, localhost</source>'''Note:''' Even though your machine's name is ''vm3. You should get a different error this timeyoursenecaid. This time ops'', your postfix MTA will also receive emails addressed to the problem is domain called: yoursenecaid.ops</li></ol><br>::In order for this to work, we need to add a DNS record that your service isn't listening on will point mail sent to the domain towards one of the outside interface, it's currently SMTP servers configured to listen only on the loopback (lo) interfaceaccept it.

<ol><li value=== Listening "5">Add an '''MX''' record to the forward lookup zone on '''host''' so that all interfaces ===incoming mail addressed to the domain is sent to your vm3.</li><li>Restart the service and use the '''dig''' command to confirm that it works.</li><li>Send an email from your '''vm2''' to '''root@yourdomain.ops'''</li><li>Confirm that it arrives on your '''vm3''' machine</li></ol>

The configuration file When email is sent from either vm, it is addressed from the domain, but receiving MTAs might query why mail sent from vm2 doesn'''/etc/postfix/maint match the address of the MX record for the domain. This would be a red-flag for potential spam.cf''' Edit To avoid this, we can relay all mail sent from vm2 (or any other machine in our network) through vm3 so that it and change '''inet_interfaces''' properly appears to '''all'''come from the mail server that matches the MX record for the domain.

At this point we should also set the string that will end up in the '''FromPerform the following steps:''' header in messages sent by this server. Change '''mydomain''' to your domain name and and '''myorigin''' to '''$mydomain'''.

# Move to your vm2 machine.# Direct your '''vm2''' MTA to relay mail through vm3, by making the following editing change for the '''/etc/postfix/main.cf''' file:<br><source>relayhost = vm3.<yourdomain>.ops</source># Restart the '''postfix''' service.# Next, confirm (with netstat) that ityou must instruct your '''vm3'''s now listening on all interfaces, not just loopback, and test connecting machine to allow your vm2 machine to pass email through it from by making the following editing change to the host'''/etc/postfix/main.cf''' file:<br><source>mynetworks = 192.168.X.0/24</source>NOTE: Substitute in your '''own network''' for X<br><br># Restart the '''postfix''' service.

So far you're probably not using vm1 as === Install and Configure the DNS server for your host. We'll need to change that for email to work correctly. Normally you don't need an entire DNS server for running an email server, you just add an MX record in your registrar's web interface, but we haven't paid for our domains so we don't have a registrar with a web interface.Local Delivery Agent (LDA/MDA) ===

Modify your /etc/resolv.conf to use vm1 as Postfix is capable of performing the name server. This will function of an LDA, but its LDA capabilities are limited, thus postfix is generally not be a permanent change but used for thatpurpose. Currently, the most popular LDA is ''LMTP''s ok, but we only need will be installing, configuring, and using an LDA called '''Dovecot''' since it for the email labs. Youis also popular and we will setting up Dovecot as an '''IMAP'''ll have to remember to do server later in this every time you reboot your hostlab. Using both Postfix and Dovecot will actually increase the performance of our IMAP server.

If you regular (internet) DNS lookups slow down significantly - try to update '''Perform the DNS server configuration on vm1 to forward requests to the Google nameserver (8.8.8.8) instead of your host.following steps:'''

==INVESTIGATION 2#Move to your '''vm3''' machine.#Dovecot is not installed when you installed your Virtual machines in previous labs.<br>Install the Dovecot application by issuing the following command:<br><source>yum install dovecot</source>#Edit your '''/etc/postfix/main.cf''' file and scroll down to (or search for) '''mailbox_command'''. Add the following line: SETUP THUNDERBIRD MUA =<br><source>mailbox_command =/usr/libexec/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"</source>

We ::NOTE: Do <u>'''not'''</u> replace any variables, those are far from having a working email server but at this point we have enough set automatically by Postfix when it runs the LDA. If you are interested in learning more about the Dovecot application, you can read about dovecot-lda [http://wiki.dovecot.org/LDA/Postfix here] and [http://wiki.dovecot.org/LDA here].<br><ol><li value="4">Finally, edit the '''/etc/dovecot/conf.d/10-mail.conf''' file and indicate where you want your mail delivered by including the following line:<source>mail_location = maildir:~/Maildir</source></li><li>Restart your postfix service.</li><li>While the emails are still stored only on VM3, they will now be easier for other machines/services to access.</li><li>Due to permissions on the directories where mail will now be able stored, root will no longer receive mail. Check the logs for an indication as to test it with Thunderbirdwhy.</li></ol>

'''Record steps, commands, and your observations in INVESTIGATION 2 in your OPS335 lab log-book''' ==INVESTIGATION 3: USING THUNDERBIRD (MUA) FOR VM2 and VM3 MACHINES == === Accessing Received Mail Messages on VM3 VIA IMAP === First, we will set up the IMAP server so we can read email. The current way we have configured our mail server on our VM3 machine should allow all the email for anyaccount@yoursenecaid.ops should be delivered to our '''vm3''' machine. We will set up Dovecot with IMAP to get easy access to that email. '''Perform the following steps:''' #The process configuration file for the Dovecot service (which is not the same thing as dovecot-lda) is: '''/etc/dovecot/dovecot.conf'''. Modify the '''protocols''' option so that Dovecot will work with IMAP connections, no POP3 or LMTP.# Start the dovecot service, and ensure it will always start automatically when the machine boots.# Use the '''ss''' command to confirm the service is a little challenging because Thunderbird tries really hard listening, and use '''nc''' on the '''host''' to prevent confirm you can connect to it.# You'll probably fail, so using the information gathered from '''ss''', modify the firewall on vm3 to allow IMAP connections from connecting your local network and try '''nc''' again. Once it works, do not forget to save this change so it will still be there the next time you reboot.#If you can connect - it's now time to a do something wrong, that is allow connections to our IMAP server that doesnover an unencrypted connection.# Edit the '''/etc/dovecot/conf.d/10-auth.conf''' file and set '''disable_plaintext_auth''' to '''no'''.# Then edit the '''/etc/dovecot/conf.d/10-ssl.conf'''t work (file and ours mostly doesnset '''ssl't work '' to '''yes'''.<br><br>'''Note:''' This combination of parameters will allow your username and password to be sent over the internet in plain text, for anyone interested to look at . In a later lab we'll set up secure SMTP and IMAP connections, for now this pointis all we have time for.<br><br># Restart dovecot so the changes take effect. === Connecting to IMAP Servers Using Thunderbird=== '''Perform the following steps:''' #On your '''host''' machine, return to the Mail Account Setup dialog box (eg. near top of lab). # Set it up a '''new email account'''. You will be using account settings to connect to your '''vm2''' for '''SMTP ''' and '''vm3''' for '''IMAP'''. Use <u>no </u> encryption, and use normal password authentication for IMAP (we don't have an IMAP server running yet, but that's ok).Refer to the diagram below for reference:

<ol><li value="3">Try to connect to your IMAP server with Thunderbird wonby clicking on your '''Inbox't let ''.</li><li>If nothing happens, then check the Thunderbird Activity Manager for any errors. If the connection is successful, you should see the '''Trash''' box <u>appear</u> below Inbox.</li><li>Use the Thunderbird application to send an email to your myseneca address. If you proceed with 've done everything right, it will send the message successfully</li><li>Verify that your message has been sent. Check your myseneca email and look at '''/var/log/maillog''' on vm2 (your email server).</li></ol> === Sending a Mail Message from VM2 (Using Thunderbird)=== '''Perform the following steps:''' #Use the "Done" button because '''ss''' and '''nc''' commands (like you did in lab 4a) to confirm your service is listening on the correct ports/interfaces. You will fail probably have to connect open the appropriate firewall port on '''vm3''' to allow incoming '''SMTP''' connections.<br><br>'''Note:''' You should be able to send email to any regular user <u>on</u> '''vm3''' using the email address '''yourusername@yoursenecaid.ops''' using the Thunderbird application on your host machine (which is configured to IMAPuse the account on your vm2).<br><br> <ol><li value="2">Create a new account on your '''vm3''' machine using only your <u>first</u> name. We will use this account as a one-time "test" if the mail message has been received on your VM3 machine (from your VM2 machine). Use <br><br>'''Note:''' It is <u>'''important'''</u> that you '''<u>don't</u>''' create this same account name on your vm2 machine, since you want to easily identify the difference between the sending and receiving SMTP servers.<br /><br /></li></ol> <ol><li value="Advanced config3" button >Use the new account in Thunderbird to send an email to bypass that '''firstname@yoursenecaid.ops''' and then check the contents of '''/home/firstname/Maildir/new/''' on your '''vm3''' machine. There should be a file there with the contents of your email.</li><li>If there is no file, then checkthe log file '''/var/log/maillog''' to see what went wrong.</li><li>If you can see a file in the '''/home/firstname/Maildir/new/''' directory, then review the procedures on how you got the email server working (since you have performed many steps and set up many services).</li><li>Refer to the diagram at the top of this lab. Which services have you currently set up? Record your findings in your lab Logbook.</li></ol>

Now use Thunderbird to send an {{Admon/important |Encountering error messages when sending email to your myseneca address. |If you've done everything right cannot properly receive sent e- it will send mail messages, check the '''/var/log/syslog''' file for errors.<br><br> If you locate an error message successfully but it will fail to save it in that file such as: '''Fatal: Error reading configuration: Invalid settings...''', then add the Sent folderfollowing <u>parameter</u> in '''/etc/dovecot/dovecot.conf''':<br />'''postmaster_address <nowiki>=</nowiki> DOMAIN''' (where DOMAIN is actually <u>your</u> domain).<br /><br />After you have saved those changes, because thatthen '''restart''s done with IMAP and you don't have an IMAP server yetyour dovecot service. This problem can also be resolved by properly setting the hostname of your machine to include the domain.}}

Still - verify that {{Admon/important |Backup your VMs!|You MUST perform a '''full backup''' of ALL of your message has been sent. Check VMs whenever you complete your myseneca email and look at '''/var/log/maillogOPS335 labs''' or when working on vm2 (your email server)'''OPS335 assignments'''.You should be using the dump or rsync command, and you should use the Bash shell script that you were adviced to create in order to backup all of your VMs.}} '''Record steps, commands, and your observations in INVESTIGATION 3 in your OPS335 lab log-book'''

Students should be prepared with '''all required commands (system information) displayed in a terminal (or multiple terminals) prior to calling ===Online Submission===Follow the instructor instructions for signofflab 4b on blackboard.<!--===Andrew'''.s sections===

'''Arrange evidence (command output) for each You may choose to:* Submit screenshots of these items your work on your screenBlackboard, then ask your instructor in which case you don't need to come to the lab.* Or come to review them and sign off on the lab, show me your work, and talk to me about it. I want to hear what you's completion:''ve learned and answer any questions you have. You'll get the same grade regardless of how you choose to submit your work.

#What is the purpose of the Thunderbird application?# List the steps to configure your DNS to allow your Thunderbird application to connect to your mail server.# What is the purpose of the '''Dovecot''' package?# What is the purpose of the '''mydestination''' parameter contained in the '''/etc/postfix/main.cf''' file?# Why are '''IMAP''' and '''POP''' email servers placed on separate machines (vms)?# What is the purpose of the '''mail_location''' parameter contained in the '''/etc/dovecot/conf.d/10-mail.conf''' file?# Why is root not able to receive mail with the changed mail location? What could you change to allow mail to be sent to root again?