1,576
edits
Changes
m
<a name="Installation Requirements" id="Fedora_16_Installation_.28on_Main_Host_-_f16host.29"></a><h1> <span class="mw-headline">Information Gathering</span></h1><a name="Introduction" id="Introduction"></a><h2> <span class="mw-headline">Introduction</span></h2>
<a name="Objectives" id="Objectives"></a><h2> <span class="mw-headline">Objectives</span></h2>
<a name="Required_Materials_.28Bring_to_All_Labs.29" id="Required_Materials_.28Bring_to_All_Labs.29"></a><h2> <span class="mw-headline">Required Materials (Bring to All Labs)</span></h2>
<a name="Prerequisites" id="Prerequisites"></a><h2> <span class="mw-headline">Prerequisites</span></h2><ul><li> <a href="[https://scswiki.senecaccdot.onsenecacollege.ca/%7Efacwiki/sec520SEC520/labs/SEC520_Lab_1.html">Lab_1 SEC520 Lab 1</a>]
<a name="Linux_Command_Online_Reference" id="Linux_Command_Online_Reference"></a><h2> <span class="mw-headline">Online Tools and References</span></h2>
<tbody><tr valign="top">
<td><b>Other</b></td>
<li><a href="http://linuxmanpages.com/man1/whois.1.php" target="_new">whois</a></li>
<li><a href="http://linuxmanpages.com/man1/host.1.php" target="_new">host</a></li>
<li><a href="http://linuxmanpages.com/" target="_new">Online Linux Manpages</a></li>
</tbody></table>
<a name="Resources_on_the_web" id="Resources_on_the_web"></a><h2> <span class="mw-headline">Course Notes</span></h2>
<a name="Performing_Lab_2" id="Performing_Lab_2"></a><h1> <span class="mw-headline">Performing Lab 2</span></h1><a name="Task1" id="Investigation_1:_How_to_Perform_a_Fedora_DVD_Install_on_Your_Removable_Hard_Drive"></a><h2> <span class="mw-headline">Task #1: Using Search Engines to Obtain Target Server Information </span></h2>
<div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"> <div style="float: left; margin-left: -40px;"><a href="https://scs.senecac.on.ca/wiki/index.php/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files/35px-Idea.png" height="35" border="0" width="35"><{{Admon/a></div> <div><b>tip|sensepost.com</b><br>|This is a website that is dedicated to internet security, and provides a platform to help gather information regarding a server. In fact, examples from the textbook: <b>Penetration Tester's Open Source Toolkit</b> use examples from this website. We will be using this site for the majority of lab2...</div> </div>}}
<a name="Task2" d="Investigation_2:_How_many_file_packages_and_files_are_installed_on_the_system.3F"></a><h2> <span class="mw-headline">Task #2: Server Detection, Link Analysis & Domain Name Expansion</span></h2>
</ol>
<a name="Task3" id="Investigation_1:_How_to_Perform_a_Fedora_DVD_Install_on_Your_Removable_Hard_Drive"></a><h2> <span class="mw-headline">Task #3: Foot-printing</span></h2>
<a name="Task4" d="Investigation_2:_How_many_file_packages_and_files_are_installed_on_the_system.3F"></a><h2> <span class="mw-headline">Task #4: Obtaining User Information</span></h2>
<div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"><div style="float: left; margin-left: -40px;"><a href="https:{{Admon//scs.senecac.on.ca/wiki/index.php/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files/35px-Important.png" height="35" border="0" width="35"></a></div><div><b>important|Install metagoofil program</b><br>|<br><br> The harvester program is already installed in your Kali system, but you will need to install the program metagoofil. Issue the command (as root):<br><br><b>apt-get install metagoofil</b><br><br></div> </div>}}<br><br></li>
<a name="Task5" d="Investigation_2:_How_many_file_packages_and_files_are_installed_on_the_system.3F"></a><h2> <span class="mw-headline">Task #5: Verification / The "Tank" Server</span></h2>
<div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"><div style="float: left; margin-left: -40px;"><a href="https://scs.senecac.on.ca/wiki/index.php{{Admon/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files/35px-Important.png" height="35" border="0" width="35"></a></div><div><b>important|Location of dnsmap Utility in Kali Linux</b><br>|The <b>dnsmap</b> utility is a time-saving method of determining reverse dns lookups in a batch mode involving an input file of collected dns entries.<br><br>This utility is contained in your Kali Linux boot media under the file pathname: <b>/pentest/enumeration/dns/dnsmap</b></div></div>}}
<a name="Completing_the_Lab" id="Completing_the_Lab"></a></p><h1> <span class="mw-headline"> Completing the Lab </span></h1>
<a name="Preparing_for_Quizzes" id="Preparing_for_Quizzes"></a><h1> <span class="mw-headline"> Preparing for Quizzes </span></h1>
Protected "SEC520/labs/Lab 2": OER transfer ([Edit=Allow only administrators] (indefinite) [Move=Allow only administrators] (indefinite))
<dl><dd><ul><li>This lab teaches various methods of <b>gathering information</b> from a <b>targeted computer system</b>. Normally, an individual or a company can be hired to perform <b>Penetration Testing</b> in order to detect weaknesses in an organization's computer system. The first phase (called the <b>"reconnaissance phase"</b>
is considered to be a "harmless activity", where a person can simply gather information to be used later in other aspects of penetration
testing (network <i>scanning</i> and <i>enumeration</i>).
</li></ul>
<br><br>
<ol><li>Use the <b>search engine website (google.ca)</b> to obtain computer system information (including IP address).
</li><li>Use various open-source applications to perform IP address associations with IP address (<b>Link Analysis</b>).
<p><br>
</p>
<ul>
<li> <b>SATA Hard Disk</b> (in removable disk tray).
<p><br>
</p>
</li></ul>
<p><br>
</p>
<table cellpadding="12">
<td><b>Information Gathering</b></td>
<td><b>Foot-printing</b></td>
<td><b>User Information</b></td>
<td><b>Verification</b></td>
</tr>
<td>
<ul>
<li><a href="[http://www.google.ca/" target="_new">Google Search Engine</a> ] (site, filetype, link)</li> <li><a href="[http://news.netcraft.com/" target="_new">Netcraft</a>]</li> <li><a href="[http://github.com/sensepost/BiLE-suite" target="_new">BiLE Utilities</a>]</li>
</ul>
</td>
<td>
<ul>
<li>WHOIS Online Proxies:<br>
(<a href="[http://whois.domaintools.com/" target="_new">whois.domaintools.com</a>])
</li>
</ul></td>
<td>
<ul>
<li><a href="[http://www.ehacking.net/2011/08/theharvester-backtrack-5-information.html" target="_new">theHarvester.py</a>]</li> <li><a href="[http://www.ehacking.net/2011/12/metagoofil-backtrack-5-tutorial.html" target="_new">Metagoofil.py</a>]</li>
</ul>
</td>
<td>
<ul>
<li><a href="[http://www.bing.com/" target="_new">www.bing.com</a>]</li> <li><a href="[http://www.computerhope.com/unix/unslooku.htm" target="_new">nslookup</a>]</li> <li><a href="[http://www.ehacking.net/2011/02/dnsmap-dns-network-mapper.html" target="_new">dnsmap</a>]</li>
</ul>
</td>
<td>
<ul>
</ul>
</td>
</tr>
<p><br>
</p>
<ul>
<li><a href="[http://cs.senecac.on.ca/%7Efac/sec520/slides/sec520_w1_l2.odp" class="external text" title="http://cs.senecac.on.ca/~fac/sec520/slides/sec520_w1_l2.odp" rel="nofollow">odp</a>]| <a href="[http://cs.senecac.on.ca/%7Efac/sec520/slides/sec520_w1_l2.pdf" class="external text" title="http://cs.senecac.on.ca/~fac/sec520/slides/sec520_w1_l2.pdf" rel="nofollow">pdf</a>]| <a href="[http://cs.senecac.on.ca/%7Efac/sec520/slides/sec520_w1_l2.ppt" class="external text" title="http://cs.senecac.on.ca/~fac/sec520/slides/sec520_w1_l2.ppt" rel="nofollow">ppt</a>](Slides: Reconnaissance)</li> <li><a href="http://www.youtube.com/watch?v=AHEt0mUZH_0" target="_new">Reconnaissance</a> (YouTube Video)</li> <li><a href="http://libcat.senecac.on.ca/vwebv/holdingsInfo?searchId=89542&recCount=50&recPointer=0&bibId=315433" target="_new">Penetration Tester's Open Source Toolkit (E-book)</a> (Chapter 2: Reconnaissance)</li>
</ul>
<p><br>
</p>
<p>With the "information gathering" phase of penetration testing, it is
recommended to obtain as much data regarding a targeted organization.
the above-mentioned techniques, there are other techniques and tools to
help gather useful server information of a targeted organization.</p>
<br>
INSTRUCTIONS:
search (i.e total number of links at the top of the search results), and record the total number of links for this type of search in
your lab logbook.</li>
<li>Now, enter the following directive in the Google search box: <b>site:sensepost.com</b><br><div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"> <div style="float: left; margin-left: -40px;"><a href="https://scs.senecac.on.ca/wiki/index.php/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files{{Admon/35px-Important.png" height="35" border="0" width="35"></a></div> <div><b>important|Enter Site Directive in Google Search Textbox</b><br>|Don't enter the "site" directive in the URL textbox at the top of the web-browser - enter this directive in the Google SEARCH text; otherwise, the directive will not work. Also make certain remain in the google web-page when performing this operation...</div> </div>}}</li>
<li>You should notice a change in the display of links. How does this
search method differ from the previous search method using only the
just collected during this lab for penetration testing? (Record your
answer in your lab log-book)</li>
<li>Repeat the information-gathering process for the following URL: <b>linux.senecac.on.ca</b> for practice.<br><div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"> <div style="float: left; margin-left: -40px;"><a href="https:/{{Admon/scs.senecac.on.ca/wiki/index.php/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files/35px-Idea.png" height="35" border="0" width="35"></a></div> <div><b>tip|Gathering Information in your Own Server at Home</b><br>|Just for Interest, it is not that difficult to obtain SOME information regarding your own computer system at home. First, determine your IP address by using the <b>ifconfig</b> command for Linux, or the <b>ipconfig</b> command in windows. One very quick way to determine your IP Address is to simply type <b>IP Address</b> in the URL Window of your web-browser. Knowing your own IP Address at home is useful during the <b>link analysis</b> and <b>domain name expansion</b> steps in the next task...</div> </div>}}</li>
<li>Proceed to Task #2<br><br></li>
</ol>
</p>
<br><br>
<p>In this section, we will use the site information (obtained from <i>Task #1</i>)
INSTRUCTIONS:
<ol>
<li>Assuming that your web-browser is still running, click on the following link (which should open in another browser window): <b><a href="[http://www.netcraft.com" target="_blank">http://www.netcraft.com</a>]</b>.<br /><b>NOTE:</b> Do not worry if you are redirected to another URL (eg. news.netcraft.com) - it will provides the same information we require.<br /><br /></li>
<li>Let's find out additional information regarding the <b>sensepost.com</b> website. In the <b>What's that site running?</b> box, enter the following:<br><b>sensepost.com</b></li>
<li>Record the following server information for "sensepost.com" (and record in your lab log-book):<ul><li>IP Address</li><li>Type of Operating System</li><li>Name Server</li><li>Country Origin</li><li>Date First Noticed (Tracked)</li><li>Frequency of Uptimes</li></ul></li>
categorize relationships between other websites, and the <i>"target"</i> website
called <b>"sensepost.com"</b>. You will be downloading, installing and running
serveral open-source tools (a series of packages packaged as <b>BiLE</b> (which stands for: <i>"Bi-directional Link Extraction"</i> tools) to asssist in obtaining this information.<br><br></p> <div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"> <div style="float: left; margin-left: -40px;"><a href="https:{{Admon//scs.senecac.on.ca/wiki/index.php/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files/35px-Important.png" height="35" border="0" width="35"></a></div> <div><b>important|Installing Dependencies for BiLE.pl, BiLE-Weigh.pl</b><br>|You may need to download the <b>BiLE</b> Utilities, consisting of useful Perl Scripts. Your Kali Linux distribution most likely comes with Perl already loaded. On the other hand, prior to running these Perl Scripts, you may be required to first install the application called <b>HTTrack</b>. You can do this by installing "httrack" via "apt-get" or use a graphical application (such as <b>Synaptic Package Manager</b>)</div></div>}}
<br /></li>
<li>Issue the command: <b>which httrack</b> to confirm that this dependent application has been installed (refer to warning message above).</li>
<li>In a web-browser, go to the following website (which will open in a separate browser window): <b><a href="[http://github.com/sensepost/BiLE-suite" target="_blank">http://github.com/sensepost/BiLE-suite</a>]</b></li> <li>Download the <i>Perl Scripts</i> called <b>BiLE.pl</b>, <b>BiLE-Weigh.pl</b>, and <b>tld-expand.pl</b> to your Kali Linux system.<br><br><div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"> <div style="float: left; margin-left: -40px;"><a href="https://scs.senecac.on.ca/wiki/index.php/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files/35px-Important.png" height="35" border="0" width="35"></a><{{Admon/div> <div><b>important|Perl Scripts Containing Errors When Executed</b><br>|If errors occur, <b>check to see if that Perl Scripts were
properly downloaded. If they contain HTML code, an alternative to
downloading is to display the Perl Script in the web-browser, copying and pasting the code to the file on your computer</b> (<i>as opposed to right-clicking link and saving to your computer</i>). </div> </div>}}<br></li>
<li>Run the following command: <b>perl BiLE.pl sensepost.com output.sensepost.com</b> (assuming BiLE.pl is located in the current directory).<br><br>Note: This process may take serveral minutes to complete.<br><br></li>
<li>When the process has completed, a report called "<b>output.sensepost.com.mine</b>"
target website, as well as the output-file (generated by the BiLE.pl
Perl Script.</li>
<li>Issue the following command: <b> perl BiLE-weigh.pl sensepost.com output.sensepost.com.mine</b> (Assuming BiLE.pl Script and "output.sensepost.com" are contained in the current directory).<br><br> <div class="messagebox" style="background-color: #f9f6b7; border: 1px solid #c4c295; color: black; padding: 5px; margin: 1ex 0; min-height: 35px; padding-left: 45px;"> <div style="float: left; margin-left: -40px;"><a href="https:{{Admon//scs.senecac.on.ca/wiki/index.php/File:Important.png" class="image" title="Important.png"><img alt="" src="SEC520_Lab_1_files/35px-Important.png" height="35" border="0" width="35"></a></div> <div><b>important|Error: Sort: open failed: +1: No such file or directory</b><br>|If you run the <b>BiLE-Weigh.pl</b> command, and encounter the above error, then make the following editing changes for this script:<br><br><b>change following line:</b><nowiki> 'cat temp | sort -r -t ";" +1 -n > @ARGV[1].sorted';</nowiki><br><br><b>to read:</b><nowiki> `cat temp | sort -r -t ":" -k 2 -n > @ARGV[1].sorted`;</nowiki><br><br>(Note: ` in this case is "Left-Tick" representing command substitution - not to be confused with a single-quote.<br /><br /></div> </div>}}<br><br></li>
<li>View the contents of the file "output.sensepost.com.sorted" in your
current directory. Notice the ranking of the relavance of links
<li> Issue the command: <b>perl tld-expand.pl sensepost.com.domains.txt sensepost.com.domains.variations.txt</b>. What do these variations represent in terms of reconnaissance? Record your finds in your lab log-book.</li>
<li>Proceed to Task #3<br><br></li>
<p><b>Answer Task #2 observations / questions in your lab log book.</b>
</p>
<br>
As opposed to the Information Gathering phase (that collects information such as IP Addresses), the Foot-printing phase tends to gain a “clearer picture” of the structure of the organization's computer system. This can include <b>relationships among servers</b>, as well as noting <b>IP Address ranges</b>.
<br><br>
Footprinting (in simpler terms) means <b>Network Mapping</b>.
<br />
<br>
You will be using the information collected in Task #1 to assist with obtaining User information in this task.
<br>
<p>
INSTRUCTIONS:
</p><ol>
<li>Issue the command <b>theharvester --help</b>, to learn how to run this script again with the following options:<ul><li>Domain: <b>sensepost.com</b></li><li>Number of limited results: <b>100</b></li><li>Data Source: <b>google</b></li><li>Output filename: <b>~/sensepost.user</b><br><br></li></ul></li>
<li>Record any user information that you consider relevant (for penetration testing) in your lab log-book.</li>
</p>
<br>
<br>
It is important to "double-check" the validity of your collected information - in particular, your IP addresses. If any servers are no longer running, this can waste a tremendous amount of time during the scanning process. Remember: the longer a scan takes to execute, the more vulnerable you are as the penetration tester to detection.
<br><br>
<br><br>
<p><b>Arrange evidence for each of these items on your screen, then ask
your instructor to review them and sign off on the lab's completion:</b>
<p><br>
</p>
<ol>