website vulnerability scanner

Sarath Pillai's picture
Website vulnerability Scanning

One of the inevitable resource that has become part of our day to day life is the world wide web. Networks and inter networking originated from a series of research conducted at ARPA(Advanced Research Projects Agency). ARPANET(Advanced Research Projects Agency Network) is the worlds first working TCP/IP network. 

Design and configuration flaws are a norm in any technology related to any field. Technology keeps of improving day by day due to fixes and improvement done by smart people in the form of iterations, and version updates. During the initial day's of internet, the people involved in using the network were all committed to that technology and most of them only had good intentions in mind. But later on, there emerged a need to improve the security and design of many network related services, due to break in attempts and information breach done by people with ill intentions. Almost all the services provided by a company, needs to be made publicly available, because that's the way they communicate with their customers.

A website is the primary interface, that a company needs to make available publicly. The backbone or the underlying protocol, that makes a website available is none other than the HTTP protocol(which is the highest used protocol on the internet.).

Related: How does HTTP work

After the protocol comes the web server software that serves the incoming requests from a client computer. There are a lot of web server software's to select from. But as i told before, services and technology keeps of improving, due to fixes done in the form of version updates. But vulnerabilities in web server software's and any other application for that matter, is not always because of design flaws and bugs.

Another major factor that makes a publicly available service vulnerable is human error(or say miss configuration). There are a large number of security checklist that a system administrator needs to go through, due to which its quite normal to miss some or the other.

Related: Linux Security Checklist for a system administrator

So a good approach towards solving this problem is to have another tool that can be used as a security and vulnerability scanner, to check and report the current configurations and other details, for a system administrator to analyze. We will be discussing one such tool, called as Nikto, that can be used to scan the following things.

  • Vulnerability due to misconfiguration
  • Files and programs that are dangerous for security
  • Old or outdated software programs

The main advantage of using Nikto is that, unlike other tools out there,it can be used against almost all web server softwares out there. Another advantage is that it can be used against a server having SSL enabled, Host authentication and a lot of other things. 

Let's get started by installing Nikto on a Linux system.

[root@localhost ~]# wget http://cirt.net/nikto/nikto-2.1.5.tar.gz
--2013-05-20 02:20:28--  http://cirt.net/nikto/nikto-2.1.5.tar.gz
Resolving cirt.net... 174.142.17.165
Connecting to cirt.net|174.142.17.165|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 371663 (363K) [application/x-gzip]
Saving to: `nikto-2.1.5.tar.gz'

100%[=============>] 371,663      272K/s   in 1.3s

2013-05-20 02:20:33 (272 KB/s) - `nikto-2.1.5.tar.gz' saved [371663/371663]

The above wget command will download nikto version 2.1.5.tar.gz, on your system. Now let's unzip the tar file, and see what's inside the directory.

[root@localhost ~]# tar -xvf nikto-2.1.5.tar.gz

Unpacking the tar file, will create a folder named, nikto-2.1.5 on the current working directory. Inside the directory there is a pearl script named "nikto.pl", which is the primary script that will be used, for scanning web servers. For running nikto.pl, script, you need to have Perl language interpreter installed on the system.

Using the nikto.pl script is very simple. Its run the same way you run any Perl programs on Linux. Let's see the default nikto.pl command output without any argument.

[root@localhost nikto-2.1.5]# perl nikto.pl
- Nikto v2.1.5
---------------------------------------------------------------------------
+ ERROR: No host specified

       -config+            Use this config file
       -Display+           Turn on/off display outputs
       -dbcheck            check database and other key files for syntax errors
       -Format+            save file (-o) format
       -Help               Extended help information
       -host+              target host
       -id+                Host authentication to use, format is id:pass or id:p
       -list-plugins       List all available plugins
       -output+            Write output to this file
       -nossl              Disables using SSL
       -no404              Disables 404 checks
       -Plugins+           List of plugins to run (default: ALL)
       -port+              Port to use (default 80)
       -root+              Prepend root value to all requests, format is /direct
       -ssl                Force ssl mode on port
       -Tuning+            Scan tuning
       -timeout+           Timeout for requests (default 10 seconds)
       -update             Update databases and plugins from CIRT.net
       -Version            Print plugin and database versions
       -vhost+             Virtual host (for Host header)
                + requires a value

        Note: This is the short help output. Use -H for full help text.

The default output shows you a message, that it at least requires a host argument(which is the target server, on which to conduct the test), along with some of the options available. We will first scan a Apache web server on the local machine, to see and understand the output, before going ahead with advanced options of nikto.

[root@localhost nikto-2.1.5]# perl nikto.pl -host 127.0.0.1
- Nikto v2.1.5
---------------------------------------------------------------------------
+ Target IP:          127.0.0.1
+ Target Hostname:    localhost.localdomain
+ Target Port:        80
+ Start Time:         2013-05-20 02:39:35 (GMT5.5)
---------------------------------------------------------------------------
+ Server: Apache/2.2.3 (CentOS)
+ Server leaks inodes via ETags, header found with file /, inode: 3736673, size: 40634, mtime: 0x66de2bc0
+ The anti-clickjacking X-Frame-Options header is not present.
+ Apache/2.2.3 appears to be outdated (current is at least Apache/2.2.22). Apache 1.3.42 (final release) and 2.0.64 are also current.
+ Allowed HTTP Methods: GET, HEAD, POST, OPTIONS, TRACE
+ OSVDB-877: HTTP TRACE method is active, suggesting the host is vulnerable to XST
+ OSVDB-682: /usage/: Webalizer may be installed. Versions lower than 2.01-09 vulnerable to Cross Site Scripting (XSS). http://www.cert.org/advisories/CA-2000-02.html.
+ OSVDB-3092: /manual/: Web server manual found.
+ OSVDB-3268: /icons/: Directory indexing found.
+ OSVDB-3268: /manual/images/: Directory indexing found.
+ OSVDB-3233: /icons/README: Apache default file found.
+ 6544 items checked: 0 error(s) and 10 item(s) reported on remote host
+ End Time:           2013-05-20 02:39:44 (GMT5.5) (9 seconds)
---------------------------------------------------------------------------
+ 1 host(s) tested

If you see the above output, nikto provides us with a very detailed information about the target server where we ran a test. The web server details provided by nikto are as under.

  • The web server IP & Hostname
  • Web server software name and its version
  • If the current version of the web server software installed is updated or an outdated version. If the version is outdated
  • OSVD vulnerabilities.

 

What is OSVD?

OSVD stands for Open Source Vulnerability Database, It provides an indepth information about identified vulnerabilities of different software products. You can any time have a look at the database and get the information regarding, the vulnerability ID of your interest.

It was established during 2002 as a result of a defcon and black hat meeting.

Each and every vulnerability exposed are identified by a number. For example let's search one of the OVSD vulnerabilities on our example nikto test conducted a couple of minutes back.

OSVD 877: Is a vulnerability related to http trace method. Trace is an http method similar to another http request methods. It provides information about how a request is being received at the server end to the client.

Which means the client can easily find the request detail, that the server is receiving from it. Although HTTP Trace is a method that can be used to troubleshoot problems at web server end, it can sometimes reveal some critical information to the client.

It's better to disable the trace method in Apache web server, and is also suggested by nikto during our sample run against our example web server.

So the first thing that we need to do is to look for possible fixes related to the OSVD id number's disclosed by nikto. The information about the OSVD id numbers can be found at the official site for OSVD

HTTP Trace method vulnerability details can be found at this page of OSVD database.

OSVD ID number information provides you description, date of disclosure of the vulnerability, Solution, as well as the products affected by this vulnerability.

Let's now see some command line options and usage of nikto Perl script in Linux.

The below command can be used to run nikto on a different port number on http web server. By default if no port option is used as a command line argument then the default http 80 port is assumed. Let's see how to run nikto on a different port number for example, SSL port 443.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h localhost -p 80

Even using that port number of 443 is also not required, because by default if no port is specified nikto will first scan port number 80, and if no port 80 is available, then it will scan port 443. Alternatively you can use -ssl.

You can also test multiple ports at the same time, as shown below.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h localhost -p 80,90,300

You can do a test on multiple hosts by specifying the host and port options line by line in a file. For example, i will be showing ip addresses in the host, you can even mention your publicly available websites line by line. For example, let's create a file with the list of IP/Host names along with the required port number one per line.

[root@myvm1 nikto-2.1.5]# cat sample-websites
172.16.130.14
192.168.0.45
localhost
example.com
[root@myvm1 nikto-2.1.5]# perl nikto.pl -h localhost -h sample-websites

As shown in the above example, you can use the -h, option to specify the file containing the number of hosts and websites line by line.

All protocol's should adhere to the complete working methods mentioned in the RFC(Request for comments). The main purpose behind this is to have a universally same behaviour regardless of the platform used.

However some applications do not completely adhere to the RFC specifications, due to which the response of an application that works on a specific protocol might sometimes be different from the required response as mentioned in the RFC. Nikto works on the principle of sending a large number of requests to the web server and then analyze the response.

Due to this reason, the web servers that do not completely follow the RFC specifications, might give a different output from the desired one, which causes a slightly incorrect nikto test.

The best example of this is that some web servers reply with a string "Not Found"(along with a HTTP 200 ok reply) instead of a 404, for files and URL's that are not present on the server. Due to this Nikto will not be able to find the details of some files. To solve this problem Nikto converts the reply string to an md5 hash(after removing date and timestamp from the reply) and compares the hash with the future outputs. This comparison is done by making several requests to the file's that are not present. However this will cause thousands of requests on the web server for non existent files.

You can disable this by using the option -no404
as shown below.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -host 192.168.159.128 -no404

Nikto run's its default test with the help of a configuration file called nikto.conf. This file is inside the unzipped tar archive(where nikto.pl is also located). Let's go through some of the noteworthy contents inside this configuration file.

  • The port numbers that you do not want to scan. For example, if you do not want to scan the ports 30 and 40 then you can enter those port number's in following way.

     SKIPPORTS=30 40

  • You can even modify the user agent used to run the scan. For example, the default nikto.conf, file contains the below user agent.

    Mozilla/5.00 (Nikto/@VERSION) (Evasions:@EVASIONS) (Test:@TESTID)

  • You can also mention the reference URL, for the request initiated by Nikto tool. This can be done by modifying the below line of nikto.conf file.

     RFIURL=http://cirt.net/rfiinc.txt?

  • Default version of the HTTP Protocol that will be used for running the Nikto test. By default the version value is set to version 1.0.

     DEFAULTHTTPVER=1.0

  • Nikto uses different plugins to add different functionalities to its scan. By default, these plugins are located inside the plugins folder, where the nikto tar archive was unzipped. However you can change the directory as per your wish by modifying the plugins directive inside the configuration file.

     PLUGINDIR=/opt/nikto/plugins

     You can also have a look at the plugins that are currently enabled with nikto, by running the below command(i have not shown the complete command output in the below example).

   

[root@myvm1 nikto-2.1.5]# perl nikto.pl -V
---------------------------------------------------------------------------
Nikto Versions
---------------------------------------------------------------------------
File                               Version      Last Mod
-----------------------------      --------     ----------
Nikto main                         2.1.5
LibWhisker                         2.5
JSON-PP.pm
db_404_strings                     2.003
db_content_search                  2.000
         2012-07-04                1.0
  • There is one argument in the nikto configuration file, that is used to modify or add the number of http request types that can be used to send requests to the web server to identify the server and collect other details.

    By default nikto uses two http methods called GET and HEAD.

    CHECKMETHODS=HEAD GET

As we have discussed in the beginning of this tutorial, nikto is not at all a stealthy scanner to scan web servers. The web server administrator will be easily able to detect the Nikto scan. One of the major reason is that the requests does provide its own information in multiple request fields.

It runs thousands of tests on a single server, to analyze the output, and find dangerous files,versions etc. But the access logs on the web server will contain the complete information about this Nikto scan. Nikto was not designed by keeping stealthiness in mind. Because its a security testing tool, and is not made for breaking it.

Let's have a look some access log contents that's made by the nikto scan.

127.0.0.1 - - [21/May/2013:09:18:55 -0700] "GET /sitemap.gz HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003582)"
127.0.0.1 - - [21/May/2013:09:18:55 -0700] "GET /content/sitemap.gz HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003583)"
127.0.0.1 - - [21/May/2013:09:18:55 -0700] "GET /icons/README HTTP/1.1" 200 5108 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003584)"
127.0.0.1 - - [21/May/2013:09:18:55 -0700] "GET /localstart.asp HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003585)"
127.0.0.1 - - [21/May/2013:09:18:55 -0700] "GET /ampache/update.php HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003586)"
127.0.0.1 - - [21/May/2013:09:18:56 -0700] "GET /ms/ HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003747)"
127.0.0.1 - - [21/May/2013:09:18:56 -0700] "GET /ma/ HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003748)"
127.0.0.1 - - [21/May/2013:09:18:56 -0700] "GET /mz/ HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003749)"
127.0.0.1 - - [21/May/2013:09:18:56 -0700] "GET /mm/ HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003750)"
127.0.0.1 - - [21/May/2013:09:18:56 -0700] "GET /na/ HTTP/1.1" 500 605 "-" "Mozilla/5.00 (Nikto/2.1.5) (Evasions:None) (Test:003751)"

If you look at the above access logs you can clearly detect that the scan is being conducted by some tool called as nikto. You can even hide this by modifying request paramters inside the scrupt source code.

But even modifying the source code is not a  better method, because a good IDS(Intrution Detection System) system will still be able to detect that the scan has occured.

There are plugins in nikto that will help to hide and go undetected from the IDS. Nikto uses something called as LibWhisker for achieving this.

What is LibWhisker ?

Its a plugin/library that's written in perl, for making custom made http packets for making requests that will help to stay stealthy to a certain extent to IDS.

This library includes 9 different methods of remaining stealthy from the IDS systems. Lets go throgh some of them and see how to use then while scanning with nikto.

  • Random URI encoding (non-UTF8)

URI encoding is normally done by web browsers, before sending the request to the web server. But the accepted charecters that can be send are only ASCII. Hence if you have something outside of ASCII, you need to place a valid ASCII format there.

URI does contain some values that are not ASCII, due to which they have to be first converted to ASCII, before sending them to the server, which causes them to add some hexadecimal charecters along with a % sign. This technique of modifying this URI encoding is IDS evasion technique number 1.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -e 1
  •  Add directory self-reference

This method will add a /./ infront of each request, and is called as evasion technique 2. This is a well known technique and IDS will detect it. To use this technique, you need to run the below command.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -e 2
  • Premature URL ending

In this method, random folders are added before the file requested in the URL. For examplAe, if you want to request a file called as sample.txt, then it will be done by the following method by prepending multiple folders before it.

/somefolder/somefolder/sample.txt

This test can be conducted by the following method, and is evasion number 3.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -e 3
  • Prepend long random string

This method can be used to prepend longer strings before the actual request url.  This is called evasion number 4, and can be used as shown below.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -e 4
  • Fake parameter

Some kind of parameters is used along with the URL. These are fake paramters and looks something like the below.

/someurl/long=fakeparamter

This kind of scan can be conducted as shown below.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -e 5

A URL example of fake paramter is shown below.

GET /modules.php?set_albumName=album01&id=aaw&op=modload&name=gallery&file=index&include
 

You can get the complete list of evasion technique id's and their details from the help page of nikto perl script.

 -evasion+          Encoding technique:
                               1     Random URI encoding (non-UTF8)
                               2     Directory self-reference (/./)
                               3     Premature URL ending
                               4     Prepend long random string
                               5     Fake parameter
                               6     TAB as request spacer
                               7     Change the case of the URL
                               8     Use Windows directory separator (\)
                               A     Use a carriage return (0x0d) as a request spacer
                               B     Use binary value 0x0b as a request spacer


You can even use a proxy server, if you have one, before conducting a test. You just need to pass useproxy with the proxy server and port as an argument to the command.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -useproxy 192.168.159.128:8080

Nikto conducts almost all the tests by default if not tuned. Nikto provides a command line option to tune its tests. This helps in including and excluding certain types of tests. Like for example

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -T 23
- ***** SSL support not available (see docs for SSL install) *****
- Nikto v2.1.5
---------------------------------------------------------------------------
+ Target IP:          192.168.159.128
+ Target Hostname:    myvm1.sarath.com
+ Target Port:        80
+ Start Time:         2013-05-23 21:08:44 (GMT-7)
---------------------------------------------------------------------------
+ Server: Apache/2.2.22 (EL)
+ Cookie PHPSESSID created without the httponly flag
+ Retrieved x-powered-by header: PHP/5.2.17
+ The anti-clickjacking X-Frame-Options header is not present.
+ Root page / redirects to: login.php
+ OSVDB-877: HTTP TRACE method is active, suggesting the host is vulnerable to XST
+ 125 items checked: 0 error(s) and 4 item(s) reported on remote host
+ End Time:           2013-05-23 21:08:45 (GMT-7) (1 seconds)
---------------------------------------------------------------------------
+ 1 host(s) tested
[root@myvm1 nikto-2.1.5]#

In the above test nikto only conducted two tests. The two types of tests that were conducted are:

  1. Misconfiguration on the web server.
  2. Revealing of information

-T we have used above allows to fine tune the test and the number 23 will ask to only conduct the above mentioned two tests. A full list of tuning options can be found at help page of nikto.

-Tuning+           Scan tuning:
                               1     Interesting File / Seen in logs
                               2     Misconfiguration / Default File
                               3     Information Disclosure
                               4     Injection (XSS/Script/HTML)
                               5     Remote File Retrieval - Inside Web Root
                               6     Denial of Service
                               7     Remote File Retrieval - Server Wide
                               8     Command Execution / Remote Shell
                               9     SQL Injection
                               0     File Upload
                               a     Authentication Bypass
                               b     Software Identification
                               c     Remote Source Inclusion
                               x     Reverse Tuning Options (i.e., include all except specified)

Nikto also provides a method, to save the result of a test conducted, and that same test can be reconducted with the help of replaying the same test. This is done by first saving the test output.

[root@myvm1 nikto-2.1.5]# perl nikto.pl -h 192.168.159.128 -T 23 -Save 23-test

After completing the above tests, there will be a directory with the name of 23-test. Now lets replay the same test.

[root@myvm1 nikto-2.1.5]# perl replay.pl -file 23-test/myvm1.example.com_80_2013-05-23_999960.txt

The above command, will rerun the already saved test that was previously conducted.

Rate this article: 
Average: 3.5 (54 votes)

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.