January 29, 2014

Configure Nagios Monitoring

Prerequisite

Before you begin verify you server and client configuration. From the server test client connection.

$ /usr/lib64/nagios/plugins/check_nrpe -H <client_ip_or_hostname>
NRPE v2.14

To setup server, see http://magnus-k-karlsson.blogspot.se/2014/01/install-nagios-core-35-on-rhel-6-from.html.

And for client, see http://magnus-k-karlsson.blogspot.se/2014/01/install-nagios-agent-nrpe-on-rhel-6.html

Client Nagios Plugins Configuration

First decide what you want to monitor. Then find a plugin that do the job. The standard plugins are documented here (http://nagios-plugins.org/doc/man/index.html).

You could also get help documentation for a specific plugin by executing it with option '--help'.

$ /usr/lib64/nagios/plugins/check_disk --help
check_disk v1.4.16 (nagios-plugins 1.4.16)

After you have tested it and is satisfied, you now define it as a nagios command.

$ vi /etc/nrpe.d/commands.cfg
...
command[check_sda2_disk]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 15% -p /dev/sda2
...

Restart nrpe to let new configuration take effect.

$ service nrpe restart
Shutting down nrpe:                                        [  OK  ]
Starting nrpe:                                             [  OK  ]

Server Nagios Plugins Configuration

From the server test that you can call the client nrpe command.

$ /usr/lib64/nagios/plugins/check_nrpe -H <client_ip_or_hostname> -c check_sda2_disk

If this did not work, go back to the prerequisites and verify you server and client installation. If good, lets continue and define the nrpe command on the server.

$ vi /etc/nagios/conf.d/server1.domain.local.cfg
...
# 'check_nrpe' command definition
define command{
        command_name    check_nrpe
        command_line    $USER1$/check_nrpe -H <client_ip_or_hostname> -c $ARG1$
        }
...

Then we need to define a service that will run this command.

$ vi /etc/nagios/conf.d/server1.domain.local.cfg
...
define service{
        use                             generic-service         ; Name of service template to use
        host_name                       yourhostname.domain.local
        service_description             Checks partition sda2 at 20 proc and 15 proc
        check_command                   check_nrpe!check_sda2_disk
        }
...

Finally you will need to define a host, if you have not already done that.

$ vi /etc/nagios/conf.d/server1.domain.local.cfg
...
define host{
        use                     linux-server            ; Name of host template to use
                                                        ; This host definition will inherit all variables that are defined
                                                        ; in (or inherited by) the linux-server host template definition.
        host_name               yourhostname.domain.local
        alias                   yourhostname.domain.local
        address                 192.168.1.117
        }
...

Before we restart the nagios server, you can test you new configuration

$ nagios -v /etc/nagios/nagios.cfg 

Nagios Core 3.5.1
...
Total Warnings: 0
Total Errors:   0
..

If everything is fine, you can now restart you nagios server.

Open you nagios web admin and login in and check you new service.

January 21, 2014

The NTP Daemon on RHEL 6

Installation

Check that NTP services is started at boot.

$ chkconfig ntpdate --list
ntpdate         0:off 1:off 2:off 3:off 4:off 5:off 6:off

If not automatically started.

$ chkconfig ntpdate on

If NTP service is not running, start it.

$ service ntpdate start

Configuration

Default configuration is almost always sufficient.

$cat /etc/ntp.conf
# For more information about this file, see the man pages
# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).
...
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.rhel.pool.ntp.org iburst
server 1.rhel.pool.ntp.org iburst
server 2.rhel.pool.ntp.org iburst
server 3.rhel.pool.ntp.org iburst
...

Test

Check system clock against a reliable source.

$ date
Tue Jan 21 07:07:06 CET 2014

January 13, 2014

Install Nagios Agent NRPE on RHEL 6 from EPEL Repository

Introduction

The Nagios Plugins are the components that do the actual monitoring work. They are all bash scripts and are located in:

$ ll /usr/lib64/nagios/plugins/

To make the plugins/agents talk with server/Nagios Core you use the NRPE (Nagios Remote Plugin Executor).

Agent/Client Installation

Install the EPEL (Extra Packages for Enterprise Linux) repository. [https://fedoraproject.org/wiki/EPEL]

$ rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Install NRPE and all Nagios plugins.

$ yum install nrpe nagios-plugins-all

Agent/Client Configuration

Add the Nagios Core server IP adress.

$ vi /etc/nagios/nrpe.cfg
...
allowed_hosts=127.0.0.1,192.168.122.93
...

Start the nrpe service or restart if you have previously started it, to let the new configuration take effects.

$ service nrpe restart

The NRPE is using port 5666 (see /etc/nagios/nrpe.cfg) to communicate with the server, so we need to open that port in the firewall. Below is the current settings in iptables for the client computer we are trying to monitor.

$ iptables -vnL --line-numbers
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
2    53027   71M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
3        7   588 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
4        1    60 ACCEPT     tcp  --  *      *       192.168.122.0/24     0.0.0.0/0           state NEW tcp dpt:22 
5        0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:21 
6        1    60 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           LOG flags 0 level 4 
7        1    60 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 

Chain OUTPUT (policy ACCEPT 31259 packets, 3978K bytes)
num   pkts bytes target     prot opt in     out     source               destination  

Open tcp port 5666 for incoming traffic.

$ iptables -I INPUT 6 -m state --state NEW -p tcp --dport 5666 -j ACCEPT

Nagios Core Server Configuration

Install the nagios nrpe plugin on the server.

$ yum install nagios-plugins-nrpe

Then we need to enable it. Add the below command to the end of the file.

$ vi /etc/nagios/objects/commands.cfg
...
define command{
    command_name check_nrpe
    command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}

Before we can proceed, we now need to understand how Nagios arrenge items in it's admin GUI.

If you look at the selected menu items to the right, I have selected:

  • Hosts
  • Services
  • Host Groups
  • Service Groups

Now if we open the main nagios configuration file, you will see a similiar structure of the configuration files.

$ vi /etc/nagios/nagios.cfg
...
# OBJECT CONFIGURATION FILE(S)
# These are the object configuration files in which you define hosts,
# host groups, contacts, contact groups, services, etc.
# You can split your object definitions across several config files
# if you wish (as shown below), or keep them all in a single config file.

# You can specify individual object config files as shown below:
cfg_file=/etc/nagios/objects/commands.cfg
cfg_file=/etc/nagios/objects/contacts.cfg
cfg_file=/etc/nagios/objects/timeperiods.cfg
cfg_file=/etc/nagios/objects/templates.cfg

# Definitions for monitoring the local (Linux) host
cfg_file=/etc/nagios/objects/localhost.cfg

# Definitions for monitoring a Windows machine
#cfg_file=/etc/nagios/objects/windows.cfg

# Definitions for monitoring a router/switch
#cfg_file=/etc/nagios/objects/switch.cfg

# Definitions for monitoring a network printer
#cfg_file=/etc/nagios/objects/printer.cfg


# You can also tell Nagios to process all config files (with a .cfg
# extension) in a particular directory by using the cfg_dir
# directive as shown below:

#cfg_dir=/etc/nagios/servers
#cfg_dir=/etc/nagios/printers
#cfg_dir=/etc/nagios/switches
#cfg_dir=/etc/nagios/routers

cfg_dir=/etc/nagios/conf.d
...

You can open the /etc/nagios/objects/localhost.cfg and compare how items are arrenged in the web admin GUI.

Now when we have got a basic understanding of the internal configuration structure we are going to put our new configuration file in /etc/nagios/conf.d/.

$ vi /etc/nagios/conf.d/virtual1.example.com.cfg

###############################################################################
#
# HOST DEFINITION
#
###############################################################################

# Define a host for the remote machine

define host{
        use                     linux-server            ; Name of host template to use
                                                        ; This host definition will inherit all variables that are defined
                                                        ; in (or inherited by) the linux-server host template definition.
        host_name               virtual1.example.com
        alias                   virtual1.example.com
        address                 192.168.122.196
        }

###############################################################################
#
# SERVICE DEFINITIONS
#
###############################################################################

define service{
        use                             generic-service         ; Name of service template to use
        host_name                       virtual1.example.com
        service_description             Current Load
        check_command                   check_nrpe!check_load
        }

Finally restart nagios service and watch you new Host and Service in the web admin GUI.

If you run into problems, open the default nagios log file.

$ less /var/log/nagios/nagios.log

And RHEL default log file. And also read mine previous blog how to configure iptables and logging. [http://magnus-k-karlsson.blogspot.se/2014/01/configure-iptables-for-ftp-server-vsftp.html]

$ less /var/log/messages

Install Nagios Core 3.5 on RHEL 6 from EPEL Repository

Introduction

The Nagios is one of the most poplur product for monitoring IT Infrastructure, such as servers, network and services.

The system overview of the Nagios modules:

  • Nagios Core - The Server with Monitor GUI.
  • NRPE (Nagios Remote Plugin Executor) - The communication handler between server and client/agent.
  • Nagios Plugins - Client or Agent that do the actual monitoring, e.g. check server load or if service is running.

Remember Nagios is not supported by Red Hat and installing Nagios from EPEL repository, is on you own risk. But if you think of the alternative, which is installing Nagios from source and compiling it with gcc is not an attractive alternative. Since then you have to manually uphold the patch level you want in production. And beside you do not want to have the gcc compiler installed in production, due to security risks.

For manual installation see http://nagios.sourceforge.net/docs/3_0/quickstart-fedora.html.

Installation

Install the EPEL (Extra Packages for Enterprise Linux) repository. [https://fedoraproject.org/wiki/EPEL]

$ rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Install the Nagios Core and all plugins.

$ yum install nagios nagios-plugins-all

Start Nagios Server.

$ service nagios start

Start Nagios Server Web GUI.

$ service httpd start

Nagios Core Web GUI

http://localhost/nagios

username: nagiosadmin
password: nagiosadmin

To change the default password for admin, see /etc/httpd/conf.d/nagios.conf.

$ htpasswd /etc/nagios/passwd nagiosadmin

Reference for htpasswd - Manage user files for basic authentication. [http://httpd.apache.org/docs/2.2/programs/htpasswd.html]

Configuration Files

The Nagios main configuration file.

$ /etc/nagios/nagios.cfg

The Nagios Core Admin Server is built with CGI scripts.

$ /etc/nagios/cgi.cfg

HTTPD configuration for Nagios Core

$ /etc/httpd/conf.d/nagios.conf

Appendix. RPM Dependency and Installation

$ repoquery -q --requires --plugins nagios
/bin/sh
chkconfig
group(nagios)
httpd
initscripts
libc.so.6(GLIBC_2.8)(64bit)
libcrypt.so.1()(64bit)
libdl.so.2()(64bit)
libdl.so.2(GLIBC_2.2.5)(64bit)
libgcc_s.so.1()(64bit)
libgcc_s.so.1(GCC_3.0)(64bit)
libgcc_s.so.1(GCC_3.3.1)(64bit)
libgd.so.2()(64bit)
libjpeg.so.62()(64bit)
libm.so.6()(64bit)
libm.so.6(GLIBC_2.2.5)(64bit)
libnsl.so.1()(64bit)
libperl.so()(64bit)
libpng12.so.0()(64bit)
libpthread.so.0()(64bit)
libpthread.so.0(GLIBC_2.2.5)(64bit)
libresolv.so.2()(64bit)
libutil.so.1()(64bit)
libz.so.1()(64bit)
mailx
nagios-common
perl(:MODULE_COMPAT_5.10.1)
php
rtld(GNU_HASH)
user(nagios)
$ yum install nagios
...
==============================================================================================================================================================
 Package                              Arch                          Version                                 Repository                                   Size
==============================================================================================================================================================
Installing:
 nagios                               x86_64                        3.5.1-1.el6                             epel                                        1.2 M
Installing for dependencies:
 gd                                   x86_64                        2.0.35-11.el6                           rhel-x86_64-server-6                        142 k
 libXpm                               x86_64                        3.5.10-2.el6                            rhel-x86_64-server-6                         51 k
 nagios-common                        x86_64                        3.5.1-1.el6                             epel                                         17 k
 php                                  x86_64                        5.3.3-27.el6_5                          rhel-x86_64-server-6                        1.1 M
 php-cli                              x86_64                        5.3.3-27.el6_5                          rhel-x86_64-server-6                        2.2 M
 php-common                           x86_64                        5.3.3-27.el6_5                          rhel-x86_64-server-6                        525 k

Transaction Summary
==============================================================================================================================================================
Install       7 Package(s)
$ yum install nagios-plugins-all
...
==============================================================================================================================================================
 Package                                    Arch                     Version                                     Repository                              Size
==============================================================================================================================================================
Installing:
 nagios-plugins-all                         x86_64                   1.4.16-10.el6                               epel                                    14 k
Installing for dependencies:
 fping                                      x86_64                   2.4b2-10.el6                                epel                                    31 k
 lm_sensors                                 x86_64                   3.1.1-17.el6                                rhel-x86_64-server-6                   123 k
 nagios-plugins                             x86_64                   1.4.16-10.el6                               epel                                   200 k
 nagios-plugins-breeze                      x86_64                   1.4.16-10.el6                               epel                                    14 k
 nagios-plugins-by_ssh                      x86_64                   1.4.16-10.el6                               epel                                    35 k
 nagios-plugins-cluster                     x86_64                   1.4.16-10.el6                               epel                                    29 k
 nagios-plugins-dhcp                        x86_64                   1.4.16-10.el6                               epel                                    35 k
 nagios-plugins-dig                         x86_64                   1.4.16-10.el6                               epel                                    34 k
 nagios-plugins-disk                        x86_64                   1.4.16-10.el6                               epel                                    69 k
 nagios-plugins-disk_smb                    x86_64                   1.4.16-10.el6                               epel                                    17 k
 nagios-plugins-dns                         x86_64                   1.4.16-10.el6                               epel                                    35 k
 nagios-plugins-dummy                       x86_64                   1.4.16-10.el6                               epel                                    26 k
 nagios-plugins-file_age                    x86_64                   1.4.16-10.el6                               epel                                    15 k
 nagios-plugins-flexlm                      x86_64                   1.4.16-10.el6                               epel                                    16 k
 nagios-plugins-fping                       x86_64                   1.4.16-10.el6                               epel                                    35 k
 nagios-plugins-game                        x86_64                   1.4.16-10.el6                               epel                                    31 k
 nagios-plugins-hpjd                        x86_64                   1.4.16-10.el6                               epel                                    33 k
 nagios-plugins-http                        x86_64                   1.4.16-10.el6                               epel                                    76 k
 nagios-plugins-icmp                        x86_64                   1.4.16-10.el6                               epel                                    38 k
 nagios-plugins-ide_smart                   x86_64                   1.4.16-10.el6                               epel                                    31 k
 nagios-plugins-ircd                        x86_64                   1.4.16-10.el6                               epel                                    16 k
 nagios-plugins-ldap                        x86_64                   1.4.16-10.el6                               epel                                    33 k
 nagios-plugins-load                        x86_64                   1.4.16-10.el6                               epel                                    31 k
 nagios-plugins-log                         x86_64                   1.4.16-10.el6                               epel                                    15 k
 nagios-plugins-mailq                       x86_64                   1.4.16-10.el6                               epel                                    18 k
 nagios-plugins-mrtg                        x86_64                   1.4.16-10.el6                               epel                                    31 k
 nagios-plugins-mrtgtraf                    x86_64                   1.4.16-10.el6                               epel                                    31 k
 nagios-plugins-mysql                       x86_64                   1.4.16-10.el6                               epel                                    41 k
 nagios-plugins-nagios                      x86_64                   1.4.16-10.el6                               epel                                    31 k
 nagios-plugins-nt                          x86_64                   1.4.16-10.el6                               epel                                    36 k
 nagios-plugins-ntp                         x86_64                   1.4.16-10.el6                               epel                                    48 k
 nagios-plugins-ntp-perl                    x86_64                   1.4.16-10.el6                               epel                                    18 k
 nagios-plugins-nwstat                      x86_64                   1.4.16-10.el6                               epel                                    39 k
 nagios-plugins-oracle                      x86_64                   1.4.16-10.el6                               epel                                    16 k
 nagios-plugins-overcr                      x86_64                   1.4.16-10.el6                               epel                                    33 k
 nagios-plugins-perl                        x86_64                   1.4.16-10.el6                               epel                                    14 k
 nagios-plugins-pgsql                       x86_64                   1.4.16-10.el6                               epel                                    33 k
 nagios-plugins-ping                        x86_64                   1.4.16-10.el6                               epel                                    36 k
 nagios-plugins-procs                       x86_64                   1.4.16-10.el6                               epel                                    67 k
 nagios-plugins-real                        x86_64                   1.4.16-10.el6                               epel                                    32 k
 nagios-plugins-rpc                         x86_64                   1.4.16-10.el6                               epel                                    16 k
 nagios-plugins-sensors                     x86_64                   1.4.16-10.el6                               epel                                    14 k
 nagios-plugins-smtp                        x86_64                   1.4.16-10.el6                               epel                                    70 k
 nagios-plugins-snmp                        x86_64                   1.4.16-10.el6                               epel                                    70 k
 nagios-plugins-ssh                         x86_64                   1.4.16-10.el6                               epel                                    32 k
 nagios-plugins-swap                        x86_64                   1.4.16-10.el6                               epel                                    32 k
 nagios-plugins-tcp                         x86_64                   1.4.16-10.el6                               epel                                    38 k
 nagios-plugins-time                        x86_64                   1.4.16-10.el6                               epel                                    32 k
 nagios-plugins-ups                         x86_64                   1.4.16-10.el6                               epel                                    35 k
 nagios-plugins-users                       x86_64                   1.4.16-10.el6                               epel                                    29 k
 nagios-plugins-wave                        x86_64                   1.4.16-10.el6                               epel                                    14 k
 net-snmp-utils                             x86_64                   1:5.5-49.el6                                rhel-x86_64-server-6                   173 k
 postgresql-libs                            x86_64                   8.4.18-1.el6_4                              rhel-x86_64-server-6                   201 k
 qstat                                      x86_64                   2.11-9.20080912svn311.el6                   epel                                   162 k

Transaction Summary
==============================================================================================================================================================
Install      55 Package(s)
...

January 8, 2014

Virtualization with KVM on RHEL 6

Install KVM

$ yum groupinstall "Virtualization" "Virtualization Client" "Virtualization Platform" "Virtualization Tools"

Verify Installation

Check that kvm kernel modules are loaded. There should be two: kvm and either kvm_intel or kvm_amd.

$ lsmod | grep kvm
kvm_intel              54285  0 
kvm                   332980  1 kvm_intel

If kvm_intel or kvm_amd is not loaded, try to load it. E.g. kvm_intel.

$ modprobe kvm_intel

In my case, I recieved an error. And in the general RHEL log file, i recieved the following.

$ cat /var/log/messages
Jan  4 20:39:23 tester2 kernel: kvm: disabled by bios

So for my HP Elite 7500 I had to enable Virtualization in the BIOS:

  1. Computer Setup
  2. Security
  3. System Security
  4. Virtualization Technology (VTx/VTd): Enable
  5. Save and reboot

NOTE: REALLY TRY TO FIX THE PROBLEM OF NOT HAVING LOADED THE KVM_INTEL OR KVM_AMD, OTHERWISE WILL THINGS GO REALLY SLOW.

Start KVM

$ service libvirtd start

Prerequisite RHEL DVD

The easiest way to distribute the RHEL 6 DVD is to actually use the ISO. To make it accessible for the KVM, we place it under libvirt default images folder.

$ cp rhel-server-6.5-x86_64-dvd.iso /var/lib/libvirt/images/

Finally we need to restore the SELinux type for the ISO file.

$ restorecon -F /var/lib/libvirt/images/rhel-server-6.5-x86_64-dvd.iso

Install Guest Virtual Machine Graphically

The easiest way is to do it graphically with Virtual Machine Manager

$ virt-manager
If you had a kickstart file, you could here press TAB and enter the URI for the kickstart file.

Make sure you auto starts the default network. Otherwise you have to manually start it, every time you boot.

Since this is a virtual server, you can fill the entire virtual disk with RHEL.

A nice feature before writing the new partition table to disk is to preview. You do that with the option at the bottom.

Again the RHEL installation will stretch the entire virtual disk, so cleaning the Master Boot Record, MBR, is no problem. But if you were to install a system along with other OS, you must have good strategy how these OS can work alongside and be booted.

The most common installation is for server is Minimal and for client it is Desktop.

Install Guest Virtual Machine CLI

$ virt-install --prompt
What is the name of your virtual machine? 
virtual1.example.com
How much RAM should be allocated (in megabytes)? 
2048
What would you like to use as the disk (file path)? 
/var/lib/libvirt/images/virtual1.example.com.img
How large would you like the disk (/var/lib/libvirt/images/virtual1.example.com.img) to be (in gigabytes)? 
10
What is the install CD-ROM/ISO or URL? 
/var/lib/libvirt/images/rhel-server-6.5-x86_64-dvd.iso

Install Guest Virtual Machine CLI with Kickstart

You can automate the entire installation process of new virtual guest in line, with the help of a kickstart file.

To do this you need to extract the ISO file, for example an FTP server. On the FTP server

$ mkdir /media/rhel6

$ mount -o loop /home/magnus/Downloads/rhel-server-6.5-x86_64-dvd.iso /media/rhel6

$ cp -ar /media/rhel6/. /var/ftp/pub/pub/inst
$ virt-install --name virtual2.example.com --ram 2048 --vcpus 2 --disk path=/var/lib/libvirt/images/virtual2.example.com.img,size=20 --location ftp://192.168.1.3/pub/inst --extra-args "ks=ftp://192.168.1.3/pub/minimal-installation-ks.cfg"

Manage Virtual Machine

List all virtual machines.

$ virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     virtual2.example.com           running

Auto start of virtual machine.

$ virsh autostart virtual2.example.com
Domain virtual2.example.com marked as autostarted

Disable auto start of virtual machine.

$ virsh autostart --disable virtual2.example.com
Domain virtual2.example.com unmarked as autostarted

Stop virtual machine.

$ virsh destroy virtual2.example.com
Domain virtual2.example.com destroyed

Start virtual machine.

$ virsh start virtual2.example.com
Domain virtual2.example.com started

Remove Virtual Machine

$ virsh destroy virtual2.example.com

$ rm -f /var/lib/libvirt/images/virtual2.example.com.img

$ rm -f /etc/libvirt/qemu/virtual2.example.com.xml

January 7, 2014

Configure iptables for FTP Server, vsftp

Server Installation

$ yum install vsftpd

$ service vsftpd start

Now the ftp server is ready to be used. Lets create a simple text file in the root of the ftp server.

$ echo "Test" > /var/ftp/pub/test.txt

Configure iptables

Existing iptables rules.

$ iptables -vnL --line-numbers
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
2    12592   18M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
3        0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
4        0     0 ACCEPT     tcp  --  *      *       192.168.122.0/24     0.0.0.0/0           state NEW tcp dpt:22 
5        0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 

First we will add a LOG operation just before the last line in INPUT of iptables that rejects the incoming traffic.

$ iptables -I INPUT 5 -j LOG

Then we try to list the root of the ftp server. Which will fail, because we have not opened the firewall ftp port and which you will see in the log file.

$ tail -f /var/log/messages
...
Jan  7 21:13:59 virtual1 kernel: IN=eth0 OUT= MAC=52:54:00:51:e4:07:52:54:00:00:99:4a:08:00 SRC=192.168.122.1 DST=192.168.122.196 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=53779 DF PROTO=TCP SPT=38835 DPT=21 WINDOW=14600 RES=0x00 SYN URGP=0

Now lets open the ftp port in the firewall. We will insert the rule just before the log rule.

$ iptables -I INPUT 5 -m state --state NEW -p tcp --dport 21 -j ACCEPT

Now lets try to list the root again, which will fail. Now look at the log.

$ tail -f /var/log/messages
...
Jan  7 21:21:49 virtual1 kernel: IN=eth0 OUT= MAC=52:54:00:51:e4:07:52:54:00:00:99:4a:08:00 SRC=192.168.122.1 DST=192.168.122.196 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=18564 DF PROTO=TCP SPT=39862 DPT=29736 WINDOW=14600 RES=0x00 SYN URGP=0

You might now wonder why ftp is trying to open port 29736 and the reason is that vsftp is using passive ports to communicate. To fix this we need to add to add ftp filter rules to iptables. The relevant filter rules are found in the kernel module filter directory.

$ ls /lib/modules/`uname -r`/kernel/net/ipv4/netfilter/*ftp*
/lib/modules/2.6.32-431.el6.x86_64/kernel/net/ipv4/netfilter/nf_nat_ftp.ko
/lib/modules/2.6.32-431.el6.x86_64/kernel/net/ipv4/netfilter/nf_nat_tftp.ko

Now add those two filters two iptables

$ vi /etc/sysconfig/iptables-config

Now save your iptables new rules and restart iptables.

$ service iptables save && service iptables restart

And finally try to list the content of the root in the server again and this should work.

Scanning network for open ports with nmap command

Whenever you are configuring a firewall, you want to make sure afterwards that the firewall really is configured properly and to do that is to scan it. Below follows two great links that helped me.

http://www.cyberciti.biz/tips/linux-scanning-network-for-open-ports.html
 
http://www.cyberciti.biz/networking/nmap-command-examples-tutorials/

January 6, 2014

Precaution When Working With Remote Firewall (iptables)

Configure a remote machine's firewall, can be hazardous. So here is one way of making it a little more safer.

First backup current firewall rules.

$ iptables-save > /tmp/iptables.bak

Secondly create a script with all you firewall rules, which are well tested. Below follows an example for a stateful firewall.

#!/bin/bash          

# Delete all existing rules
iptables -F

# Accept all incoming traffic to loopback interface
iptables -A INPUT -i lo -j ACCEPT

# Accept all already establed/ongoing conversation
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Accept PING
iptables -A INPUT -p icmp -j ACCEPT

# Accept SSH in port 22, but only from LAN (192.168.122.*)
iptables -A INPUT -m state --state NEW -p tcp --dport 22 -s 192.168.122.0/24 -j ACCEPT

# Reject all other incoming traffic
iptables -A INPUT -j REJECT

# And the same thing goes for FORWARD
iptables -A FORWARD -j REJECT

# Make changes permenant
service iptables save

Now with help of the cron job like command at, we can schedule a resturation of the original firewall settings in for example 20 min. Or how long you think it will take to test and verify your new firewall configuration.

# at now + 20 min
at> iptables-restore /tmp/iptables.bak
at> <EOT>
job 4 at 2014-01-06 01:27

You exit the at editor with Ctrl+D. Now you can list you at command with:

# at -l
4 2014-01-06 01:27 a root

And if your firewall configuration all checks out good, you can delete the at job with:

# atrm 4

And if it dont, you just have to wait 20 min, before the old configuration is restored

Configure Stateful Firewall

iptables Configuration Files

Contains persisted kernel filtering rules.

$ /etc/sysconfig/iptables

Contains configuration for iptables.

$ /etc/sysconfig/iptables-config

List iptables filtering rules

$ iptables -vnL --line-numbers

options:

  1. v - verbose, if not used you will, e.g. for which interface the rule applies. Every system have atleast two interfaces: loopback and eth0.
  2. n - numeric values
  3. L - list

Basic iptables

iptables divides traffic into 3 catogories: INPUT, FORWARD and OUTPUT.

  1. INPUT. Is for incoming traffic.
  2. FORWARD. Is traffic passing through, typical scenario is a router.
  3. OUTPUT. Is outgoing traffic.

Each category (or formely chain) has a default action (or formely target): ACCEPT, DROP and REJECT is the most common.

Then for each category you define a list of rules. The list is strictly ordered, which means if rule number 3 of 6 matches, the looping of the list is ended and the action is applied, that is defined for rule.

Demo

Lets get started with an open system, where the default action is ACCEPT.

$ iptables -vnL --line-numbers
Chain INPUT (policy ACCEPT 7 packets, 508 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 4 packets, 464 bytes)
num   pkts bytes target     prot opt in     out     source               destination   

First we accept all incoming traffic to loopback interface.

$ iptables -A INPUT -i lo -j ACCEPT

ACCEPT all already establed/ongoing conversation.

$ iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

If you want to enable PING, add below.

$ iptables -A INPUT -p icmp -j ACCEPT

Accept SSH in port 22, but only from LAN (192.168.122.*)

$ iptables -A INPUT -m state --state NEW -p tcp --dport 22 -s 192.168.122.0/24 -j ACCEPT

If you need VNC it is working in port 5900, and accept only connections from LAN (192.168.122.*)

$ iptables -A INPUT -m state --state NEW -p tcp --dport 5900 -s 192.168.122.0/24 -j ACCEPT

Finally we need to add either a REJECT or DROP rule at the end since our default action to ACCEPT incoming traffic.

$ iptables -A INPUT -j REJECT

And the same thing goes for FORWARD.

$ iptables -A FORWARD -j REJECT

If you have above typed some typo, e.g. line number 4, you delete that row with.

$ iptables -D INPUT 4

And if you need to insert a specific rule at specific row, e.g. line number 4.

$ iptables -I INPUT 4 ...

Test

To test your firewall you can for example use nmap. Take notice though that you are not violating any policy when using a port scanning utility.

$ nmap 192.168.122.196

Make persistent

Write rules to /etc/sysconfig/iptables.

$ service iptables save

And if you really want to make sure that the new configuration is loaded restart iptables service.

$ service iptables restart

Log

Logging firewall activity is a great way to debug your configuration. But beware if used on a production machine connected to Internet, you should make room for a big log partition.

I will insert the log action just before mine INPUT REJECT, wich is for me on line 5.

$ iptables -I INPUT 5 -j LOG

Now you can tail the default syslog log.

$ tail -F /var/log/messages

January 4, 2014

How To Make a Bootable USB to Partition HDD

A great tool for partitioning a hard disk is gparted. You find a longer description here http://gparted.org/liveusb.php#linux-method-b.

In brief the steps are the following

  1. Download gparted-live-0.17.0-1-i486.zip from http://sourceforge.net/projects/gparted/files/gparted-live-stable/0.17.0-1/.
  2. unzip gparted-live-0.4.5-2.zip -d /media/usb/
  3. cd /media/usb/utils/linux
  4. bash makeboot.sh /dev/sdd1

Use Tree View in RHEL File Browser (Nautilus)

The default file browser mode in Nautilus is with icon, but that view requires quite a lot of clicks to navigate your files. A better view is the tree view.

To change to the tree view, is not that obvious. One would first guess that it can be changed from the View menu. But you have to open the Preferences window.

  1. Select List View for View new folders using.
  2. Select tab Behavior and click Always open in browser windows..

Finally close all Nautilus windows and restart.