Sunday, May 14, 2006

Daily Checklist ... ... ...

Check List for Server Safeguards :

Check if any rpms have been installed Lately :

[root@localhost ~]# rpm -qa --last |more
xinetd-2.3.13-2mdk Sat 13 May 2006 08:14:31 PM EDT
anonftp-3.0-31mdk Sat 13 May 2006 08:10:01 PM EDT
wu-ftpd-2.6.2-6mdk Sat 13 May 2006 08:07:00 PM EDT

Checking People Connected to a Specific PORT :
For Example (port - 21,22,25 etc ...)

I checked for Port 25

[root@localhost ram]# lsof -i :25

master 5761 root 11u IPv4
telnet 21932 root 3u IPv4
smtpd 21933 postfix 6u IPv4
smtpd 21933 postfix 10u IPv4

12532 TCP localhost:smtp (LISTEN)
106579 TCP localhost:4835->localhost:smtp (ESTABLISHED)
12532 TCP localhost:smtp (LISTEN)
106600 TCP localhost:smtp->localhost:4835 (ESTABLISHED)

Another way is to use TCPDUMP to see any active connections are made to
a Particular port :

[root@localhost ~]# tcpdump -l -i eth0 port 22

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes

This will list the IP addresses from where the connections are made.

11:28:17.137522 IP localhost.ssh > localhost.1460: F 1463822750:1463822750(0)
ack 1461555328 win 8192

11:28:17.138112 IP localhost.1460 > localhost.ssh: F 1:1(0) ack 1 win 8192

11:28:17.138133 IP localhost.ssh > localhost.1460: . ack 2 win 8192

The above shows that I have made a connection from localhost to localhost on port 22

Imagine Systems stops working suddenly, One thing that comes immediately
to our mind is what changed :

#find /ram -mmin -30

The above commmad will find Files that have been changed in /ram in the
last 30 Minutes

#find / -mtime -1

The above command will recursively list all the file from / that have changed in the last

Checking Logs

Check for Log Files created in /var/log , Files will be created as per applications
used by you.

Common Log files are

/var/log/boot.log --- Systems services that has been Started/Stopped
May 14 12:46:28 localhost xinetd: xinetd shutdown succeeded
May 14 12:46:35 localhost xinetd: xinetd startup succeeded

/var/log/messages --- Check Logins

May 14 12:50:27 localhost sshd[22453]: Accepted password for sriram
from port 3242 ssh2

/var/log/secure --- Check Xinetd Services

May 14 09:30:07 localhost xinetd[5396]: START: ftp pid=21491 from=
May 14 09:31:27 localhost xinetd[5396]: EXIT: ftp pid=21491 duration=80(sec)
May 14 11:15:32 localhost xinetd[5396]: START: ftp pid=21925 from=
May 14 11:17:13 localhost xinetd[5396]: EXIT: ftp pid=21925 duration=101(sec)

/var/log/auth.log --- Check Authetication Failures

May 14 11:15:32 localhost xinetd[5396]: START: ftp pid=21925 from=
May 14 11:17:13 localhost xinetd[5396]: EXIT: ftp pid=21925 duration=101(sec)
May 14 11:28:17 localhost sshd[22092]: fatal: Timeout before authentication for
May 14 12:50:27 localhost sshd[22453]: Accepted password for sriram from port 3242 ssh2
May 14 12:52:18 localhost sshd[22503]: Accepted password for sriram from
port 3245 ssh2


The program helps users to print out their bootup messages.

Checking Disk Space of a Particular Folder,Files

[root@localhost ram]# du -h -s Beatles/
27M Beatles/

Install Dsniff for Packet Sniffing

[root@localhost ram]# dsniff
dsniff: listening on eth0
05/14/06 16:58:47 tcp -> distrib-coffee.ipsl.jus.21 (ftp)
USER anonymous

05/14/06 17:35:24 tcp -> 202.41.x.x.21 (ftp)
USER xyz
PASS xyz123

Check for Current connections made FROM and TO Server

[root@localhost ram]# netstat -apln --inet
Active Internet connections (servers and established)

Will List the Complete details of From-IP-Port to To-IP-Port live connections.

Rejecting a Particular connections with out using IPTables

Say you do not want to reject request for a particular IP Address
Incomming and Outgoing

For Rejection:
[root@localhost ram]# route add -host 202.x.x.x reject

Again to allow:
[root@localhost ram]# route del -host reject

If for eg. if you want to block a particular site do this

[root@localhost ram]# ping
PING ( 56(84) bytes of data.

Now to block
[root@localhost ram]# route add -host reject

You can also block the Domain Name,
[root@localhost ram]# route add -host reject

after blocking try in your browser,

The message I got in my browser was,
"Connection was refused when attempting to contact"

To unblock Later
[root@localhost ram]# route del -host reject

Preventing DDOS Attack- Say your Load on the server is very high
For a Particular Service say HTTP- Port 80

Then you may do the Following :

1) At command prompt execute the below command

bash#netstat -lpn|grep :80 |awk '{print $5}'|sort

2) Check each block of ips.

Like let me say , that you have more than 30 connection from a single ip.
Under normal cases there is no need for that many number of connection
requests from a single IP. Try to identify such ips/networks from the list you get.

3) If more than 5 host/ip connects from the same network then its a clear sign of DDOS .

4) Block that ips/networks using iptables /Apf

[root@localhost ~]# iptables -I INPUT -s 202.87.X.X -j REJECT

If you have apf then just add the ips which you want to block in the file /etc/apf/deny_hosts.rules

5) Keep on continuing this process untill the attack on the machine gets reduced.

To Unblock
[root@localhost ~]# iptables -D INPUT -s 202.87.x.x -j REJECT

Suppose 202.87.x.x is website than that site will also be blocked.

Some Commands :

# lspci list all your pci devices
# netstat -arn show your network route information
# netstat -ap 2 | grep EST show established connections, updates every 2 sec
# netstat -Cr print routing information from routing cache
# iptables -nL show your current iptables configuration in numeric form
# ping -c ping X times.
# ping6 ping ipv6 addresses
# dmesg print or control the kernel ring buffer, bootup messages
# uptime check your linux servers uptime and load

Try this when your System Hangs - To Trace the Problems ?

Try the following:
1. Capture an alt-sysrq-t or alt-sysrq-p or alt-sysrq-b or alt-sysrq-m backtrace when the hang occurs.
2. Capture whatever is on the screen.
3. Look for kerlnel oops in the /var/log/messages or /var/log/syslog after reboot.
Note : sysrq is the Delete Key

Securing and Hardenning Linux Check this