AWS EC2 Commandd Lines

1) **** Script to get a list of hosts from AWS for a Particular Region

#!/bin/bash
source /Users/name/ec2/bashrc_prod_east
ec2-describe-instances |grep Name |grep -v aws:autoscaling:groupName |awk '{print $5}'

$count='wc -l';
echo "Total $count hosts found on Prod East"


2)  Launch an Instance :
bash$ ec2-run-instances  ami-76817c1e -k  sunfod-dev-us-east-1-general --instance-type t2.small  -s subnet-058d0a6b


RESERVATION    r-f39fd8    1500819717
INSTANCE    i-12d47f9    ami-7817c1e        ip-10-224-12-98.ec2.internal    pending    sunfod-dev-us-east-1-general    0        t2.small    2014-09-11T03:39:36+0000    us-east-1c                monitoring-disabled        10.224.12.18    vpc-9d1f9bf3    subnet-058d0a6b    ebs                    hvm    xen        sg-34da035b    default    false
NIC    eni-3ac6764c    subnet-058d0a6b    vpc-9d1f9bf3    150081971781    in-use    10.224.12.18    ip-10-224-12-18.ec2.internal    true
NICATTACHMENT    eni-attach-34284b57    0    attaching    2014-09-10T23:39:36-0400    true
GROUP    sg-34da035b    default
PRIVATEIPADDRESS    10.224.12.18    ip-10-224-12-18.ec2.internal
bash$







3) Set up hostnames using ec2-create-tags:

bash$ ec2-create-tags ami-76817c1e i-12d47af9  --tag "Name=YourHostname"

4) Add Security Groups from AWS console or need to check command line options for that.

While loop to run command multiple times

while true; do ls ; done

How TO - Multiple grep Search Pattern

bash-2.03$ cat benchmark.out |egrep -e "SOLARIS26SPARC"
  P4D/SOLARIS26SPARC/2006.1/109255 (2006/10/17)
  P4/SOLARIS26SPARC/2006.1/104454 (2006/08/11)

bash-2.03$ cat benchmark.out |egrep -e "SOLARIS26SPARC" |egrep -e "17"
  P4D/SOLARIS26SPARC/2006.1/109255 (2006/10/17)
bash-2.03$

An abbreviated description of the Linux Boot up process

The LILO boot loader starts the kernel

The Linux kernel configured by lilo or rdev decompresses and must find and mount the root filesystem. If LILO or the kernel were not configured properly there can be a problem here.
The kernel after loading the root filesystem, starts the "init" program which may be located in /sbin/init. Reads /etc/inittab for configuration information.
/etc/inittab - init reads this file for configuration information. This file determines the starting run level and contains a line like:
si::sysinit:/etc/rc.d/rc.sysinit

/etc/rc.d/rc.sysinit - In this case this entry in "/etc/inittab" causes the script file "/etc/rc.d/rc.sysinit" to be run.
To add terminals or dial in modem lines on a system, add more lines in the /etc/inittab file, one for each terminal or dial in line similar to the following:
7:2345:respawn:/sbin/mingetty tty7

One of /etc/rc.d/rc or /etc/rc or /etc/init.d/rc is started from inittab which does initialization commands for boot at the set runlevel. This is done by running startup and shutdown scripts for various services to be run at the given run level.

/etc/rc.local is started from one of the startup scripts in the rc script file. This is where you should add custom features for your system.

getty - Init starts a separate getty (or mingetty) for each terminal for which logins are to be allowed. Getty is restarted by init after each user has logged out so new users can log in. Network logins are not done by getty but by a different deamon per way of logging in (telnet or rlogin handled by the inetd internet super daemon).

Getty outputs the welcome message in /etc/issue, reads the username and runs the login program. If the user is telnetting, the message in /etc/issue.net is first output.

The login program reads the password and runs the shell if the username and password are correct. The shell is based on entries in the /etc/passwd file and will run at the user's privilege level rather than with root privileges.

The shell (such as bash) runs the /etc/profile script file. However in the case of a system with shadow passwords, environment strings can be set first in a file called /etc/login.defs. Also a users resources can be limited in a file called /etc/limits. The $HOME/.bash_profile script is then run, but if it is missing /etc/.profile is run.
 

Pertinent files:

/dev/console - A virtual console device usually called /dev/ttyn where "n" is a terminal number.
/etc/ioctl.save - Contains the consoles ioctl(2) states
/var/run/utmp - A file where information on current system users is stored. The commands "w" and "who" can be used to display information in this file.
/var/log/wtmp - Has a record of all logins and logouts
/dev/initctl - Init's control fifo buffer file. Init listens on this fifo file for messages. Telinit uses this to communicate with init. The file initctl is not a script file.
/etc/passwd - Contains information about the user including the ID, name, home directory, and the path to the preferred shell program. If not using shadow passwords, this file may also contain user passwords.
In /etc/rc.d/rc0.d are kill and start scripts for various services. The kill scripts start with the letter "K" and the startup scripts start with the letter "S".

Sending a HTML file inline with Subject using  Sendmail command !

echo "Subject: Testing" | cat - file.html-old| /usr/lib/sendmail your-mail@mailid.com

Nawk to convert CSV file in HTML column/row format

/usr/bin/awk has a limit for the printf string of 398 characters.
/usr/xpg4/bin/nawk has no limit.

Nawk  to convert CSV file in HTML column/row format

nawk 'BEGIN{
FS=","
print  "MIME-Version: 1.0"
print  "Content-Type: text/html"
print  "Content-Disposition: inline"
print  "<HTML>""<TABLE border="1"><TH>SA TEAM</TH><TH>Host Name</TH><TH>Host ID</TH><TH>User ID</TH><TH>Login Shell</TH><TH>GCOS Field</TH><TH>Data Source</TH><TH>Domain Name</TH><TH>Acct Status</TH><TH>Lock Type</TH>"
}
 {
printf "<TR>"
for(i=1;i<=NF;i++)
printf "<TD>%s</TD>", $i
print "</TR>"
 }
END{
print "</TABLE></BODY></HTML>"
 }
' file-to-convert.csv > file.html

Sending HTML Mail by way of report !!!

Sending  HTML Mail by way of report !!!

I wanted to do below :

Step 1) Convert .csv comma seperated file to .html file

Step 2) Sending the HTML file as inline mail (had to use sendmail command , mailx wont support sending mail by way of HTML inline):

 

 

Step 1) Convert .csv comma seperated file to .html file

CSV file :test.csv: (This  need to convert to .html)
EQADM,edtsdb445s4s,832caef0,gpatmon,/bin/ksh,prete Erick Whindleton
EQADM,eqzd56s5s,83cd26bd,gpatmon,/bin/ksh,rete WHINDLETON,KED_RA,none
EQADM,eqzrtshs,8343f9a5,gpatmon,/bin/ksh,hshsrick Whindleton
EQADM,ed876b4c,832cab9c,gpatmon,/bin/ksh,sjsjs Erick Whindleton
EQADM,eq765qa25-phys,83c3802d,gpatmon,/bin/ksh,tsgsgsgWHINDLETON
EQADM,eq234canj1,80c9b573,gpatmon,/bin/ksh,ERICK C WHINDLETON
EQADM,eqznj10-phys,83193e0b,gpatmon,/bin/ksh,Erick Whinrtsrsksksk
EQADM,eqzdsnj2-phys,8338abba,gpatmon,/bin/ksh,ERICK C WHINDLETON

For this I used awk command :

bash-2.03$ awk 'BEGIN{
FS=","
print  "<HTML>""<TABLE border="1"><TH>one</TH><TH>two</TH><TH>three</TH><TH>four</TH><TH>five</TH><
TH>six</TH><TH>seven</TH><TH>eight</TH><TH>nine</TH><TH>ten</TH>"
}
 {
printf "<TR>"
for(i=1;i<=NF;i++)
printf "<TD>%s</TD>", $i
print "</TR>"
 }
END{
print "</TABLE></BODY></HTML>"
 }
' test.csv > file.html


Step 2) sending the HTML file as inline mail :

#!/usr/bin/ksh

export MAILTO=sendmailto@mailid.com
export CONTENT="index.html"
export SUBJECT="Html Format"
(
 echo "Subject: $SUBJECT"
 echo "MIME-Version: 1.0"
 echo "Content-Type: text/html"
 echo "Content-Disposition: inline"
 cat $CONTENT
) | /usr/sbin/sendmail $MAILTO

Configuring Apache to run perl

Configuring Apache to run perl :

I have below RPM installed: (Default Fedora 9 Sulphur)
----------------------------------
httpd-2.2.8-3.i386
mod_perl-2.0.3-21.i386

In Fedora 9 perl specification :
===============================

$cat /etc/httpd/conf.d/perl.conf

*)
LoadModule perl_module modules/mod_perl.so

*)
Alias /perl /var/www/perl

< Directory /var/www/perl >
SetHandler perl-script
AddHandler cgi-script .cgi .pl
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
Options +ExecCGI
< /Directory >

*)
< Location /perl-status >
SetHandler perl-script
PerlResponseHandler Apache2::Status
Order deny,allow
#Deny from all
Allow from all
< /Location >

in my /etc/httpd/conf/httpd.conf i have below line
Include conf.d/*.conf


Now put your perl script in /var/www/perl

test.pl
========
#!/usr/bin/perl
Content-type: text/html;
print 'mod_perl rock';
[root@linux perl]#

Point your browser ==> http//hostname/test.pl

Should show mod_perl rock!

rename file extentions ...

I wanted to rename all .txt file extensions as .html

$rename s/\.html/.txt/ *.html

Oh, also, this isn't a "standard Unix(TM) command, but it does come with most Perl installations that I know of and Perl is on most Unix machines that I know of.

Also i tried for loop :

for i in *.txt; do mv "$i" `basename $i`.html

But this renames a file file1.txt as file1.txt.html

anyone know how get avoid .html added after .txt ?

Try this :

for i in "${i%.txt}"; do mv "$i" "${i%.txt}".html; done

Exit Status - Shell script

Whenever a shell script is executed it checks the exit status
of command to verify if it executed sucessfully.

True = 0
False = Non Zero Value

You can check exit status value as below:

Eg:

#!/bin/bash

echo hello
echo $?
# Exit status 0 returned because command executed successfully.

lskdf
Unrecognized command.
echo $? # Non-zero exit status returned because command failed to execute.

$./script.sh

hello
0

./script.sh line 5 lskdf command not found.
127

Shell script if statement

Shell script using if statement to check if process is running:
================================================================

#!/bin/bash

sendmail=`(ps -ef |grep -v grep |grep sendmail)`
if [ -z "$sendmail" ];

then
echo "Sendmail is not running"

else
echo "Sendmail is running"

fi


WHat each line means:
---------------------
1) #!/bin/bash
This generates a Process ID for shell

2) PROCESS=`(ps -ef |grep -v grep |grep sendmail)`
Defining Variable Process:
# When we grep a process ps -ef |grep sendmail, the output result also shows "grep sendmail" line, in order to avoid that we use grep -v grep.


3) if [ -z "$sendmail" ];

-z option tells it should return a null value(no value)

Check for more : man bash

-n str True if string str is not a null string
-z str True if string str is a null string

while running df command it says read only filesystem

Taking further from my previous post,

I was not able to run df -k command :

It says , no filesystems processed:


Solution:

To remount root filesystem with read-write :
$mount -o remount,rw /

This will make a entry in /etc/mtab for read,write access to root filesystem

Using Linux Rescue

While trying to install Fedora core 8 on my Virtualbox(VM)
It got hanged after installation was done ...

Here's what I did:

1) Boot through Fedora core 8 CD

2) Select rescue installed system.

3) You will be asked to select the language, which defaults to English. Select the appropriate language and press OK to continue.

4)You will be asked to select the keyboard type, which defaults to us (USA). Select the appropriate keyboard type and press OK to continue.

5) do you want to start network interface - NO

6) Now it says, the rescue environment will find linux install and mount it under /mnt/sysimage - Select Continue

7) now it will say: your sytem is mounted under /mnt/sysimage and will reboot
if you exit - OK

8) now you you get a shell prompt




Once at shell prompt, Type :

i) chroot /mnt/sysimage
ii) fsck /dev/sda1 or fsck -p /dev/sda1

remove disk.
Reboot and check ... your system should be up !

Check OS Script ...

I had to check all hosts in a list , if its windows or unix.

Here's what helps identify whether it is Unix or Windows,
Note: Our Windows systems did not have ssh daemon running,


bash-2.03$ cat checkos.sh
#!/bin/sh
for host in `cat list_of_host`
do
if version=`(sleep 1 ; echo "") | telnet $host 22 | grep -i SSH`
then
echo "$host $version" >> HOSt-SSH-Running
else
echo "$host" >> HOST-SSH-Not-running
fi
done


The above scripts verify if the list of host in file has ssh daemon
running on port 22, in case if it finds running then it appends
that host to HOSt-SSH-Running and rest in HOST-SSH-Not-running.

Finding Common Occurence in Files and Writing to a New file !

I wanted to compare lines in 2 files and write things which
are common in a different file.

Eg,
File1 (has line like below)
===========================
host1
host2
host3
etc.....


File2 (has line like below)
===========================
host2
host3
host4
host5
etc ...

Which ever is common among file1 and file 2 should be written to file3

Grep does it :

grep -f file1 file2 > file3

file3 will have host2, host3 from above example shown under it.

Shell Script Comparing files ...

I have a file (file1) which is like

host1
host2
host3
host4
the list goes on............


Now I want the above lines in files to be compared with files under
/opt/new/*

File names under /opt/new are as below:
Dev
Prod
QA

And suppose host1 from file1 is found under Dev(file under /opt/new)
than it should write under a seperate file New-list as host1-DEV

Ans:

while read line; do echo ${line}-$(grep -l $line /opt/new/*); done < file1 > new-file

The above line reads from file1 and compares the list of hosts given under it with
all files under /opt/new/* and if the host on file matches on any of file.

Then it writes to a new file (new-file) as
host-filename, here host is the one listed in file1 and filename is
the file under which it found the host .

eg if it finds host1 under dev file (/opt/new)

It will write as host1-dev under new-file !

echo Command

Use echo command to display text or value of variable.

echo [options] [string, variables...]
Displays text or variables value on screen.
Options
-n Do not output the trailing new line.
-e Enable interpretation of the following backslash escaped characters in the strings:
\a alert (bell)
\b backspace
\c suppress trailing new line
\n new line
\r carriage return
\t horizontal tab
\\ backslash

Identify Zombie Process in Solaris and Clean it :

bash-2.03$ ps -ef |grep defunct (The one marked in dark below is Parent Process)
zasshr 8134 24291 0 0:00 <defunct>
...........................
..It will show u a list.........................



bash-2.03$ ptree 24291
24291 /opt/VRTSvcs/bin/CitiEquity/CitiEquityAgent -type CitiEquit
8134 <defunct>
8205 <defunct>

...........................
..It will show u a list.........................

To Find total count for the defunct Parent Process :

bash-2.03$ ptree 24291 |wc -l

158

To clean the defunct Process:

$ preap 24291
24291: exited with status 0

Solaris Faq

What is the command to do an interactive boot from the ok promt?
boot -i

How can i disable STOP+A utility on SUN machines,
which brings system into OK> prompt???.
In /etc/system set abort_enable=0 will disable STOP-A

Where are the templates stored that are copied into the user's home
directories for their personal customizations?
/etc/skel

What SPARC emergency keyboard sequence will take
the system to the ok prompt (forth monitor) but will send output to TTYA?
STOP+A

What file controls system wide password aging?
/etc/shadow

What flag used with patchadd will prevent a later back out by
preventing patchadd from backing up files? If this flag is used,
the patch cannot be removed.
patchadd -d

What file do you put the umask setting in?
The UMASK value for bourne and korn shell users can
be modified system wide by editing the "umask" entry in the
"/etc/profile" file. To change the default UMASK for the C shell,
modify the UMASK variable in "/etc/default/login" file.

Which of the following commands can tell you whether
packets are being delayed or dropped on your network?
spray
% spray -c 100 -d 20 0 -l 2048 pluto
The following example sends 100 packets to a host (-c 100)
with each packet having a size of 2048 bytes (-l 2048). The packets
are sent with a delay time of 20 microseconds between
each burst (-d 20). If you don't use a delay, you may run out of buffers.

Partition sizes can be set manually or from what configuration?
/etc/format.dat

Give the command that will display your default boot device.
eeprom boot-device & not printenv as suggested

How many different kill signals are there?
EXIT HUP INT QUIT ILL TRAP ABRT EMT FPE KILL
BUS SEGV SYS PIPE ALRM TERM USR1 USR2 CLD PWR
WINCH URG POLL STOP TSTP CONT TTIN TTOU VTALRM PROF
XCPU XFSZ WAITING LWP FREEZE THAW CANCEL LOST XRES RTMIN RTMIN+1 RTMIN+2 RTMIN+3 RTMAX-3 RTMAX-2 RTMAX-1 RTMAX


What software install group do you need to select in order to load the compilers?
sunwcprog
End User System Support Software Group (SUNWCuser)
Developer System Support Software Group (SUNWCprog)

What command can you use to display all of your groups?
groups - To display full list
id -a - To diaplsy full list of groups with group id numbers


What does this file /etc/path_to_inst contain and what is the
importance of this file
/etc/path_to_inst contains each device,
its instance name and number along with its physical name.

What is the command can reconfigure devices with out reboot?
devfsadm - solaris 8 onwards

How to restore a corrupted file sytem or trouble shoot bad
super blocks for a FS c0t2d0s5 ?
Well you can also just use the #FSCK -N /dev.dsk/c0t2d0s5
to get the back up block number and then using one of the numbers
that display #FSCK -F ufs -o b=32 can be executed.

How will you add a virtual IP address to a server. Given the interface
qe0 and IP 10.10.1.150
# ifconfig qe0:1 10.10.1.150 up
where "qe0" is an interface (e.g., le0) and N is a number between 1 and
. Removing the pseudo interface and associated address is done
with "ifconfig qe0:1 10.10.1.150 down".
As with physical interfaces, all you need to do is make the
appropriate /etc/hostname.qe0:X file.

Boot phases of Solaris Operating Environment are:
1.boot PROM
2.boot programs like bootblk,ufsboot
3.kernel initialization like loading modules
4. init phase

How do you determin which Run Level the sytem is running
who -r

How would you find out what version of Solaris is currently running?
Run the command showrev. There's a man page on showrev and
there are switches to help you do whatever you're trying to do


As a system administrator, how would you figure out the system
transaction is slow between the system you logged into and from the system you did telnet login?
-time

# Display the parent/child tree of a process ?
- ptree Example: ptree 1267

# Show the working directory of a process ?
- pwdx Example: pwdx 1267

# Display the processes current open files ? -
pfiles Example: pfiles 1267

Alternative for top command ? -
prstat -a

Listing USB devices on Fedora 7

lsusb command can list all usb devices found,
I had Fedora core 7(moonshine) installed and i did not find the command,

I had to install usbutils which brings the command:

[root@linuxbox ~]# yum -y install usbutils
Installed: usbutils.i386 0:0.71-2.1
Complete!

[root@linuxbox ~]# lsusb
Bus 004 Device 001: ID 0000:0000
Bus 003 Device 001: ID 0000:0000
Bus 002 Device 001: ID 0000:0000
Bus 001 Device 001: ID 0000:0000
Bus 001 Device 004: ID 07a6:8511 ADMtek, Inc. ADM8511 Pegasus II Ethernet
Bus 001 Device 003: ID 046d:c016 Logitech, Inc. M-UV69a Optical Wheel Mouse

[root@linuxbox ~]# lsusb -d 07a6:8511 -v <--- gives details
Bus 001 Device 004: ID 07a6:8511 ADMtek, Inc. ADM8511 Pegasus II Ethernet
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 1.10
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 8
idVendor 0x07a6 ADMtek, Inc.
idProduct 0x8511 ADM8511 Pegasus II Ethernet
bcdDevice 1.01
iManufacturer 1 ADMtek
iProduct 2 USB To LAN Converter
iSerial 3 0001
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 39
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xa0
Remote Wakeup
MaxPower 160mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 3
bInterfaceClass 0 (Defined at Interface level)
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0008 1x 8 bytes
bInterval 1