Find and Replace in Vi editor

Syntax for Text Substitution inside the vi editor is,

:[range]s[ubstitute]/{pattern}/{string}/[Flags] [count]

Following are the possible Flags

[g] -> replace all occurances in the line.
[c] -> Confirm each substitution
[i] -> make case insensitive

Case 1: Replace a text with another in the whole file

:%s/oldText/newText/g

where %s indicates all lines in the file.

Case 2: Replace a text within a single line

:s/oldText/newText/g

No range was specified therefore it replaces text in the single line.

:s/I/oldText/newText/g

where I indicates case insensitive

Case 3: Replace a text within a range of lines

:1,10s/oldText/newText/g

replaces text between line 1 and 10

Case 4: Replace text in only the first N lines

:s/oldText/newText/g 5

replaces the text in 5 lines from the current cursor position.

Case 5: Replace the whole word and not the partial word

:s/\<his\>/her/

The standard replace will make the word history with herstory. Enclose the word with < and > to ensure the search is for the whole word and not for partial one.

Case 6: Interactive Find and Replace

:%s/oldText/newText/gc

This will prompt for confirmation.

replace with newText (y/n/a/q/l/^E/^Y)?

y -> replace the highlighted word. After replace moves to the next word.
n -> does not replace the highlighted word. Moves to the next word.
a -> replace all the highlighted words.
l -> replaces the current highlighted word and terminates the find and replace.


Thanks

Shell Script to FTP

The following is a shell script to transfer file from one server to another server.
#!/bin/ksh HOST='172.16.4.41' -- Remote server ip
USER='oracle' -- Remote server User
PASSWD='******' -- Password ftp -n $HOST << EOF
quote USER $USER
quote PASS $PASSWD
bin
prompt off
cd /backup/RMANBKUP/spool/
lcd /backup/RMANBKUP/spool/
mget test*
bye
EOF

Thanks

nohup

When a user logs out of his session, the shell is killed. Hence any background processes started in the shell also gets killed. To avoid this and make background processes run even when the user logs out use nohup command.

When nohup (no hangup) command is prefixed with any command, the command continue to run even when the user who run that command logs out.

$ nohup compress comp.dmp &
$

The & indicates that the command runs in the background.

When you log out the session the parent is killed, in such situation the kernel assigns the orphan (child) to the system's init process (PID 1) the parent of all shells. Hence when the user logs out, init takes over the parentage of any process run with nohup.

If you run more than one command in a pipeline, then use the nohup command at the beginning of each command in the pipeline,

$ nohup grep 'ORA-' alert_prod.log & nohup sort &

Thanks

at jobs

Schedule jobs using 'at' command

at command is used to schedule jobs. It takes a series of commands as input and execute it at a particular time in the future. It differs from cron which is used for recurring executions (every hour, every day etc).

at offers the following keywords to be used for scheduling, now, noon, midnight, today and tomorrow. It also accepts the + operator. You can use hours, days, weeks, months and years keywords along with the + operator. For eg,

at 13
at 2.15pm
at midnight
at now + 1 day
at 3:30 January 26, 1986

To schedule job,

$ at 21:29 /export/rmanbkp.sh
$ at 13:15
at> /data_back/expprod.sh
ctrl-d
$

To view the scheduled jobs,

$ at -l
oracle.1266893940.a Mon Feb 22 21:29:00 GMT 2010
oracle.1269888300.a Mon Mar 29 13:15:00 GMT 2010

To remove a job

$ at -r oracle.1269888300.a

Note: There is no way you can find out the name of the program scheduled to run. Only job number is dislayed.

Thanks

Compress Files

To compress and uncompress the files in unix you can use the following commands.

1. Compress and uncompress
2. gzip and gunzip
3. bzip2 and bunzip2

Compress and uncompress

The output file has the extension as .Z

Example:

$ compress -v test.txt
test.txt: Compression: 59.74% This file is replaced with test.txt.Z

To compress file to another directory
$ compress -c proddmp.dmp > /data_back/proddmp.dmp.Z

$ uncompress test.txt.Z
$

To uncompress to another directory
$ uncompress -c proddmp.dmp.Z > /oradump/proddmp.dmp

gzip and gunzip

The output file has the extension as .gz

Example:

$ gzip test.txt
test.txt.gz

$ gunzip test.txt.gz

note: You can use gzip -d instead of gunzip. Both are same.

To view the compressed text file without decompressing use gzcat or gzmore if available in the platform.

$ gzcat test.txt.gz
$ gzmore test.txt.gz

To compress the folder

$ gzip -r /oracle/test
$ gunzip -r /oracle/test

bzip2 and bunzip2

The output file has the extension .bz2 The usage of bzip2 and bunzip2 is similar to gzip and gunzip.

Thanks

svmon

The svmon command captures a snapshot of virtual memory, so it is useful for determining which processes, user programs, and segments are consuming the most real, virtual, and paging space memory.

To display the total real memory pages that are in use and free and the paging space pages that are in use and free.

# svmon -G

To display the processes using the most real memory

# svmon -Pu -t x |grep -p Pid|grep '^.*[0-9]'

where x is the top x no. of processes.

To display the processes using the most paging space

# svmon -gP -t x |grep -p Pid|grep '^.*[0-9]'

In the O/P the pgsp indicates the number of 4kb pages reserved or used in the paging space by this process. Inuse column indicates the total real memory pages used.

To find out the most utilized segments

# svmon –S –t 10

With the -S option, svmon sorts segments by memory usage and displays the
statistics for the top memory-usage segments.

The different segments are

1. Persistent - Used to manipulate JFS files and directories
2. Working - Used to implement data areas of processes and shared memory.
3. Client - Used to implement some virtual file systems such as CD ROM etc.
4. Mapping - Used to implement the mapping of files in memory.
5. Real memory map – Used to access I/O space from virtual address space.

Thanks

lslv, lspv, lsvg

Some important commands

To display all the disks installed on an AIX machine

$ lsdev -Cc disk

To see what volume groups the disks belong to

$ lspv

This lists all the physical disks available on the system with their pvids assigned to it with the Volume Group name associated with this disk. "None" if NOT part of LVM.

To list information about the disks that belongs to the volume group.

$ lsvg -p

To list which logical volumes occupy the volume group

$ lsvg –l

To list all available volume groups

$ lsvg

Logical Volume Commands

To list all logical volumes under a volume group

$ lslv -l

To list the Logical Volume details with partition size , characteristics of volume and mirror copies.

$ lslv

To list the Physical Volume where the LV is located.

$ lslv -l

Note: vgname - volume group name and lv - logical volume

Thanks

iostat (Disk Utilization Report)

iostat (Disk Utilization Report)

To display report for all the disks

$ iostat -d 10 10

To display for only a subset of disks

$ iostat -d hdisk0 hdisk1 hdisk2 10 10

Important column

• If %tm_act (% of the time the disk was active) is greater than 70, then indicates a IO bottleneck.
• kb_read indicates the amount of data read from the drive.
• kb_wrtn indicates the amount of data written to the drive.
• tps indicates the no. of transfers requested per second to the drive. A transfer is an IO request to that drive.

Thanks

iostat

The iostat tool is used to detect the IO bottleneck.

Syntax

$ iostat [options] [drives] [interval] [count]

where
options - d, s (d - disk utilization report, s - system throughput report)

vmstat

Vmstat reports statistics about processes, Virtual memory, paging activity, faults, CPU activity and disk transfers.

Syntax:

$ vmstat [interval] [count]

Analysing the report

• The value of r should not be more than the number of CPUs consistently.
• If the value in b is high and sy is high then there is a possibility of paging or thrashing.
• A high fr:sr ratio (1-(fr/sr))*100 indicates performance bottleneck.
• If cy is greater than 0, then there is a severe memory shortage. Increase the RAM.
• If cs is too high then it indicates a performance problem. Tune time slices with schedo.
• If us + sy is greater than 80, it is CPU bound.
• If wa is high when id is not 0, tune I/O.

Thanks

Topas

The topas command is a performance analysis tool. It displays statistics such as CPU use, events and queues, memory, paging, network, disk performance.

$ topas –P –i 60

The o/p displays the top processes with the interval of 60 seconds. Get the process id and find the sessions in oracle.

Thanks

SAR (Context Switch)

To monitor the context switching activity

# sar -wP ALL 10 5

Context switching happens when a multi-process OS suspends running one process or thread and starts another. Context switching occurs during any of the following,

• A time slice expires.
• A process exits
• A process puts itself to sleep or in a stopped state.
• A real-time priority process becomes ready to run.

Thanks

SAR (Paging)

To monitor paging activity

# sar -r 10 5

The o/p has the following columns

Slots - No. of free pages (4kb pagesize) available. (672979 *4*1024) kb.
Cycle/s- No. of page replacement cycles per second.
Fault/s - No. of page faults per second.
Odio/s - No. of non-paging disk I/O per second.

Thanks

SAR (System Buffer Usage)

To monitor the system's buffer activity (not Oracle's buffer cache).

# sar -b 10 5

It provides the number of transfers per second between system buffers and block devices.

Bread/s, bwrit/s -No. of physical I/O. Gives overall write activity on server.
Lread/s, lwrit/s -No. of reads and writes from and to system’s buffer cache.
Pread/s, pwrit/s -No. of I/O on raw devices.
%rcache, %wcache-Buffer cache hit ratio for read and write requests. (100* (lread –bread)/lread )

Inference

• %rcache > 90% indicates the potential for bad disk I/O.
• %wcache < 70% indicates the potential for bad disk I/O.

Thanks

SAR (IO)

To monitor the disk usage

# sar -d 10 5

The o/p has the following columns

%busy- The portion of time the device was busy servicing a transfer request.
avque - The average number of requests in queue.
r+w/s- Number of read and write requests per second.
Kbs/s - The number of Kbytes transferred per second.
avwait- The average time each request waits in the queue before it is serviced.
avserv- The average time taken for servicing a request.

Inference

• %busy > 50 on a device indicates contention.
• Avwait should not be greater than avserv
• A high %busy and high avque indicates a big I/O bottleneck.

Thanks

SAR (Cpu) Individual

To monitor one CPU at a time.

# sar -P 2 10 5

The o/p displays the result for the 3rd CPU, 10 seconds apart, 5 times.

# sar -P 0,3 10 5

The o/p displays the result for the 1st and 4th CPU, 10 seconds apart, 5 times.

# sar -P ALL 10 5

The o/p displays the result for all the CPU, 10 seconds apart, 5 times.

Thanks

SAR (Cpu)

To monitor CPU usage

# sar -u 10 5

The o/p has the following columns.

%usr - Percent of CPU running in user mode (executing user code).
%sys - Percent of CPU running in system mode (executing OS code).
%wio - Percent of CPU running idle with a process waiting for block I/O.
%idle - Percent of CPU that is idle.

Inference

• A low %idle indicates CPU intensive job.
• A high %wio indicates a disk contention.
• A high %sys (>20) indicates a bottleneck. May be due to swapping or paging.
• A high %usr indicates applications not tuned properly or over utilization of CPU.

Thanks

ps (IO)

To display processes in order of I/O (PGIN)

$ ps avx |head -1 ; ps avx | grep -v PID | sort -rn +4 | head -10

PGIN represents the number of page ins caused by page faults. Because all AIX I/O is classified as page faults, this value represents the measure of all I/O volume.

Thanks

ps (Memory)

To display processes in order of real memory (RSS) use

$ ps avx | head -1 ; ps avx | grep -v PID | sort -rn +6 | head -10

The RSS value is the size of working segment and code segment combined together in memory in 1 KB units.

To display top 10 Memory Consuming Process (SZ)

$ ps -ealf | head -1 ; ps -ealf | sort -rn +9 | head -10

or

$ ps aux | head -1; ps aux | sort -rn +4 | head -10

The SZ represents the virtual size in kilobytes of the data section of the process. This number is equal to the number of working-segment pages of the process that have been touched (that is, the number of paging-space slots that have been allocated)times four. File pages are excluded. If some working-segment pages are currently paged out, this number is larger than the amount of real memory being used.


To display top 10 Memory Consuming Process (%MEM)

$ ps aux | head -1 ; ps aux | sort -rn +3 | head -10

Thanks

ps (Cpu)

To display top 10 CPU consuming process

$ ps aux | head -1; ps aux | sort -rn +2 | head -10

To display process in order of CPU time

$ ps avx | head -1 ; ps avx | grep -v PID | sort -rn +3 | head -10

This is the total accumulated CPU time for the life of the process and not the clock time.

To display processes in order of being penalized

$ ps -eakl | head -1 ; ps -eakl | sort -rn +5 | head -10

Penalized process are awarded less CPU time. Use nice to increase the priority.

To display processes in order of priority

$ ps -eakl | sort -n +6 | head -10

The lower value in the PRI column has the high priority.

To display processes in order of nice value

$ ps -eakl | head -1; ps -eakl | sort -rn +7 | head -10

By running the above commands you will get the process ids of the top resource consuming sessions. Use the pids in the following SQL statements to get the session details.

SQL> Select * From v$session s, v$process p
Where s.paddr = p.addr
and p.spid in ('pid', ‘pid’);

Thanks