VI Cheat Sheet

There are plenty of VI cheatsheets out there, this one is directed at commands that I find handy when editing/seraching logs and config files.

Command Effect
:set ic ignore case while searching
/ search for
:w write out to
:g//s/// global find/replace (replaces with
dd delete line
dw delete word
d) delete rest of sentence
yy yank/copy line
p paste (from copy or delete)
wq save and quit
q! quit without saving
#G go to line #
G go to the last line in a file
:g/^$/d delete empty lines
:g//d delete lines with in them
mx mark beginning of a block "x"
d'x cut from beginning of block "x" to cursor
:%s/find/replace/ replace every occurance of the word "find" with "replace"

If you're scanning through logs with a lot of redundant information in them, it's often handy to use the pattern matching line delete from above, that way you can get rid of a lot of the extra fluff that you're sure you won't need.

Begginner's Shell Notes

I wrote this forever ago and some of it is totally wrong. Don't read this or you'll get MORE STUPIDER

The following table contains a quick reference for commonly used shell commands. You should Also See Advanced Shell Commands for other helpful topics.

Desired Effect	Command
Beep	echo $'\a'
Copy From Remote Server	scp @:
Spawn a new command without losing command line control	&
Copy to Remote Server	scp @:
Delete a File	rm
Grep in Files with a Certain Name Patten	find -name *.env | xargs grep "value"
Grep Recursively	grep -R
Grep with a Regular Expression	grep -E "bluh|muh|zuh" *
Install RPM	rpm -Uvh
Kill a Proccess	kill -9
Make a Symbolic Link	ln -s
Present Working Directory	pwd
Repeat a Command at Regular Intervals	watch --interval= ""
Search Previous Commands	ctrl+r (or history | grep )
Set Rights	chmod [GXMMS:uga][GXMMS:+-][GXMMS:rwx]
Switch Users	su
Tar	tar cvzf
Untar	tar -xvf
Uninstall RPM	rpm -e
View Directories	ls -d */
View Files with Timestamps, etc.	ls -l
View Files with Timestamps, sorted by date	ls -lrt
View Info	which
View Mounts	df -k
View Processes	ps -aux
View Rights	ls -al
View What RPMs You Have Installed	rpm -qa | grep
Watch End of File for Specific Output	tail -F | grep
Wildcard	*
Wildcard (Digit)	?

In addition to these commands, it's also good to have scripts to automate searching over multpile servers, performing routine checks, etc.

Here's a sample grepping script for Sun2:

echo grepping for $1 in logs 0-9 on all servers
for i in 1 2 3 4 5 6;
do
ssh lsdev@gx-mmsc00$i grep $1 /var/lsurf/log/gx-mmsc/mmsc-server.log.?;
echo $'\a'
done;

It's also handy to have scripts that simply type out your ssh commands for you.

echo connecting to ssh shell01.sun2.lightsurf.net...
ssh shell01.sun2.lightsurf.net

Here's an example of another script from the Daily Health Checks section:

echo
echo -------------------------
echo -
echo - DAILY HEALTH CHECK
echo -
echo -------------------------
echo
echo files in /gx/msgs/retry:
ls -l /gx/msgs/retry/
echo
echo Number of files in /gx/dist/sart:
ls -l /gx/dist/sart/ | wc -l
echo
echo Files over 2 megs in /gx/dist/sart:
find /gx/dist/sart ( -path './proc' -prune ) -o ( -size +2000000c -exec ls -l {} \; )
echo
echo Files in /lsurf/local_file_storage:
ls -l /lsurf/local_file_storage/
echo

If you place your scripts in your ~/bin directory, you can run them from the commandline no matter where you are on that machine.

Clampin a Java Date Object (Rounding Date to remove hours, minutes, and seconds)

Man. It seems like there should be a better way of doing this, but for now... here's what I came up with:
private static final SimpleDateFormat noHoursFormatter = new SimpleDateFormat("dd-MMM-yyyy");
private static final SimpleDateFormat millisFormatter = new SimpleDateFormat("SSS-mm-hh-dd-MMM-yyyy");

public static Date pushToBeginningOfDay(Date date){
try {
return noHoursFormatter.parse(noHoursFormatter.format(date));
} catch (ParseException e) {
return null;
}
}

public static Date pushToENDOfDay(Date date){
try {
return millisFormatter.parse("999-59-23-"+noHoursFormatter.format(date));
} catch (ParseException e) {
return null;
}
}

Looping Over an Indefinite Number of Command Line Parameters

It's often the case that you'll have to do something on a big list of MDNs or other parameters. To expedite your processing, you can make a script to lop over any number of parameters and execute a command. This example will run an LDAP querey on every number you add as a parameter:

while [GXMMS: "$1" != "" ]
do
/lsurf/openldap-2.1.17/bin/ldapsearch -x -H ldap://localhost:9009 -b ou=consumer,o=sprintpcs mdn=$1
echo $'\a'
shift
done

so instead of having to paste in each number, I can just enter this command:

BulkLDAPSearch.sh 9178069377 9173098528

The key elements here are the while loop and the shift command, which pops the first argument off ot the parameter list so the next takes its place. The "echo $'\a'" bit makes the computer beep in between each iteration (turn it off before it gets annoying!).

Get The External IP of Any Machine You're On

If you ever need to figure out the extenal IP of your machine (/sbin/ifconfig only returns your internal IP) you can use the follwing command:

wget -O- "http://checkip.dyndns.org"

From your desktop you can simply his the webpage listed in the weget, but if you're logged into a machine on ref, you'll need to wget it. You should get something like the following in response:

--14:23:32--  http://checkip.dyndns.org/
          => `-'
Resolving checkip.dyndns.org... done.
Connecting to checkip.dyndns.org[GXMMS:63.208.196.105]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [GXMMS:text/html]

   [GXMMS:<=>                                                                  ] 0
            --.--K/s
            \\\Current IP Check\\
                \Current IP Address: 63.215.196.180\
            \
   [GXMMS: <=>                                                                 ] 106
   103.52K/s

14:23:32 (103.52 KB/s) - `-' saved [GXMMS:106]

This is the bit you're interested in:

Current IP Address: 63.215.196.180

Using sed

A crash course in sed!

Often times, you'll end up with a file that isn't quite the format you need. Sed is a powerful tool that allows you to do a global find and replace from the comfort of your command line.

Take this example:

I've done a grep and piped the output to a file. I'd like to strip all lines in a file down from this:

/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:19:351] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:77
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:19:515] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:80
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:19:868] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:85
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:20:46] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:88
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:20:748] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:78
...

To this:

77
80
85
88
78
...

Bear in mind that the file I'd created wsa well over 1000 lines and hand-editing was not an option. A quick call to sed, and I had the file I was hoping for:

sed "s#.*call done time:\(.*\)#\1#g" calltimes.txt > justnumbers.txt

This same line works with any string within the ( ) markers.

h3: Search and Replace: sed "s#...#...#g

The three # chars separate the parts of the command. The s is for search and replace. Then the first "..." is the regex that matches the pattern you want to replace and then the second "..." is the replacement string. The g means "replace globally (don't stop after one)."

In the replacement string, & means the entire matched text, and \1, \2 ... \9 means the particular submatches surrounded by ( ) in the match regex.

For example,

sed "s#\(.*\) blah\(.*\)#\1 \2#g"

run on "glug blah bug" would result in "glug bug". Also notice that it it whitespace sensitive.

If you want to append something to the end of a matched string, simply use the & in your replace string. For example

sed "s#.*#& man"

run on "hey" would result in "hey man"

If you have a file that has bad linebreaks that are showing up s text elements (such as ^M), you can get rid of them by searching for \r:

sed "s#\(.*\)\r#\1#g" fileWithBadBreaks.in > file.out

Alternately, you can just chop off the last character in the line by reading any number of characters, then one final character:

sed "s#\(.*\).#\1#g"

Here's a new one! This takes any message tracing line and removes the subject:

sed "s#\(.*subject=\"\)[^\"]*\(\.*\)#\1\2#g" lessthan.sart > what.txt;

That's it for the crash course. This should handle most of your cases without a problem. If not, I'll defer to the man pages.

Using Awk

Awk is a powerful tool for handling text files.

Here is an example of taking a file created with P4 filenenames and changing it to a script that will copy all the files listed in /tmp/sh pver to a secondary location:

awk '{print "cp gx-6.1.1"$1" SecondaryGX"$1}' /tmp/sh > out.sh

more later!

Automate Logins

If you want to log onto a machine without having to type your password over and over...

From your user home:

ssh-keygen -b 1024 -t rsa

then press enter like 3 times

cat .ssh/id_rsa.pub

put that into dest host:

~username/.ssh/authorized_keys