Friday, September 11, 2009
Thursday, April 16, 2009
VI Cheat Sheet
Command Effect
:set ic ignore case while searching
/
:w
:g/
dd delete line
dw delete word
d) delete rest of sentence
yy yank/copy line
p paste (from copy or delete)
wq save and quit
q! quit without saving
#G go to line #
G go to the last line in a file
:g/^$/d delete empty lines
:g/
mx mark beginning of a block "x"
d'x cut from beginning of block "x" to cursor
:%s/find/replace/ replace every occurance of the word "find" with "replace"
If you're scanning through logs with a lot of redundant information in them, it's often handy to use the pattern matching line delete from above, that way you can get rid of a lot of the extra fluff that you're sure you won't need.
Begginner's Shell Notes
The following table contains a quick reference for commonly used shell commands. You should Also See Advanced Shell Commands for other helpful topics.
Desired Effect | Command |
---|---|
Beep | echo $'\a' |
Copy From Remote Server | scp |
Spawn a new command without losing command line control | |
Copy to Remote Server | scp |
Delete a File | rm |
Grep in Files with a Certain Name Patten | find -name *.env | xargs grep "value" |
Grep Recursively | grep -R |
Grep with a Regular Expression | grep -E "bluh|muh|zuh" * |
Install RPM | rpm -Uvh |
Kill a Proccess | kill -9 |
Make a Symbolic Link | ln -s |
Present Working Directory | pwd |
Repeat a Command at Regular Intervals | watch --interval= |
Search Previous Commands | ctrl+r (or history | grep |
Set Rights | chmod [GXMMS:uga][GXMMS:+-][GXMMS:rwx] |
Switch Users | su |
Tar | tar cvzf |
Untar | tar -xvf |
Uninstall RPM | rpm -e |
View Directories | ls -d */ |
View Files with Timestamps, etc. | ls -l |
View Files with Timestamps, sorted by date | ls -lrt |
View Info | which |
View Mounts | df -k |
View Processes | ps -aux |
View Rights | ls -al |
View What RPMs You Have Installed | rpm -qa | grep |
Watch End of File for Specific Output | tail -F |
Wildcard | * |
Wildcard (Digit) | ? |
In addition to these commands, it's also good to have scripts to automate searching over multpile servers, performing routine checks, etc.
Here's a sample grepping script for Sun2:
echo grepping for $1 in logs 0-9 on all servers
for i in 1 2 3 4 5 6;
do
ssh lsdev@gx-mmsc00$i grep $1 /var/lsurf/log/gx-mmsc/mmsc-server.log.?;
echo $'\a'
done;
It's also handy to have scripts that simply type out your ssh commands for you.
echo connecting to ssh shell01.sun2.lightsurf.net...
ssh shell01.sun2.lightsurf.net
Here's an example of another script from the Daily Health Checks section:
echo
echo -------------------------
echo -
echo - DAILY HEALTH CHECK
echo -
echo -------------------------
echo
echo files in /gx/msgs/retry:
ls -l /gx/msgs/retry/
echo
echo Number of files in /gx/dist/sart:
ls -l /gx/dist/sart/ | wc -l
echo
echo Files over 2 megs in /gx/dist/sart:
find /gx/dist/sart ( -path './proc' -prune ) -o ( -size +2000000c -exec ls -l {} \; )
echo
echo Files in /lsurf/local_file_storage:
ls -l /lsurf/local_file_storage/
echo
If you place your scripts in your ~/bin directory, you can run them from the commandline no matter where you are on that machine.
Clampin a Java Date Object (Rounding Date to remove hours, minutes, and seconds)
private static final SimpleDateFormat noHoursFormatter = new SimpleDateFormat("dd-MMM-yyyy");
private static final SimpleDateFormat millisFormatter = new SimpleDateFormat("SSS-mm-hh-dd-MMM-yyyy");
public static Date pushToBeginningOfDay(Date date){
try {
return noHoursFormatter.parse(noHoursFormatter.format(date));
} catch (ParseException e) {
return null;
}
}
public static Date pushToENDOfDay(Date date){
try {
return millisFormatter.parse("999-59-23-"+noHoursFormatter.format(date));
} catch (ParseException e) {
return null;
}
}
Looping Over an Indefinite Number of Command Line Parameters
while [GXMMS: "$1" != "" ]
do
/lsurf/openldap-2.1.17/bin/ldapsearch -x -H ldap://localhost:9009 -b ou=consumer,o=sprintpcs mdn=$1
echo $'\a'
shift
done
so instead of having to paste in each number, I can just enter this command:
BulkLDAPSearch.sh 9178069377 9173098528
The key elements here are the while loop and the shift command, which pops the first argument off ot the parameter list so the next takes its place. The "echo $'\a'" bit makes the computer beep in between each iteration (turn it off before it gets annoying!).
Get The External IP of Any Machine You're On
If you ever need to figure out the extenal IP of your machine (/sbin/ifconfig only returns your internal IP) you can use the follwing command:
wget -O- "http://checkip.dyndns.org"
From your desktop you can simply his the webpage listed in the weget, but if you're logged into a machine on ref, you'll need to wget it. You should get something like the following in response:
--14:23:32-- http://checkip.dyndns.org/
=> `-'
Resolving checkip.dyndns.org... done.
Connecting to checkip.dyndns.org[GXMMS:63.208.196.105]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [GXMMS:text/html]
[GXMMS:<=> ] 0
--.--K/s
\\\Current IP Check\ \
\Current IP Address: 63.215.196.180\
\
[GXMMS: <=> ] 106
103.52K/s
14:23:32 (103.52 KB/s) - `-' saved [GXMMS:106]
This is the bit you're interested in:
Current IP Address: 63.215.196.180
Using sed
A crash course in sed!
Often times, you'll end up with a file that isn't quite the format you need. Sed is a powerful tool that allows you to do a global find and replace from the comfort of your command line.
Take this example:
I've done a grep and piped the output to a file. I'd like to strip all lines in a file down from this:
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:19:351] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:77
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:19:515] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:80
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:19:868] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:85
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:20:46] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:88
/var/lsurf/log/gx-mmsc/mmsc-server.log.0:[GXMMS:1-12-2006] [14:46:20:748] [GXMMS:FINEST] LDAPSPIdResolver.resolveSPId() LDAP call done time:78
...
To this:
77
80
85
88
78
...
Bear in mind that the file I'd created wsa well over 1000 lines and hand-editing was not an option. A quick call to sed, and I had the file I was hoping for:
sed "s#.*call done time:\(.*\)#\1#g" calltimes.txt > justnumbers.txt
This same line works with any string within the ( ) markers.
h3: Search and Replace: sed "s#...#...#g
The three # chars separate the parts of the command. The s is for search and replace. Then the first "..." is the regex that matches the pattern you want to replace and then the second "..." is the replacement string. The g means "replace globally (don't stop after one)."
In the replacement string, & means the entire matched text, and \1, \2 ... \9 means the particular submatches surrounded by ( ) in the match regex.
For example,
sed "s#\(.*\) blah\(.*\)#\1 \2#g"
run on "glug blah bug" would result in "glug bug". Also notice that it it whitespace sensitive.
If you want to append something to the end of a matched string, simply use the & in your replace string. For example
sed "s#.*#& man"
run on "hey" would result in "hey man"
If you have a file that has bad linebreaks that are showing up s text elements (such as ^M), you can get rid of them by searching for \r:
sed "s#\(.*\)\r#\1#g" fileWithBadBreaks.in > file.out
Alternately, you can just chop off the last character in the line by reading any number of characters, then one final character:
sed "s#\(.*\).#\1#g"
Here's a new one! This takes any message tracing line and removes the subject:
sed "s#\(.*subject=\"\)[^\"]*\(\.*\)#\1\2#g" lessthan.sart > what.txt;
That's it for the crash course. This should handle most of your cases without a problem. If not, I'll defer to the man pages.