Friday, November 20, 2009

More Xargs

just learned the coolest xargs trick today. normally xargs can only represent one input symbol using the -I, but you can make it multiply by combining xargs with "sh -c". example:

ls --format=single-column | xargs -n1 sh -c 'g++ -g $1 -o ${1%.cpp}' -

takes all files from the current directory, each line contains 1 file name, pass it to xargs, which executes sh -c 'g++ -g $1 -o ${1%.cpp}' - where the file name will be used as the $1 variable for the shell, - is used as placeholder for $0, and ${1%.cpp} shrinks the file of .cpp string at the end if it contains it. sh is used to execute an command if -c is used

This way we can access multiple variables with xargs, each can be placed in different locations of the command we want to execute.

Wednesday, November 18, 2009

xargs xargs xargs.. ??

I was learning about how to use xargs today.. quite an interesting unix tool. What it lets you do is to trigger a command on each input simple you received from input stream stdin. for example:

ls -l | awk '/.*/ { print $8}' | xargs -n1 -I{} gcc -o {}.o -g {}

long list the current directory with detail, take the last column with awk which contains a list of names, take each name and execute "gcc -o .o -g on them

xargs can print the command that is being generated using the -t option, or -p option for you to confirm each command that will be executed on the file. The -I{} indicates a special symbol {} is going to be used for the location where the symbol name should be put into instead of the default at the end of the command. the -n indicate how many symbols you will be using for each command, -l indicates how many lines of symbols you will be using for each command

xargs called without any command will act the same as an echo command.

It is essentially an easy replacement for for loops in a shell script

Saturday, November 14, 2009

Shell Scripting

For the past few weeks, I was working in broadcom, developing a shell script for their automated nightly build emails. It's been a struggle, and here is what I learned about shell scripts, and its limit:

Things I learned:
1. "echo" command will always echo out a line, use -n to prevent that, -e to let echo interpret the \ characters, which is turned off by default
2. "read" is the command used to read values from user, or from a file. but read automatically interpret the \ values, which is the opposite of echo. you need to use -r to disable that functionality
3. some general review of "sed":
1) you can give multiple sed expression replacements using the -e before every replacement string
2) sed only reads from a file and produce output to int stdout, don't know if there is a way to work around it
3) the format for the regular expression: '[general purpose] / [search pattern] / [replacement pattern] / [output options and search options]'
4) general purpose can be s for substitude, d for delete and such
5) use () in search pattern, those represent a single unit if you wish to output those patterns in the braket using \1 to \9 in the replacement pattern
6) . * ( ) ^ $ are all special parsing characters in search pattern, you need to use \ to indicate it is not otherwise
7) you can use & in the replacement pattern to indicate all the stuff that found to match your search pattern
8) in output options, g means global, which means sed does not quit until all matches are found, p means print the replacement pattern, w means write the replacement to a file indicated
9) sed has an option of -n, which means does not produce any output, normally, sed will put all characters not matched into the stdout, -n prevent that, using p in the output options is the exception to the no output rule, which prints the replacement pattern only into the stdout
10) eval is bad, it only evaluates the immediate expression in the next symbol, and each symbol is terminated by space, not \n
11) IFS is a variable indicating the end of a line or a group of symbols representing a command, which is useful for parsing the symbols with a for loop
12) be careful of if command condition spacing, $var=value is not parsable while $var = value is!
13) if command options: -a = and, -o = or, -z means string is empty, -f means file exists, -d means directory exists,
14) functions in shell script are weird, first of all it does not indicate that it takes any variable, but they do and they can take as many as they want. $# tells how many variables are passed to a shell function. $1 to $n indicate each parameters being passed to a shell script. the function act just like a shell command, no brakets needed to put parameters in it

Limitation:
right now I am still stuck in trying to do this: having two variables, one holds the name of the other, and you want to use echo and eval to show the value held by the other variable. the problem right now is in eval, it does not parse anything after the second space, which if the value in the second variable has space, those other parts will be lost..

I wonder how to fix this

Something to try out:
I wonder how the function environments in shell script functions are generated...