Useful Unix Commands

Preprending a line to a file under unix, do echo "This is my line" | cat - filename > newfile. You can't redirect filename to filename itself , though.

A little shell trick to remove the first lines from a file is tail -n +x file where x is the number of the first line you want in your output. So to remove the first line from a file yould set it to +2.

Restarting apache: /etc/init.d/httpd restart

Restarting samba: The documentation of Samba tells you it'll reload it's config file (usually /etc/samba/smb.conf) when sent a SIGHUP signal, which translates to a kill -1 pid (see man signal -S 7 for the list of signals). You also can restart it, with the usual /etc/init.d/smb restart, just like you'd do with other daemons.

To zip/unzip .zip files unter unix, use zip/unzip instead of gzip/gunzip. See here. For cleaning up your directory tree under KDE, kdirstat is nice.

Using rpm:
rpm -U packagefile install (really update)

rpm -e package erease

rpm -qa query: list all installed packages

rpm -ql package query: list all files installed for package


When the installer complains that a library is missing, e.g.

error: failed dependencies:

        libblas.so.3   is needed by octave-2.1.35-4

go to rpmfind.net, and put in the name of the library to find the RPM that provides it, download that and install it. When looking for file names, try leaving out the .number at the end, to find the more up to date versions, too. Apache, by the way, is called httpd here.

Basic I/O redirection: see Unix Shell.
tee file	pipe input to output and to file (like a 'T' crossing: one way in, two out)
cmd1 | xargs cmd2	use the output of cmd1 as args for cmd2

Finding Stuff
find dir -name 'pattern'	files in subtree with pattern in the name. find . -name *.txt -exec grep -l 'myword' '{}' \; lists all text files that contain myword. The -l grep option prints the filename instead of the line. The quotes protect the {} and the search term from shell interpolation. Alternatively, one can use grep myword /dev/null, because /dev/null will be treated as a second file name, and when there is more than one filename, the name is printed by default (the -H option would do the same). The {} is the placeholder for the filename, the \; is needed to pass a ; through to find, so it knows where the -exec arguments end.
find dir -iregex 'pattern'	files in subtree with pattern in the name, using case independent real regexes (bash only).
locate pattern	prints from a nightly built index of the filesystem all names containig pattern. (fast)
which name	print full path to name prog. Useful if there are several versions istalled, and you want to know which one is first on the path.
man [-k] [-sn] name	print the manual entry for name. -k searches full text ("keyword"). -sn forces the search in a section. Section1 is the commands you usually need. Man pages are usually stored under /usr/(local/)man/man[1-6]

System information cat /proc/cpuinfo Find processor information under linux on the command line.
alias	all aliases
showrev, uname [-p]	processor and kernel information. showrev -p option prints list of installed patches. uname -a gives all kernel information. To find out which linux release is installed, on Redhat/Fedora, you can also try less /etc/redhat-release
df [-k]	free disk space [in kbytes] for all file systems. du dir prints the used space for a dir.
netstat [-n] [f]	[ip] adresses and status of the network
vmstat	cpu usage and memory
iostat	io information
nslookup ip-address	domain name bound to the ip-adress
ps [-elf]	active processes. -elf prints nearly all of them in long form (and is easy to remember).
top	eternally print active processes until ^C
crontab -l	cron jobs for current user

Displaying and formatting
Whenever input is expected, it can be given as a filename afterwards (command file.txt) or fed from another command via a pipe (command-1 | command-2).
grep[-i][-v] pattern	print [case-independend] all lines in input [not] matching the pattern. Patterns . * [] ^ $ behave normally, + ? | () have to be escaped with \. \w is word (alpahanumeric), \W non-word, \b word-boundary, \B non-boundary. Inside [] [:alpha:]=a-zA-Z [:digit:]=0-9.
more or less	paginate input
head [-n]	prints first few [n] lines of file.
tail [-n]	prints last few [n] lines of file.
cat	print input
wc [-l]	count bytes, words, lines [only]
dos2unix	remove all \r from file (these are in there because on dos, newline is \n\r instead unixe's \n).
strings	extract ascii srings from binary files. Take a look at Word or kernel error messages.

Compressing and expanding
Usually, since FTP cannot recursively get or put dir trees, these are tape-archived (tar'd) into one file, compressed with compress(.Z) or gzip (.gz), so the result is a filename.tar.gz file. This in turn has to be uncompressed and extracted to be used. Beware: DOS clients do not like two dots and tend to save this as filename_tar.tar or filename_tar.gz.
tar c[v]f filename.tar dir	[verbosely] roll dir into tar ball. Be careful that dir does not end in an / as it does with auto-completion.
tar x[v]f file.tar	[verbosely] unroll tar ball.
compress file	compresses the file into its file.tar.Z version.
uncompress file.Z	inflates back to the file version. You can also use gunzip for this.
gzip file	compresses the file into its file.gz version.
gunzip file	inflates back to the file version.
gunzip < file | tar xf -	unzip and untar. I think the < is needed as then the data is read from a stream, and thus no internmediate file is written?
unzip file.zip	unzips a .zip file
gtar xzf file	unzip and untar, even shorter.

File and directory handling
ln [-s] real symbol	create a [sym]link to another file. (Symlinks may span file systems). If you provide a dot for the symbol, the last name in the dirlist will be used, eg: ln -s /usr/local/pub . creates a link named pub
rm [-R] expr	recursivly remove all files matching expr. recursive version also removes dirs.
cp file target	copy a file.
mv source target	moves (or renames, depending on your pov) file or dierectory. To move a dirctory tree into the working dir type (eg): mv ~/texts/books . To move everything fro m a dir to another: mv * another/*
ls [dir]	list files in dir. common options are -F (append symbols for: dir, link etc), -A (list all including ones starting with .) -l (long with all info).
mkdir name / rmdir name	make or remove directories.
touch file	update the date on the file or create a new one with 0 bytes if it did not exist.
pwd	print present working directory

Process handling
kill [-9] PID	[thoroughly] kill process with PID.
command &	run process in background.
^Z	suspend process
bg	send suspended process to background
fg [%n]	fetch process with id n back to foreground.

Shell scripting
eval	Read the arguments as input to the shell and execute them as commands. Usually used to execute commands generated by command or variable substitution (because the result is the input string that is eval'd.
test	Implicitly used to evaluate conditionals.
expr	Calculate mathematical expressions.
exec	Read the input and execute it instead of the shell in the current process.

DOS vs Unix commandline

I did have to work on Windows and Unix in parallel for pretty much my entire coding life, where I had a Windows machine as workstation, and the servers were some dialect of Unix. In this situation, the following may be useful. This doesn't want to be complete.

Operation	Windows	Unix csh-based (csh, tcsh)
initialisation	NT: set to the values in "System Properties/Environment", DOS: set in autoexec.bat	set in /usr/local/env/.cshrc (sometimes /etc/.cshrc), followd by ~user/.login if login shell.
I/O redirection	cmd <in-file >out-file-new cmd >>out-file-append stderr cannot be redirected, always goes to screen.	dito.
filename expansion	? single char, * any number of any (including the dot at the beginning of filenames). eg dir test*.doc will find all files starting with test and the extention doc. dir test* and dir test*.* are the same.	dito, but * doesn't match a dot when it's the first char in a filename (as such files are used as system ressources.). Mask meta-chars with \.
whitespace protection	cmd "with blank"	cmd 'with blank'
Process piping	with cmd1 | cmd2	dito
Version	winver	showrev
environment setting	set [var=[string]] sets env vars. Note: no space behind =. Without var, all vars are shown on stdout. set vars locally for a skript between setlocal and endlocal . each setlocal must be freed by the endlocal before skript end.	set var = text with mandatory spaces. setenv name text sets and exports variable. set ! (bangs) in text are replaced by incremental numbers. array vars are defined as set var = ( foo bar baz )
path	path=newpath;%path%	set path = ( /bin /usr/bin /usr/local/bin ) path is defined as an arry.
directory listing	dir	ls
help option	command /h	man command or command --help
file identity check	comp	cmp
file difference comparison	comp f1 f2 for same sized file (default binary) fc f1 f2for text files.	cmp
file length
grepping	find findstr with regexen	grep
timing
size check
view users		who -al, whoami, groups
view host	hostname	hostname, showrev
view user on remote host	finger usrname@hostname	dito
processes		ps -elf
jobs		jobs
rights		chown, chgrp
file attribs		chmod
scheduled execution	at	cron
Sortieren	sort	sort
Output paging	more use pipeing to page output from other programs, file redir and name expansion to page contents of files.	more
html downloading	wget	wget
free memory	mem
show text file contents	type f1 [f2 ..]	cat
route tracing	tracert	traceroute
internet IP-name lookup	nslookup	nslookup
show net connections	netstat
ping if computer is on network	ping	ping
printing	lpr	lp
prompt style	prompt $p$g$p = pfad, $g = >	# superuser, % normal, set prompt = "`hostname`:` pwd`>"
variables	args for batches are stored in %1 to %9. environment var contents are accessed like this: %VARNAME% Expansion works here too, i.e %* means a list of all %1 to %9.	$var for normal vars or full arrays, $array[2-4] for array slices, $#array number of elements, $?var if var is defined 1, else 0.
executable search path	path without param shows path. entries separated by ;(semicolon). append old path with path newdir;%path%

shell

Since /bin/sh is available on every Unix system, it is the shell of choice for writing scripts, even while it has much less power than the other shells. Therefore it is a good idea to learn the basic syntax. while man sh will give you the full story, here are some of the most common constructs. Probably you'd be more portable by just learning perl and using that everywhere. The shell can be used interactively or as a script.

Flow control for if for var [in word-list] do cmds done if cmds then cmds [elif cmds then elif_cmds] [else else_cmds] fi case while case word in pattern1) cmds;; .. patternN) cmds;; esac while cmds do cmds done Tests What irks me most is the shitty syntax for conditions. Here my personal back-breakers: conditions test expressions with the syntax [ expression ] (note the spaces inside the brackets!) logical operators: and is -a, or is -o, not is !. All have to be set apart by space! Or can also be emulated by [ test1 ] || [ test2 ] if [ ! -e file ]) operators for file tests: -r file (readable), -d file (directory), -e file (exists) string comparisons:= (string equal), != (strings different). May not have space around them! numeric comparisons:-eq (numbers equal). Note the string/number thing is the other way round than it is in Perl, also note the - in -eq there must be a semicolon before the keyword that indicates the start of the dependent block, if both are on the same line: like if [ -e file -a -r file ]; then echo "Yep"; fi or for x in ABC*.txt; do sort $x; done Your usual constructs, with fi etc instead of blocks. Cmds means a list of commands. Bracketed parts are optional. Keywords must start on a fresh line, or after a semicolon. Patterns can use filename expansion (without the exeptions), word is usally a variable. For without in uses the position parms. You can use continue and break with loops.

Basics Commands Each command can be given arguments during its invocation. It turns into a process, reads data from standard input, outputs results to standard output, errors to standard error. Upon finishing it returns an errorlevel into $?, which is 0 if successful, positive otherwise. Filenames Any filename is possible, as long as the script has the magical #!/bin/sh as the first line. Conventional is name.sh For mathematical expressions, use the expr command. Remember to escape special characters (like * for multiplication as \*).

Special characters and Names I/O redirection cmd<handle redirect input to handle cmd>handle redirect output to handle cmd>>file append to file &0 handle for STDIN &1 handle for STDOUT &2 handle for STERR stream>&stream redirect stream to stream 2>file redirect STDERR to file >&2 redirect output to STDERR 2>&1 redirect STDERR to STDOUT cmd1|cmd2 pipe output of cmd1 to input of cmd2

Regexen for filename expansion * any number of chars, exept leading . and /., / ? one char [ac-e] char a, c, d, or e. [!ab] not char a or b

User-defined variables To export variable names from the script to the environment use export (instead of setenv like in csh). varname='value' Assignment of value to variable. No space around equals sign! The quotes are optional for simple words. varname= assigns "" (empty string). Example for settimg and exporting DISPLAY: DISPLAY=10.203.1.101:0.0; export DISPLAY $varname Using a variable. ${varname} Using a variable, braced form if end of varname is ambiguous. Is undef if no assignment happened. ${varname:-word} Using a variable, or word, if it is undef or empty. ${varname:=word} Using a variable, after assigning word, if it is undef or empty. ${varname/pattern/string} Replace pattern with string in the value of varname.

Automatic variables $1 to $9 args (params) 1-9 for the script. $0 the script name $# number of parameters $@ the list of all parameters as a list $* the list of all parameters as one string $? errorlevel of last command $! PID of last background process $$ PID of current process Quoting '...' noninterpolated string "..." $name, `cmd` and \ interpolated string `cmd` command substitution replaces command with its output

Various # turns the rest of the line into a comment cmd& run cmd in background cmd1 && cmd2 cmd2 only executes, when cmd1 returns success cmd1 || cmd2 cmd2 only executes, when cmd1 returns failure \ Escape. Concatenates lines.

Examples Quoting/Redirection chmod 755 `find . -type d` change rights for dirs under . rm `find . -name "*.html~"` remove all .html~ files in dir tree

 find . -name "*.html~" -exec rm {} \; ditto, but can handle whitespace in file names 

bash

To write special characters under bash, you can use the usual shortcuts, like \t for tab, when you quote them with single quotes lead by a dollar sign, such as this: $'\t'. This will then generate a tab character.

Gnarly Installation stuff

You can find all correctly intalled and documented perl modules with

perldoc perlmodlib

perldoc perllocal

The former shows the initially installed ones, the latter the locally installed ones.
There may be "incorrectly" installed modules (where someone just dropped the .pm file into your @INC path, so you can use the module, but have no documentation). You can find all modules via:

find `perl -e 'print "@INC"'` -name '*.pm' -print

(See also the perlmodlib manpage.)

cut, join, sort -- aping SQL with the shell

A neat little way to find out how many different words are in a certain column in a tab-delimited file (in this example the first column) under Unix is cat mydata | cut -f1 | sort -u | wc -l

Another useful tool for text manipulation is join. It joins sorted text files based on the first field, similar to an SQL join based on key equalities (or can show you lines missing from either file).

Hompage must-haves

Research in website design, as narrated by Phil Greenspun has shown that any home page should offer three features:

1. a navigation directory to the rest of the site
   
2. news
   
3. search
   

For me that's three out of three, now ;-) Also you should ensure that interior navigation answers the following questions:
Where am I? (breadcrumbs) Where have I been? (moot point, back button) Where can I go? (links in standard colors).
You should include a logo or site name in the upper left-hand corner of each interior page, linked to the site home page, too, so users can know if they are still on your page. Check your links, to make sure they are still alive. Linklint is a good option.

Windows Arcana

To print under NT from the commandline (MS-DOS) to a network printer, you have to tell your machine that it is a shared printer first with net use lpt1: '\\hostname\printername' /PERSISTENT:YES. Of course you can redirect to another device, but lpt1: is the default, so you don't have to mention it when using the print command. You can find out all the hosts on your network with net view and the names of all the shares on any host with net view hostname. The /PERSISTENT:YES option makes the change survive a system shutdown.

You then can print to the printer using the print filename command. Be careful: if you're printing UNIX text files, print will not insert the DOS-Textfile format \r's its expecting, and you'll get your text shifetd of the right side of the page. If printing binary stuff, you just get ugly special chars, of course.