unix sort on multiple columns

For the nth time today, i had to think how to do unix sort on multiple columns either based on numbers or alphabets. Fortunately, i had a coffee before, so i figured out that for sorting a file that looks like this:


chr01 1
chr01 10
chr01 2
chr01 5
chr01 8
chr02 9
chr02 9
chr02 19
chr02 98
chr02 92

you can run the command "sort -k1,1 -k2n,2 sorttest" from GNU coreutilities to get something that looks like this:


chr01 1
chr01 2
chr01 5
chr01 8
chr01 10
chr02 9
chr02 9
chr02 19
chr02 92
chr02 98

and run the command "sort -k1,1 -k2,2 sorttest" to get something that looks like this:

chr01 1

chr01 10

chr01 2

chr01 5

chr01 8

chr02 19

chr02 9

chr02 9

chr02 92

chr02 98

All commands were run using "GNU coreutils 8.12.197-032bb" compiled on September 2011 as part of the Ubuntu OS. 

Unix for foreach loop with range and vector

To loop through values in unix or linux bash shell one can make use of the "for" loop.

Example: Specify range

for i in {1..24};
do
echo $i
done

One can also use C style for loops like

for ((i=1;i<=25;i++));
do
echo $i
done

It can be used like the perl's foreach loop by specifying an array. Note that range can also be descending. 

for i in {1..24} 25 50 {3..4} 100 150 {5..2} qw er st {a..z};
do
echo $i
done

You can also have strings and string range not just numbers.

Unix tail bug?

The UNIX tail command is probably one of the most widely used Unix commands. There seems to be a "bug" if you want to call it that in its functionality.

Try running a tail -5 on a directory where the files have data being written into them continuously like log files. Something like "tail -5 *" which should get you just the last 5 lines from each of the files in the directory.

However, try running it a few times and you will be surprised to see that for few of the files more than 5 lines are displayed!!

If you are unlucky enough to use this in a shell script that required just 5 lines and no more, the script will keep failing few times but seem to work just fine at other times. A transient bug which can slide pass the best testing.

This is something that happens only when you use wild cards to specify the files. So presumably the error happens because the command fails to realize its already read a particular file. Using other non-standard utilities like since or multitail might solve the problem.The programmer could even become intelligent and read just the top 5 lines of the output of tail!! for each file.

More importantly is this a bug with Unix tail or with the wildcards or just a undesired side effect we have to live with? I could see this "bug" in my 9.8. Not sure if this thing is endemic to this flavor. When i tried the same in the GNU core utilities and opensuse, i have to use the correct version "tail -n 5 *" and this works fine. Its always better to have the GNU core utilities than using the standard utilities that come with some of the less frequently updated flavors .

Simple encryption method in Unix using grep command

You can encrypt text with the "grep" command in Unix. You need a file with lot of junk data, a regular expression that will be the key and the text that needs to be encoded(with one character in each line).

To encode the text do the following steps.

1)grep -f regularexpressionfile junkdatafile1>result1

• The regularexpressionfile and result1 file should have equal number of lines.
  
• regularexpressionfile cant begin with "^".

2)paste texttoencode result1>result2

3)grep -vf regularexpressionfile junkdatafile2>junkdatafile3

4)paste -d"\n" result2 junkdatafile3>encryptedtext

To decode the text do the following

1)grep -f regularexpressionfile encryptedtext|cut -c 1

Windows unix cut command - column cut mode

The second part of the Windows Unix Cut command implements the column cut mode of the Cut command. By specifying flags such as -c for column cut mode and -p for entire file cut mode, the program can be used to cut the file based on column and file respectively.

This functionality is available in Editplus which offers column select option, but the command line tool can be very useful while cutting enormous files. The latest code is given below:

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;

public class Cut {
public static void main(String[] args) throws IOException {
FileInputStream in = null;
FileOutputStream out = null;
try {
in = new FileInputStream(args[1]);
out = new FileOutputStream("cutoutput.txt");
if(args[0].equals("-p")){
int c,count=0,llim=Integer.parseInt(args[2]),ulim=Integer.parseInt(args[3]);
while ((c = in.read()) != -1) {
if((char)c=='\n'){
count =count-1;}
count++;
if(count>=llim && count<=ulim){
//System.out.println((char)c);
out.write(c);}
}
System.out.println("Total characters between range is "+(ulim-llim));
}
else if(args[0].equals("-c")){
int c,count=0,llim=Integer.parseInt(args[2]),ulim=Integer.parseInt(args[3]);
while ((c = in.read()) != -1) {
if((char)c=='\n'){
count =1;out.write('\n');/*System.out.println();*/}
count++;
if(count>=llim && count<=ulim){
//System.out.print((char)c);
out.write(c);}
}
System.out.println("Output to be viewed in Wordpad or higer only" );
}

} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}
}
}