How is email spam found out?

Email spam is a not just a problem but more a menace with the increasing number of spammers that are active. Detection of spam mails from a collection of legitimate mails is a very important data classification problem. However, the impact of classifying a legitimate email as spam is more than filtering spam. Legitimate mail classified as spam is known as ham and is the result of improper classification. Hence, the threshold is selected in such a way as to avoid false negatives.

Various features of the mail such as the senders mail id (black listed or not), content of the mail in terms of occurrence of words common to spam mail like 'free', 'buy' etc are used to judge if a mail is spam or not.

Email Spam – detection and anti detection methods

Server side filter out spam such as spam assassin make use of rules which classify mail into spam and email based on occurrence of different types of words and features. It makes use of a neural network algorithm based method to do the classification. Spam Bayes provides tools for desktop utilities such as outlook, gmail, yahoo POP3 and IMAP and many other popular email clients. It is based on stasticial analysis of the email content.

The methods which rely on content based classification of spam have been very effective as the spammer has to deliver his spam message whose content is very different from a legitimate mail in many ways.

The empire fights back

The spam empire has fought back with many changes. Tools such as Spam checker will check the mail and suggest synonyms or changes to the mail to make it look less like spam and more like legitimate mail. Although the spammer can make such changes he cant make the mail ridiculously complex and incomprehensible. The new rules and new synonyms to escape the rules is an ongoing battle between spam and spam detectors. Gmail uses the same content based approach to decide what are relevant ads for that particular mail user.

Using images in place of the words that might give them away has been a popular method among spammers to avoid detection by such content methods. Having to recognize the characters and words in the images and checking them for spam is the obvious solution. This seems to be a never ending battle.

Would a rose by any other name smell as sweet?

Whenever there is huge rush to rename anything from a street to a nation, people often wonder whats the point in it all. Wouldn't a rose by any other name smell as sweet? Well name is important, or else we would not have a rename file option in windows.

Lets see if it would smell as sweet in the below cases in which renaming was done:

1. Hot Springs is Mexico changed its name to Truth or Consequences to get a popular radio program hosted from its city.
2. Clark in Texas (America) renamed itself DISH and got itself free satellite television for 10 years in the bargain.
3. Más a Tierra in chile renamed itself as Robinson Crusoe Island

The new names rock at least in these 3 cases. May be we should have thought a little more and come up with really catchy names for our Indian cities before renaming them. We dint even get any commercial benefits from renaming them either, just the headache of having to change the names in all the places. Here are few "cool" names that we could have used instead

Kolkata - A Coal Cat
Mumbai - Bye Mom
Pune - Puma
Bengaluru - U R Ben's Gal

Do let me know if you come up with something cool to rename to.

Finding patterns and rules in english words

I took a few hundred words in English starting with the letter "G" and few starting with letter "E". Hopefully i could get some pattern or rule in the way the letter after these letters was arranged. I managed to get as many as 51 rules for the way the letters appeared. The rules were all generated just based on the occurrence of a particular alphabet at a particular position.

The most effective and pervasive rule was if Letter2(A) => D(G). So in my sample of words, the words starting with letter "G" had lot of words with second letter as "A".

If i made the search more knowledgeable by putting in knowledge of vowels and consonants, more rules will turn up with better patterns. This might be even more interesting to try and find patterns in sentences.

Biodiesel and Photobioreactors

Biodiesel is a fuel similar to diesel that is obtained from oil rich plants such as reapeseed, soy, palm oil, sunflower or used cooking oils or phototrophic microorganisms. It has the advantage of producing lesser amounts of green house gases than fossil fuels. The main reason for the success of biodiesel is because it can be used without modification to engines and distribution systems.

Biodiesel has some of the same problems as bioethanol. It can start a competition for land with other agricultural crops, causing decrease in food supply or increase in food prices. Hence, the focus is on microalgae and cyanobacteria to produce biodiesel. Since, the microbes can grow in saline environments, they are not as much a threat to food crop cultivation. These methods for production of biodiesel from microbes are still experimental and slow. Developments in bioreactor design and genetic modification of the microbes may make these methods more viable in the future.

Growing the microbes required for biodiesel production requires photobioreactors as the microbes get their energy by photosynthesis. The photobioreactors can be mainly classified into open and closed systems. Open systems are lakes and natural ponds which can be used to grow the microorganisms. Closed systems are tubular or flat panel shaped bioreactors. The tubular bioreactors can be horizontal or vertical. Closed bioreactors have the advantage of not being contaminated and can be easily controlled. Open bioreactors have cost benefits.

The design of the bioreactor is driven by various factors such as light considerations, gas exchange, nutrient availability, product recovery and contamination. Proper mixing is required to ensure time for both dark and light reactions to occur. Cooling is required to remove the heat due to high irradiation. Too much light is observed by the cells at the surface of the culture and lost as heat, this is known as the shading problem. Genetic engineering changes to the cells to have smaller photosynthetic antenna seems to reduce this problem considerably.

Idea: The shading problem can be overcome by having cells of two different types in the reactor. The first type of cells do the light reaction and are positioned at the surface of the culture. Second type of cells do the dark reaction below the surface. The two cell types interact and exchange the products of their respective reactions through the medium.

Acessing windows drives from linux terminal

Many of us use linux and windows on the same machine. The new bread of linux installs does a wonderful job of mounting the windows partitions for us. We can open the files and work on them from the comfort of the GUI.

Some files and programs require to be run from the command line or terminal, especially in the case of linux. How do we access the windows drives on the command line? Since linux does not have the C,D and E drives impemented at the command line.

The drives can be easily accessed from the /media/ folder. Just type cd /media in the terminal and you will land in the media folder. This folder will contain the list of all drives mounted in the recent past. Many of these folders will not be accessible outside the 'root' group of users. To know which folders are your windows drives, just run a ls -lrt in the terminal. The windows drive folders will have read, write and execute permission for you. Just cd into that folder and acess all the files in window drives from the linux terminal.

First contact - fear of the aliens returns

The very first time we humans meet an alien race is a dream/nightmare scenario that has been played out many times in books and movies alike. Very existence of alien life forms has been postulated and hypothesized based on probability calculations. Considering the size and content of the universe, it might be expected that life could originate in other parts of the universe. Whether these life forms will just be microbes or intelligent life forms is a totally different possibility.

The scientific community has seen contact with an alien race as a positive thing, going as far as to send out signals into space with directions to our planet. The search for extra terrestrials Intelligence is underway with many different approaches being tried. Benefits of a friendly alien race aside, the possibility of life outside planet earth gives us hope of a second home which we may one day require.

Is it really wise to be doing this? what if the alien race just plunders earths resources? Concerns regarding this have been raised by the cosmologist Stephen Hawking, famous for his book "a brief history of time". Be it microbial or any other form of life we encounter, we may not be ready to face the complications that might arise. New and deadly diseases of alien origin could wipe out the entire planet. So do we just lay low till we are technically more advanced or do we try our luck in finding an alien?

Taking candy from a baby

"Taking candy from a baby" is defined as something that is easy to achieve in wiktionary and many other sites. However, is our interpretation correct? Come to think of it, there are lot of things that are easier than taking candy from a baby. Similes such as "easy as falling of a log" would be something thats just easy to achieve, as nothing mean is involved.

A more appropriate usage would be doing something mean thats easy at the same time. Does it just reflect that being mean is so much part of everyday lives that we are just not thinking about it at all? Just take a look at this video, you will get an idea of how mean it really is. The baby will surely cry for a while. Taking candy from a baby sure requires a strong will and a stone heart.

It seems there is also an explanation that the origin of this simile has nothing to do with candies or babies. Its interpreted as taking C.A.N.D (abbreviation for a type of cargo)from Bay B of a ship. Since ships generally use Bay B as the sick bay, its impossible to locate it, let alone take the cargo out of it. So basically taking candy from Bay B was impossible. A linguist (April 1st joke) explains how the expression originated on board ships and got to mean something else after coming ashore.

We may never be able to prove if this is just a result of liberal attitudes towards being mean or the result of not having Bay B in ships.