Image Searching

Introduction

Search is a ubiquitous activity. Industry analysts believe that search is the gateway for many Internet activities. For example, when you purchase an item online, you often begin with a search for the best price. You also search to compare different model's pros and cons.

Amazon is a business built on search. You search for books by genre. Amazon has a feature that suggests books that you might find interesting. This is based on two search types - your previous purchase history, and the search histories of others whom have purchased the book you are considering to purchase.

GoogleNews allows you to search for news, and to create filters to automatically retrieve specific news stories for you. For example, if you want stories relating to your favorite sports team, particular city, industry, etc., create a filter with the name of the team, city, or industry and Google will return related stories.

Search is still in its infancy. There are no good methods for music searches. For example, I know the melody of a song, but I don't know the name. How can I input the melody?

How do we search for non-text related information? Image searching is probably the second most popular type of search next to traditional text-based search. How can I search my hard drive or the Web for all images that contain a blue shirt, or every image that contains buildings made of brick?

The remainder of this article discusses three interesting approaches:

• the Picassa 2 product from Google
  
• 
  
• tagging (using Flickr), and
  
• 
  
• A9 from Amazon
  
• 
  

Picasa2

What better place to start than at Google. Picasa2 (from Google) provides image search capabilities, the ability to quickly insert an image into a blog, email images, print images and so on. It also contains elementary functions to manipulate images. It is available free of charge for Windows and Linux-based systems (picasa.google.com).

Picasa2 begins by indexing all the images on your hard drive. You can initially search by file name.

The Locate on Disk feature is useful. For example, you have images stored on multiple drives. Once your search returns a list of images, just right-click on the image and select "Locate on Disk". This feature opens the folder where the image resides. This gives you quick access to the image plus the directory name.

Labels are also useful. These are tags. For example, you have a variety of images of
Boston
in multiple folders and drives. Which drives? Which folders? Tagging allows you to easily find these images. Tags are keywords that you associate with an image. In this example, "
Boston
" is an excellent tag. Other tags might be "2006 summer vacation",
Massachusetts
", etc. Once you create the tags, you can use them for search criteria. You don't have to worry about drives and folder names.

Search by picture color is a neat feature. Enter "purple" in the search box and Picasa2 returns all images that contain purple. Enter "orange" and images that contain orange are returned.

Picasa2 allows you to search by the following criteria:

• filename
  
• 
  
• captions
  
• 
  
• keywords
  
• 
  
• folder, label, collection names
  
• 
  
• picture color
  
• 
  
• camera maker/settings, and
  
• 
  
• date
  
• 
  

Picassa 2 is an excellent tool for anyone that works with graphics on a daily basis. It is fast and easy to use. Once you begin using it, you will wonder how you got along without it.

Tagging

Tagging is becoming one of the most popular ways to categorize information on the Web. This method allows people to enter keywords or tags that describe a particular piece of information. For example, a few years back I visited
Montreal
. If I use the name "Montreal" as a tag, entering that tag in Picassa2 or in a photo sharing website would return those images on my computer or from the Web that contain the tag "
Montreal
". The photo sharing site would also allow me to view other people's images that contain the tag "
Montreal
".

How well does this technology work for retrieving images? Tagging is both good and bad, or to be more specific, as good as the tags associated with the images. It is an example of the classic adage, "garbage in, garbage out".

I thought it would be interesting to see how well tagging worked on a variety of searches using the popular photo sharing site, Flick (www.flickr.com).

I began by entering the tag "
Boston
". As expected, it returned a multitude of pictures of the city of
Boston
. It also returned some other interesting photos. It returned a picture of one of the Boston Sports Clubs (a chain of sports clubs in
Boston
and in other cities in MA.). It also returned an image of the rock group
Boston
's self-titled album cover, a picture of a person associated with the Boston Organizer Meetup, a picture of people from the Boston WiFi Community forum, etc.

The tag "
Boston
buildings" returned 99% of what I had in mind. It did occasionally return a picture of a person and the ocean. However, in each case where the image returned was not what I expected, the tag for the picture was.

Let's imagine that you are writing an article about "purple dresses, the latest in fall fashions." I entered the tag "purple dress" into Flickr. Ninety percent of the images returned were of purple dresses. Some were dolls and teddy bears with purple dresses, but a purple dress it was. These results also included a manhole cover and some purple food.

I must admit that tagging worked better than I expected. I thought there might be more non-related images.

A9 (from Amazon)

A9 (www.a9.com) is a search engine from Amazon. (Search is a major component of Amazon's business.) It is a unique and worthy search engine in its own right. It also contains an option that returns pictures relating to your search query.

A9 allows you to specify photo sharing or photo management sites. These include: Flickr, 3dStudio, Webshot Photos, MyImage Search. Trove.net, iStockPhoto, and Smugmugfull. You can specify to search all of these sites, some of these sites, just one of these sites, or add additional sites. I performed the same experiment in A9 that I performed on Flickr.

The tag "
Boston
" returned results similar to those from Flickr. The results from Trove.net were the most accurate. Some sites returned no images.

Each site except Flickr did better with the tag "
Boston
buildings". The results were more refined. However, since Flick is one of the most popular photo sharing sites and one cannot police tag entry, the results were not as specific. This really wasn't a problem. While the other sites' search results returned 99% to 100% accuracy, Flick was around 95% accurate.

The "purple dress" tag returned interesting results. Some, as expected, returned images of purple dresses. Flickr, however, focused on the word "purple" and returned some beautiful images of purple flowers. Interestingly, the search results entered directly in Flickr returned better results than the Flickr results returned by A9.

Image search engines will be one of the next big Web applications. Picassa2 is great and tagging works well. It is no different than entering a text query in a search engine and receiving some unexpected results.

Now if I could just hum a few bars and have a search engine name that tune!