A Webmaster's Look at HTTP Internet Cookies

Nothing seems to set off more arguments than the use of cookies. User awareness (and misconceptions) about cookies is on the rise since many popular browsers are offering to filter cookies. In particular, the latest versions of Internet Explorer and Firefox gives users more control over cookies and has made more users aware of cookies on the web sites they visit.

Personally, I think most cookies are relatively innocuous. However, many users fear that cookies are an invasion of their privacy. While tools like Junkbusters, Cookie Safe (a Firefox extension), and Cookie Monster have always allowed advanced users to control cookie usage, modern browsers are allowing users with average skills to block cookies.

If your web site relies on cookies, you run the risk of alienating these users. In addition, some platforms (like WebTV or some handheld devices) can't handle cookies. How will your site handle these browsers?

Reports on how many users block some or all cookies provide conflicting numbers, but whatever the true number is, it is sure to rise as more users discover tools to control their cookie intake.Cookie Flavors

Cookies can be transient or persistent. Transient cookies disappear when the browser closes. Most browsers don't even store these cookies on disk. These cookies are useful for tracking information about a user during a single visit. For example, you might hold a user's shopping cart ID in a transient cookie.

Persistent cookies have an expiration date (that might be years in the future). Some sites use these cookies to store user preferences. You might store a user's language or color preference in a cookie that persists. These cookies usually do appear somewhere on the user's hard drive.

Cookie Problems

Depending on transient cookies may alienate users that can't or won't accept cookies. Using persistent cookies runs the same risk plus adds some new problems. If a user has more than one computer or browser, the cookie will only work on one browser - that is, each browser has its own cookie.

Another problem is the way browsers manage persistent cookies. Browsers (or users) can delete cookies on a whim. It is disconcerting when a site suddenly forgets your preferences.

Be aware that some browsers place strict limits on how much data you can actually store in a cookie. Usually you won't actually place real data in the cookie. Instead you'll store an arbitrary ID that you can use to look up the user's data in a server-side database.

The Alternative

So if you don't want to use cookies, what's the alternative? You actually don't need cookies to store transient properties at all. Instead, you can encode a key in the links you serve to the browser. Then when the browser makes a request, you can read the key from the URL (or the form data if you are using forms) and identify the source of the request. Presumably, you'd look up the resulting information in a server-side database.

Many server tools will do this automatically. For example, PHP can automatically transform every URL to uniquely identify the user (you must build PHP with the --enable-trans-sid option). To the PHP programmer, this is completely invisible. Most implementations of JSP support URL rewriting (via the encodeURL and encodeRedirectedURL methods).

Naturally, with some effort there is no reason you can't encode session information in a URL using any server-side language. For example, suppose you use Microsoft's ASP. You might want to encode an order ID for shoppers at your site. You could encode the ID in each link like this:

Buy Item 1025

At first, the ID variable will be empty, so the actual link will be: buy.asp?ID=&SKU=1025.

Then part of the buy.asp script would either read the submitted ID, or assign one if required:

ID=Request("ID")

if ID="" then

Application.Lock

if Application("NEXTID")="" then Application("NEXTID")=1

ID=Application("NEXTID")

Application("NEXTID")=ID+1

Application.Unlock

end if

%>

Now the ID variable has a unique serial number and the NEXTID application variable is ready for the next request.

Finally, the links back to the original script needs to have the new ID appended to them:

Continue Shopping

Of course, every link must pass the ID (or you could pass the ID in a hidden form field).

There are many advantages to this approach. Obviously, this will work with any browser without regard to its support (or rejection) of cookies. However, there are some downsides as well. Since the data is encoded in the URL, the user sees a big ugly URL. Also, if the user bookmarks the site, they also bookmark the session ID, which could be a problem (you could encode an expiration date into the ID, though). Adding session information to URLs can also make it difficult for search engines to properly index your pages. Another disadvantage is that URL rewriting requires more server side processing than normal pages (of course, if you are already using scripts, the incremental overhead will probably be acceptable).

Another approach is to try to set a cookie and use the URL encoding as a fallback. In fact, JSP's encodeURL method does nothing if it can set a cookie - it only takes action when cookies are not available. You can test for cookies by setting one on one page and testing for its existence on another page. Some server-side languages (like ASP) try to set a cookie for all users, so you may be able to simply test for that cookie.

One thing the URL encoding method doesn't do for you is replace persistent cookies. I've come to the conclusion that you should almost never use persistent cookies anyway. If you have customized settings, you should let the user sign in and retrieve their settings from your database. You might set a cookie to remember their sign in information as a convenience, but don't rely on a cookie to actually store the database key. Users that use more than one machine or browser will thank you and you'll avoid antagonizing users that don't want cookies.

Cookie Best Practices

Sometimes you simply have to use a cookie. However, there are some important things you should keep in mind even if you do decide to use them. The Network Working Group issues BP44 (also known as RFC 2964) to document these best practices.

A little common sense goes a long way. For example, since persistent cookies are potentially readable on the user's hard disk, you shouldn't store sensitive information like credit card numbers in them. You should probably avoid requiring cookies only for some frivolous function - make sure your site degrades gracefully if possible.

Finally, if you think your visitors have privacy concerns, consider explaining what cookies are and why you are using them on your site. If you educate users properly, most of them won't object to your cookies.

Privacy

The W3C is promoting their Platform for Privacy Preferences Project (P3P) standard. With this scheme, servers that supply cookies also provide a privacy policy. IE6 only supports the compact privacy policy, which is only a part of the P3P standard.

Users can elect to block all cookies or they can block certain kinds of cookies. For example, a user might elect to block all third party cookies (cookies that arrive as part of ads or images embedded in a Web page, for example). So if you visit www.associatedcontent.com and another site tries to set a cookie as part of that page loading, it is a third-party cookie and most modern browsers can reject it. One of the choices the user can make is to reject cookies that do not have a privacy policy. The user can also individually accept or reject cookies on a site by site basis.

The compact privacy policy uses three letter codes to indicate what information a site collects as part of its cookies. The user can choose to allow cookies that don't use personally identifiable information and the browser uses these codes to decide what information the site is collecting.

Decision

If you use a language like PHP or JSP that offers easy URL rewriting support, there is no excuse for requiring cookies. You may want to use cookies if they are available, but why not allow URL rewriting as well? If you use a language that doesn't support URL rewriting directly, you can still get the same effect by writing custom scripts. You'll have to examine your user base to decide if the extra development effort is worth it.

Meanwhile, if you do use cookies, you'd better consider supporting the P3P compact policy on your site. If you don't you may find more users refusing your cookies and leaving your site.

Resources

The P3P Project - www.w3.org/P3P

Unofficial cookie FAQ - www.cookiecentral.com/faq/

Junkbusters - www.junkbusters.com

Cookies and Mozilla (including Firefox) - kb.mozillazine.org/Cookies

Cookie Safe - addons.mozilla.org/en-US/firefox/addon/2497