Gerhard This is different. It tracks the hot areas where clicks appear. You can have the same link in the page, but at different locations and you will know which one was clicked. Check Google Adsense's heat map, this is something similar but without the ads, and fine grained per site and per page. Having said that, it is a non-free subscription service (30 day trial) from an unknown company (they are not Google Analytics for example). Not sure what they do with the data, privacy, and such, and hence would be hesitant to use them. On 8/9/06, Gerhard Killesreiter <gerhard@killesreiter.de> wrote:

Arnab Nandi wrote:

...

[drup-infrastructure seems to be bouncing me, hence cross-posting here]

http://www.clickdensity.com/ : "clickdensity records the position of every click on a page, building up a virtual heat map of visitor activity. But that's just the start. It also provides user behaviour analysis, segmentation and tracking."

I think this is a wonderful opportunity to get some free usability testing done : if we install it on drupal.org, the data generated can be used in improving not just Drupal.org, but also the Drupal interface, since we use most of the popular modules for Drupal.org anyway. The only downside might be a few microseconds of loadtime per user, but I think it's definitely worth that much.

We already collect all that data by ourselves. Drupal.org has the access log enabled and so we can in principle track every user's trail of clicks across drupal.org. I've some time ago made a graph showing such averaged trails, however it hardly generated interest:

http://drupal.org/node/30211

Cheers, Gerhard

...

Gary Feldman wrote:

Gerhard Killesreiter wrote:

...

Arnab Nandi wrote: We already collect all that data by ourselves. Drupal.org has the access log enabled and so we can in principle track every user's trail of clicks across drupal.org. I've some time ago made a graph showing such averaged trails, however it hardly generated interest:

Those graphs look like they been distilled too much to be useful from a usability sense. They show what areas are popular, but I don't think fully show what's hard to find.

Usability wasn't really what I had on my mind at that time. I just wanted to explore the idea to make use of the logs. I think we could provide the logs in an anonymized form, ie replace the IP addresses and uids by some unique string that can't be tracked back to the user, to people who'd be willing to look into this.

One thing I'd look for pogo-sticking (a term I heard from Jared Spool, though I don't know if he's the one who coined it). That's where a single user is trying to find something, starts at one page, follows a link, decides it's not right, goes back to the first page, follows a different link, decides it's no good, goes back to the first page, etc. That could probably be done automatically, but it's not trivial.

Yeah, might be difficult to spot. Especially with people who use tabbed browsing and who might open a series of tabs from the same start page.

Another thing to look for is the success rate of searches, but I think that requires a manual analysis.

How would you define a succes reate of searches? We log search requests too. Cheers, Gerhard

Gary Feldman wrote:

Gerhard Killesreiter wrote:

...

How would you define a succes reate of searches? We log search requests too.

For starters, given a search and results, does the user look at them or immediately submit a different but related search? If you look at a set of search results, do they seem to be related or disparate?

Makes sense.

I haven't looked to see if there are any automated tools that help. I could imagine automatically looking for a sequence of searches that are successively refined (e.g. search for word1, then search for word1 and word2), but that wouldn't find related word searches (e.g. search for "categories" then search for "vocabularies").

We have the referrers (yes, I know) for successive clicks, so it should be possible to see the user behaviour and might even be able to extract this automatically.

I'm sure there's more around this. I just know enough to be dangerous, i.e., I know that search facilities are used heavily at many types of sites, and therefore they are an extremely important feature (perhaps the most important part of any product-oriented web site), but I haven't looked much at what makes a search facility good or bad.

Oh, the search feature is heavily used on drupal.org, we had about 1350*50 = 67500 searches in the last week. And in fact, it is noticable how silent the "Drupal's search module sucks" crowd has become since we upgraded to 4.7. Cheers, Gerhard

Arnab Nandi wrote:

Killes, thanks for taking the thread way offtopic. Let's try and get back to Heatmaps.

You are welcome.

My point was that this service allows us to do analysis while considering a page as a 2-dimensional structure. You cannot do this with search logs, hence the point of this service, and my email. There

This service charges GPB 200 for 5 Mio clicks. Drupal.org served 7 Mio pages in July only, so we'd need some money to spend on this.

are many others like it, this one seems to be simple, available(unlike CrazyEgg) and most importantly, free.

as in beer. Also, the free thing only covers one page.

If you want to take the trouble and write your own Javascript mouse position handler for Drupal.org, it's not impossible, but someone will have to do it.

Sure, but as has been pointed out: We'd retain control of our own data, so I'd prefer it if somebody who has both skill and interest writes such a script. Furthermore, as has been pointed out by me, drupal.org is a large, untapped datamine. If we make sure the data is properly anonymized, we can certainly use that data. Unfortunately, nobody seems to have more than passing interest to exploit the data. So I don't see a point in accumulating more data by installing a non-free script which stores our data on somebody else's server.

Will the heatmaps be theme specific? Of course. But the Bluebeach theme still maintains many distinctive properties common to most Drupal themes (tabs, form layouts, etc) This can give us a lot of usability information : for example, are the tabs really useful at all?

You can easily determine this from our own accesslog data. We store the referer and the destination of each click. So, if the referer is /node and the destination is /project then I clicked on the corresponding tab. In most cases there is only one link for a specific destination on one page.

Do the disctinctive center-aligned-login boxes create problems? (people will click to the left of it if this is the case).

Ok, this you cannot get from our data.

Also, if people have a popular website and are using Bluemarine / any other popular theme, pl. go ahead and share your heatmaps! They are very valuable for Drupal development!

I think the results would not be very Drupal specific, actually.

Regarding privacy: I don't see any privacy issues at all. If we restrict the page tracking to non-user and public pages, we are shown an *aggregate* of all users' usage.

To get an aggregate, you still need to store the individual events. They especially list prices for stored clicks, so I guess that's what they'd do.

There is no concept of personal usage at all. Pl. let's not get paranoid here, these are stats from a public community website, not your private AOL porn search records.

You should extrapolate from your own customs to everybody else's.

Re: Other offtopic issues that were raised:

- Pogo sticking: Use the session variable, will need some hacking - Search Quality: Definitely improved, a simple way to test this: 1) Take the list of search queries from logs, 2) perform Google / Yahoo searches using "site:drupal.org" as added parameter, and then 3) use Mean Reciprocal Rank tests to see the overlap between drupal.org results and "state of the art" search company results.

Anybody interested in doing this? I will happily provide the data. Cheers, Gerhard

Just found this: "How to make heatmaps": http://blog.corunet.com/english/how-to-make-heat-maps -arnab On 8/11/06, Stefan Borchert <stefan@borchert.cc> wrote:

Gerhard Killesreiter schrieb:

...

Arnab Nandi wrote:

...

If you want to take the trouble and write your own Javascript mouse position handler for Drupal.org, it's not impossible, but someone will have to do it.

Sure, but as has been pointed out: We'd retain control of our own data, so I'd prefer it if somebody who has both skill and interest writes such a script.

Something like this: http://www.quirksmode.org/js/eventexample.html (enable "targets" and "mouse stuff")?

-- http://www.arnab.org

Arnab Nandi wrote:

On 8/11/06, Gerhard Killesreiter <gerhard@killesreiter.de> wrote:

...

Furthermore, as has been pointed out by me, drupal.org is a large, untapped datamine. If we make sure the data is properly anonymized, we can certainly use that data.

You should extrapolate from your own customs to everybody else's.

Logs are of this form:

ip_address | log information ip_address | log information ip_address | log information ip_address | log information ip_address | log information

Now if you discard the first column, and use only the second column. imho, i don't see how you can invade someone's privacy here. (please provide examples against this if you can) The data is not as useful as before, but is still quite worthy of analysis.

The data becomes completely useless. If I wanted this kind of data, I would look at apache logs. What makes the accesslog table interesting is that it can track the complete path a user takes through your site. sid varchar(32) NOT NULL default '', path varchar(255) default NULL, url varchar(255) default NULL, hostname varchar(128) default NULL, uid int(10) unsigned default '0', These are the interesting columns of the table: sid is the session id which is unique even for anonymous users, path stores the Drupal system path, url stores the referrer, hostname the IP. Now you can take the path of one log entry to look up where it has been used as referer and you readily find out where the user clicked. The only "problem" is that the referer is stored as fully qualified url. So, in order to anonymize the data, I should probably hash the IPs with md5 using a secret salt and do the same with the sids. All what matters is that they are unique, after all. Paths like user/nnn/edit should probably also be anonymized since only the author can edit his account usually. Of course the uid column should be omitted or similarly anonymized. Actually, IPs are only interesting for bots which don't keep session data. Would be interesting to see how google traverses our site. :p Cheers, Gerhard