[drupal-devel] [feature] AJAX form_autocomplete() field

Tue May 24 05:46:10 UTC 2005

Issue status update for http://drupal.org/node/22519

 Project:      Drupal
 Version:      cvs
 Component:    base system
 Category:     feature requests
 Priority:     normal
 Assigned to:  Steven
 Reported by:  Thox
 Updated by:   Dries
 Status:       patch

Feel free to commit.  Glance over the JS names once more, just to make
sure.  Now we are working on JS, can we (Thox, Steven, et al) take a
look at the collapsible page elements. ;-)  Rock on guys.

Dries

Previous comments:
------------------------------------------------------------------------

May 10, 2005 - 17:29 : Thox

Attachment: http://drupal.org/files/issues/form_autocomplete.patch (11.48 KB)

As discussed on #drupal, I've created a patch to add an AJAX-based
form_autocomplete() field to the Drupal core.

The documentation of the autocomplete callback remains undocumented.
See http://drupal.org/node/22471 for more details.

------------------------------------------------------------------------

May 10, 2005 - 21:18 : Thox

I've created a demonstration page that works in a number of different
browsers:

http://brandedthoughts.co.uk/node/62

------------------------------------------------------------------------

May 10, 2005 - 22:50 : Dries

Works on Firefox @ MacOS.

------------------------------------------------------------------------

May 11, 2005 - 01:14 : solon

Don't mean to throw a spanner in the works, but I get an error if the
request I type does not match any in the database/ file it is looking
through.
-- Begin Error --
JavaScript
An HTTP error undefined occured.
http://brandedthoughts.co.uk/recipe/ingredient/autocomplete
-- End Error --

Would it be best to have it display an ' informative' error message, or
stop trying to 'guess' if it definately can't find a match?

BTW: I am using Safari 2.0 Mac OsX (you don't say :P )

------------------------------------------------------------------------

May 11, 2005 - 02:16 : solon

The error no longer shows, so that issue seems to be fixed (in Safari 2
anyway. As far as I can tell, this is the only browser that it occcured
in).

------------------------------------------------------------------------

May 11, 2005 - 02:17 : moshe weitzman

one more nit - pressing escape should ideally caused tghe autocomplete
div to disappear. sometimes it is not wanted.

please submit a patch against HEAD, if possible.

------------------------------------------------------------------------

May 11, 2005 - 10:01 : Junyor

After typing a number of short terms, I start getting incorrect results.
  It seems mostly to happen with words starting with the letter "b",
oddly enough.

In Opera, the page scrolls when you use arrows to select results.  The
cursor doesn't move to the end of the selected word after you press
Enter.  If the list is open with an item selected and you tab away, the
list doesn't disappear.

I also so some weirdness in Firefox and Opera when using the mouse to
select entries.  Sometimes the drop-down wouldn't disappear.

------------------------------------------------------------------------

May 11, 2005 - 12:14 : Thox

solon: I've fixed that on the demo page now, it seems Safari can't
access the connection status property (so it shows as "undefined").

moshe: the patch above is/was against CVS HEAD, I've just posted it
wrongly. I agree about hitting escape, so I'll try to do that one.

Junyor: I might be able to stop the scrolling, but I'll have to check
on some other things first. Which version of Opera were you using and
which OS? The dropdown hides correctly for me when I tab out in Opera 8
on Windows.

A problem I find with creating this control is that I need to
completely define my own behaviour. I'd like it to act exactly as the
user expects, so I'll try ironing out the above bugs.

------------------------------------------------------------------------

May 11, 2005 - 17:12 : Junyor

I tested with Opera 8.0 and an internal build of 8.01 on Windows XP. 
I'll see if I can find a way to reproduce the problem.  E-mail me if
you have any questions about getting stuff working in Opera.

------------------------------------------------------------------------

May 21, 2005 - 14:45 : Dries

What is the status of this?  I'd like to see this move forward.  Are you
going to roll a patch Thox?

------------------------------------------------------------------------

May 21, 2005 - 18:40 : Thox

Attachment: http://drupal.org/files/issues/autocomplete.patch (13.38 KB)

New patch - mostly usability improvements

Fixed solon's Safari 2.0 error.
Pressing escape now makes the suggestions dissapear (until you type
some more).
Pressing enter now doesn't submit the form when the suggestions are
open, it simply selects the suggestion.
Changed the node author field to an autocomplete field (Dries'
suggestion).

AFAIK, the only outstanding usability issue is that Opera will scroll
up and down when the user tries to move up and down through the
suggested results. This behaviour also happens on the Google suggest
[1] site, so I'm assuming it is very difficult to avoid.
[1] http://www.google.com/webhp?complete=1&hl=en

------------------------------------------------------------------------

May 22, 2005 - 14:35 : Dries

I believe UnConeD (Steven) provide some feedback on IRC.  Is that
correct?  I hope this patch can be committed shortly but I'll hold back
this patch until approved/reviewed by UnConeD.

------------------------------------------------------------------------

May 22, 2005 - 18:51 : Thox

I've made quite a few changes to the code to clean up the following:

- Better theme compatibility (changes since 4.6)
- HTML encoding of usernames before display on screen
- Some extra general-purpose JS functions in drupal.js
- Partially cleaned up CSS

I need to check if UnConeD thinks the new JS is close enough to the
drupal coding guidelines.

------------------------------------------------------------------------

May 22, 2005 - 19:10 : Steven

No new patch attached?

------------------------------------------------------------------------

May 22, 2005 - 19:57 : Thox

Attachment: http://drupal.org/files/issues/autocomplete_0.patch (14.85 KB)

Patch attached. I also changed the markup of the suggestions to be

 rather than . This made more sense to me.

------------------------------------------------------------------------

May 22, 2005 - 23:35 : Thox

Attachment: http://drupal.org/files/issues/autocomplete_1.patch (142.74 KB)

Fixed Opera's strange behaviour of calling the wrong URL during AJAX by
making autocomplete use absolute URLs.

------------------------------------------------------------------------

May 22, 2005 - 23:39 : Thox

Attachment: http://drupal.org/files/issues/autocomplete_2.patch (15.66 KB)

Previous patch somehow got bloated with whole contents of common.inc.

------------------------------------------------------------------------

May 23, 2005 - 22:43 : Dries

Sometimes JS function names start with a capital letter, somethings with
a lower-case letter.  Please stick to lower-case letters.

What does one write in English: auto-complete, auto complete,
autocomplete or AutoComplete?  The patch mixes these.

The patch does not respect my clean URL setting.  It tries to access
user/autocomplete whereas that should be ?q=user/autocomplete.

Is the exit() required to avoid the output of user/autocomplete from
getting cached?  If so, please document this in the code.

Can't this be implemented as an extension of form_textfield()?  A
form_textfield() with callback set to NULL would fall-back on being a
regular textfield.  Like that, it is _easy_ to disable the AJAX-ity of
a textfield (eg. when it turns out to be a performance problem).

------------------------------------------------------------------------

May 23, 2005 - 22:54 : killes at www.drop.org

If there are performance concerns, I'd prefer to have a global setting
where I could swictnh Ajax off for my site.

Also, I think the permission for user/autocomplete isn't right. Anon
users shouldn't have access, I suggest 'access user profiles' or 'acess
content' as permission.

Hmm, now I see it has changed from the first version of the patch.
"administer users" is a bit too stong, IMHO. We should keep other use
cases in mind.

------------------------------------------------------------------------

May 23, 2005 - 23:16 : Bèr Kessels

allthough i think Gerhard is right about the access users thing, I
beleive we should not let this AJAX thing be held back because of that.
The user blocks like latest users et al display lists of users too, even
if one has no permissions to access user profiles. IMO the user system
needs some rethinking, to provide lower level access permissions, But
that is far out of scope for this ajax thing.

+1 for simply returning all users.

------------------------------------------------------------------------

May 23, 2005 - 23:28 : Dries

Elsewhere we print usernames too.  We deny access to the profile pages,
not to the user names.  But yes, we should be careful about other
people (external sites) abusing another site's autocomplete callback to
obtain sensible information.

------------------------------------------------------------------------

May 23, 2005 - 23:30 : Dries

Can we remove the $limit parameter from the callback?  It let's people
(external sites) retrieve all usernames in one go.  I think we can
safely hard-code the number 10. It makes the callback less vulnerable
to abuse.

------------------------------------------------------------------------

May 23, 2005 - 23:40 : killes at www.drop.org

Everybody seems to think that if I complain about permissions I'd be
complaining about too loose permissions. This time I have been
complaining about too strict permissions...

But I can of course implement my own callback, so just forget my
comment.

------------------------------------------------------------------------

May 24, 2005 - 02:16 : Steven

Attachment: http://drupal.org/files/issues/autocomplete_3.patch (16.5 KB)

Here is an improved patch:

Use more neutral color scheme (grays). I wanted to use CSS2 system
colors (WindowText, HighlightText), but Safari does not support them.
Get rid of hand cursor (confusing), use arrow instead.
Fix 404 with clean URLs off
Implement global killswitch: it checks which browser/js-features are
supported and only if all are supported is any JS executed.
Make codestyle more like Drupal's.
Note: in #drupal we agreed to use studlyCaps instead of drupal_naming
for Javascript. This is consistent with the DOM's casing and thus
cleaner. JavaScript also has the convention of keeping acronyms at the
beginning of names uppercase. So "getFile" but "HTTPGet" and
"XMLHttpRequest".
In menu callback: implode, then escape all usernames at once (tiny bit
faster).
Added "form-autocomplete" class to editbox
Added support for a 'throbber': a little indicator of what the box is
doing. The JS sets a class that can be styled by the theme, but by
default there is no throbber.
Hardcode $limit to prevent abuse

------------------------------------------------------------------------

May 24, 2005 - 02:18 : Steven

Attachment: http://drupal.org/files/issues/throbber.gif (794 bytes)

Here is an example throbber. It uses the CSS sprites technique so it
does not need any pre-loading for the animated part.

Copy the .gif to misc/ and then put this at the bottom of drupal.css:
input.form-autocomplete {
  background: url('throbber.gif') no-repeat 100% 2px;
}
input.throbbing {
  background-position: 100% -19px;
}

Shipping such an indicator with core could be considered a usabilty
improvement, and it also makes Ajax features easier to use and more
visible.

------------------------------------------------------------------------

May 24, 2005 - 05:37 : Steven

Attachment: http://drupal.org/files/issues/autocomplete_4.patch (16.95 KB)

CVS diff seems to freak out on this patch, there were two broken lines
in it. I also left in some debug code.

Hopefully this patch will work.

------------------------------------------------------------------------

May 24, 2005 - 07:06 : Steven

Attachment: http://drupal.org/files/issues/autocomplete_5.patch (20.56 KB)

(sorry for the patch-tsunami)

I was examining Gerhard's autocomplete feature for folksonomy tags and
noticed a usability problem:
the suggestion's text in the list must be exactly the same as the value
that is going to be entered in the text box.

Thus, when you have the text "drupal, coding, f" in the folksonomy tags
box, the suggestion list might offer you:
drupal, coding, flowers
drupal, coding, fun
drupal, coding, foo

This is not a very good way to present things: as you add more tags,
the suggestion list gets more and more cluttered with tags you already
have. Furthermore, Gerhard actually underestimated the folksonomy
patch: you can enter tags with commas in them by using quotes. And to
use quotes inside quotes, you must "do it ""like this"". ". Wouldn't it
be nice if the suggestions list showed the actual tag name rather than
its escaped form?

Sure and there is an easy way to solve it: instead of returning an
array of strings, return an array of key/value pairs. The key is the
string that will end up in the textbox, while the value is the string
that is displayed in the suggestion list.

This allows us to show only:

flowers
sex, drugs and rock and roll
foo

but still correctly fill in the value in the box when you select one of
them (drupal, coding, "sex, drugs and rock and roll").

The logic for which value belongs to which key is all done inside PHP
so you get a lot of flexibility in this. To the end-user, it looks like
we have some advanced JS that can fill in the ajaxed tag suggestion into
the list ;).

In fact, you can start presenting rich results: e.g. image thumbnails
or popularity scores for folksonomy tags (or larger font). We have a
lot of possibilities with this for the future and it makes the
form_autocomplete API even more interesting.

There was also another problem: results are returned as a string with
pipe (|) separated items. If a string contains a pipe, the results are
messed up because no escaping is done.

To make all this easy, I added a function
drupal_implode_autocomplete(). It takes a PHP array, whose keys are the
textfield values (e.g. "drupal, coding, flowers") while the values are
the suggestion list strings ("flowers").

The function replaces pipes in keys and values with the entity &#124;
(to prevent conflicts), puts a single pipe between each key and value,
and two pipes between each array item. This format is decoded again in
the Javascript and used automatically. The suggested textbox values are
stored in the DOM nodes for the suggestion <li>s, so they are
automatically freed when the suggestion box is hidden.

Because the changes in this patch are very much tied to Gerhard's
patch, I integrated the two patches into this one. They are both
excellent, clean applications of autocompletion.

------------------------------------------------------------------------

May 24, 2005 - 07:10 : Steven

Attachment: http://drupal.org/files/issues/autocomplete_6.patch (20.52 KB)

Last one, I promise :P. Missed a space.

------------------------------------------------------------------------

May 24, 2005 - 07:24 : Dries

The function names are not consistent.  Sometimes underscores are used,
sometimes not.  Sometimes HTTP is used, sometimes Http.  Sometimes
function names start with a capital letter (AbsolutePosition),
sometimes with a small letter. 

Otherwise, this patch looks perfect.

------------------------------------------------------------------------

May 24, 2005 - 07:35 : Steven

Attachment: http://drupal.org/files/issues/autocomplete_7.patch (20.47 KB)

No rest for the wicked?

------------------------------------------------------------------------

May 24, 2005 - 07:43 : Steven

Bah, read your follow up too late =P. I cleaned up the naming in the .js
files so they are all studlyCaps now (save for the abovementioned
leading acronym rule for functionnames/classes). The only inconsistency
is in the DOM where event names are always lowercase "onkeydown" and not
"onKeyDown". Nothing we can do about that.

If you give me the green light to commit, that would be nice. I don't
feel like submitting yet another patch ;).