[regression] ComboBox: dropdown highlighting for autocomplete incorrect for IE 8 browser

[Stan Cason]

I noticed strange behavior of the combobox dropdown highlighting when moving to version 1.8. I ran the example demo and saw that it happens in the demo on IE 8.

http://dojotoolkit.org/documentation/tutorials/1.8/selects/demo/FilteringSelect.php

I type in Al in the autocomplete and get ‚Al‚abama ‚Al‚aska

in the drop down list.

[Stan Cason]

Example Screen Shot

[Douglas Hays]

This seems to be a build problem. Not sure if it's a blocker or not. In _AutoCompleterMixin.js::doHighlight, a replace is performed using this text:

'\uFFFF$1\uFFFF'

but some process between build and runtime is interpreting these as invalid characters and modifying them (to \uFFFD), but the following regexp with the same characters that is supposed to undo them is not modified and so the invalid UTF characters remain. The actual character used is unimportant since they're meant to be temporary but they need to be guaranteed to not be present in the original text and \uFFFF is supposedly such a character. ComboBox? continues to work just fine but the popup menu of items is messed up visually (apparently only on IE). Unbuilt code seems to work fine on all browsers.

[Colin Snover]

If it is a regression, works unbuilt, but breaks built, it needs to block. Setting blocker assuming that is the case here, rcgill can reassign if it is not.

[Douglas Hays]

Same failure with IE9 running examples on ​http://livedocs.dojotoolkit.org/dijit/form/ComboBox

[Rawld Gill]

Replying to doughays:

I did a build and the nonoptimized code looks fine, so I don't think our builder is the direct problem. However, when I inspected a shrinksafe optimized layer (e.g., dojo/dijit-all.js), I see the characters mangled. closure seems to do OK.

I played a bit and found that shrinksafe would be ok if

"\uFFFF$1\uFFFF"

was replaced with

String.fromCharCode(0xffff, 0x24, 0x31, 0xffff)

I didn't check in a fix as I'd like someone more familiar with the code path to look at this suggestion.

otoh, if you still think it is something before the optimizer, please tell me the build command you are using and the file I should look at.

[bill]

So it's a shrinksafe bug that we presumably won't fix, since we'll presumably drop shrinksafe for 2.0. It would be easy to mark as wontfix but since it's a regression I suppose we should check in something like Rawld's workaround.

[Douglas Hays]

Shrinksafe (via Java) is transcoding the string text \uFFFF (6 ASCII bytes) to the shorter UTF-8 characters EF BF BF (3 bytes). But \uFFFE and \uFFFF and reserved by the Unicode as for private use and are listed as non-characters and should not be transcoded. Refer to ​http://en.wikipedia.org/wiki/Mapping_of_Unicode_characters#Private_use_characters. This is somewhat the same as #5027 in which the null character was forced to be escaped within Shrinksafe. rcgill's JS workaround is sufficient but I would recommend fixing Shrinksafe by adding similar exceptions for 0xFFFE and 0xFFFF as well. Since we're dropping Shrinksafe and it's not a problem in the closureCompiler _{I'd hate to munge the JS code that will remain beyond Shrinksafe.}

[bill]

I guess then it falls on you to fix shrinksafe.

[Douglas Hays]

proposed fix replacing util/shrinksafe/shrinksafe.jar

[Douglas Hays]

In [29703]:

Fixes #15969. Force escape of \uFFFE and \uFFFF in shrinksafe since they are non-characters per Unicode spec.