Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#15747 closed defect (invalid)

QueryReadStore: wrong encoding of the character

Reported by: charly Owned by: charly
Priority: undecided Milestone: tbd
Component: DojoX Data Version: 1.8.0rc1
Keywords: Cc:
Blocked By: Blocking:

Description

Hi,

my html charset is iso-8859-1.

<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">

The QueryReadStore? encode my character "ü" representing the UTF-8 encoding.

like: ü -> %C3%BC

I need this argument in the ISO Latin character set.

like: ü -> %FC

thx Charly

Change History (9)

comment:1 Changed 7 years ago by bill

Owner: changed from Kris Zyp to charly
Status: newpending

I don't know what code you are talking about. Sounds like you are talking about a server-side implementation of the QueryReadStore protocol that isn't part of dojo, in which case this ticket is invalid.

Please attach a test case using the "attach file" button. It should be as small as possible to still reproduce the problem.

Then, give exact instructions on how to reproduce the problem using your attached test file.

The test case is necessary both to confirm that there's a bug, and for us to be able to debug the problem.

Thanks!

comment:2 Changed 7 years ago by charly

Status: pendingnew

Thank you for your response.

Here is an example, I hope it works:

<html>

<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<head>

 
<script type="text/javascript" src="../dojo.js"
   data-dojo-config="async: true, parseOnLoad: true, isDebug: true, useCommentedJson: true, locale:'en-us'">
</script>

<link rel="stylesheet" type="text/css" href="../dojo-release-1.8.0rc1-src/dijit/themes/tundra/tundra.css" >

<body class="tundra">
<form>

<div data-dojo-type="dojox.data.QueryReadStore" 
                data-dojo-id="store_customer"
                data-dojo-props="url:'json-filteringSelectData.json',
                                requestMethod:'get',
                                doClientPaging:'true',
                                doClientSorting:'false'">
</div> 
         
           <input name="fs_offercustomer"
             id="fs_offercustomer"
             value=""
             type="text"   
             data-dojo-type="dijit.form.FilteringSelect"
             data-dojo-props="store:store_customer,
                              placeHolder:'Search',
                              searchDelay:400,
                              searchAttr:'name',
                              labelAttr:'name',
                              highlightMatch:'all',
                              required:false,
                              autoComplete:false, 
                              labelType:'text',
                              hasDownArrow:false,
                              trim:true,
                              queryExpr:'${0}*',
                              pageSize:5">
</body>
</html>

require(["dojo",        
         "dijit/form/FilteringSelect",
         "dojox/data/QueryReadStore",
         "dojo/parser"
        ],
        
  function(dojo){

  dojo.ready(function(){    
        
  });//ready
  
});//require

json-filteringSelectData.json file:

{
   "identifier": "value",
   "label": "label",
   "items": [
             { "value": "AL", "name": "Alabama",     "label": "Alabama"},
             { "value": "MU", "name": "München",     "label": "München"},
             { "value": "AK", "name": "Alaska",      "label": "Alaska"},
             { "value": "AZ", "name": "Arizona",     "label": "Arizona"},
             { "value": "AR", "name": "Arkansas",    "label": "Arkansas" },
             { "value": "CA", "name": "California",  "label": "California"},
             { "value": "CO", "name": "Colorado",    "label": "Colorado"},
             { "value": "CT", "name": "Connecticut", "label": "Connecticut" }
                            
           ]
}

I use Firefox 10.0.5 with encoding ISO-8859-1.

When I input the character "mü", I get this URL. http://localhost:8080/reports_bm/jsp/test/json-filteringSelectData.json?name=m%C3%BC

Problems:

  1. The item will not selected.
  1. I see a question mark instead of "ü".
  1. "ü" will converted to "%C3%BC", but this is the encoding with encodeURIComponent(). The encodeURIComponent() method replace each instance of certain characters representing the UTF-8 encoding of the character. The escape() method returns the hexadecimal encoding of an argument in the ISO Latin character set, "ü" to "%FC". When I put this string in my browser "localhost:8080/reports_bm/jsp/test/json-filteringSelectData.json?name=mü" the browser convert this string to "http://localhost:8080/reports_bm/jsp/test/json-filteringSelectData.json?name=m%FC" and convert "ü" to "%FC". I think this is a correct encoding and what I need.

thx Charly

comment:3 Changed 7 years ago by charly

Hello,

maybe it's helpful.

When I use requestMethod:'get' the Content-Type of the request-Header is "application/x-www-form-urlencoded" and the input string on the server-side is ü.

When I use requestMethod:'post' the Content-Type of the request-Header is application/x-www-form-urlencoded; charset=UTF-8 and the input string on the server-side is ü.

thx Charly

Last edited 7 years ago by charly (previous) (diff)

comment:4 Changed 7 years ago by bill

Status: newpending

This is unrelated to dojo, or the encoding of your web page. Presumably it's a problem with your web server, where http://localhost:8080/reports_bm/jsp/test/json-filteringSelectData.json returns a page in a certain character set but doesn't specify the character set in the response headers (or specifies it incorrectly).

Try using firebug's NET tab to monitor the return value from " http://localhost:8080/reports_bm/jsp/test/json-filteringSelectData.json?name=m%FC" and see the response headers to see what character set is specified.

Also note that specifying a query as part of the URL: http://localhost:8080/reports_bm/jsp/test/json-filteringSelectData.json?name=m%FC is meaningless since the server will presumably just will return all of json-filteringSelectData.json. You need to set it to doe filtering on the client.

comment:5 Changed 7 years ago by charly

Status: pendingnew

Okay thanks.

I have checked the server-side.

I try requestMethod:'post' and the post parameter is "name=m%C3%BC". On the server-side I get the request parameter "mü". This is correct.

I try requestMethod:'get' and the URL is http://...?name=m%C3%BC". On the server-side I get the request parameter "mü". This is incorrect.

This is weird.

thx Charly

Version 0, edited 7 years ago by charly (next)

comment:6 Changed 7 years ago by bill

Oh, sorry, I misunderstood your original post, let's start over.

Encoding as UTF-8 is the standard, see https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/encodeURIComponent. So I don't think there's any bug with QueryReadStore. Is something not behaving according to spec?

comment:7 Changed 7 years ago by charly

I thing the spec is OK.

Is it possible to set character encoding in the QueryReadStore? to iso-8859-1 or encoding with escape()?

On the Server-side I use iso-8859-1. I think the requestMethod:'get' is correct, because iso-8859-1 hex "C3" -> char "Ã"; hex "BC" -> char "¼".

thx Charly

comment:8 Changed 7 years ago by bill

Resolution: invalid
Status: newclosed

Like you said, I thought you could set requestMethod:'post' on your QueryReadStore. Not good enough?

I don't know any setting to control how parameters are encoded for a GET. QueryReadStore.js calls dojo/_base/xhr(), which then calls dojo/io-query.objectToQuery(), which calls the native encodeUriComponent(). I guess you could monkey patch objectToQuery() to work as you like.

Anyway, I'm going to close this, since the encoding parameters in a URL as utf-8 is standard procedure.

comment:9 Changed 7 years ago by charly

Thank you for your response.

I have found in dojo/io-query.js at line 31 this comment:

"FIXME: need to implement encodeAscii!!"

I guess this is a fix what I need.

thx Charly

Last edited 7 years ago by charly (previous) (diff)
Note: See TracTickets for help on using tickets.