Opened 12 years ago

Closed 12 years ago

#2440 closed defect (fixed)

Shrinksafe fails on files with UTF-8 BOM

Reported by: dmagliola@… Owned by: alex
Priority: high Milestone:
Component: ShrinkSafe Version: 0.4.1
Keywords: Cc:
Blocked By: Blocking:

Description

When uploading a file that starts with the UTF-8 signature (EF BB BF), ShrinkSafe? returns a file that only contains the string "null" inside.

I understand that this is probably not a common occurrence, I found this out because Visual Studio 2005 adds this signature to files. When replacing the first 3 bytes with spaces (20 20 20) using an hex editor, it works perfectly.

Change History (6)

comment:1 Changed 12 years ago by Adam Peller

Summary: Shrinksafe fails on UTF-8 filesShrinksafe fails on files with UTF-8 BOM

To clarify, I think you're saying it's the byte order mark, not the actual encoding which breaks shrinksafe? The Dojo build suffers from the same problem (#1660)

comment:2 Changed 12 years ago by dmagliola@…

Yes, that's exactly what i'm saying. After the byte order mark (the first 3 bytes), the file is pure plain-text (it's a normal JS file, without any unicode characters) And when you take an hex editor and wipe out the first 3 bytes (replacing the with spaces (20 20 20)), ShrinkSafe? works perfectly.

comment:3 Changed 12 years ago by Adam Peller

Component: GeneralBuildTools
Owner: changed from anonymous to alex

comment:4 Changed 12 years ago by Adam Peller

you can probably apply a similar patch to what fixed #1660. Note that there's a JDK bug so that you must identify the BOM as 0xfeff instead of 0xef 0xbb 0xbf

comment:5 Changed 12 years ago by Adam Peller

Component: BuildToolsShrinkSafe

comment:6 Changed 12 years ago by alex

Resolution: fixed
Status: newclosed

(In [10922]) ensure that we're UTF-8 end-to-end. Fixes #2440. Fixes #4712. Fixes #2331

Note: See TracTickets for help on using tickets.