Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#17008 closed defect (wontfix)

Building sources with BOM

Reported by: vita Owned by: Rawld Gill
Priority: low Milestone: 1.9
Component: BuildSystem Version: 1.9.0b2
Keywords: Cc:
Blocked By: Blocking:

Description

When I build layer from sources with BOM header, then BOM persists and is in the layer file for many times. Same problem with integrated dijit templates.

Change History (3)

comment:1 Changed 6 years ago by bill

#15236 fixed it for CSS files, but apparently (according to this ticket) not for dojo/text! inlining or javascript files.

comment:2 Changed 6 years ago by Rawld Gill

Milestone: tbd1.9
Priority: undecidedlow
Resolution: wontfix
Status: newclosed

It seems that this is a very hard problem to solve:

  1. Deferent versions of node interpret the BOM in different ways, and therefore initialize the read data in different ways.
  1. There seems to be no reliable way to sense the BOM when reading files with Rhino.
  1. Having a BOM is technically nonsense for the types of text files the builder expects (utf8).

The supposed obvious solution....

text = text.replace(/^\uFEFF/, '')

simply does not work because, even though the file may have those bytes, the string buffer in JavaScript? usually does not, but rather has some other bytes that the various runtimes automatically insert when seeing the FEFF sequence.

I suspect the code that tries to strip the BOM sequence in the CSS transform doesn't work all the time either.

So, at least for now, the answer is we won't fix. But somebody has the time to propose and prove a general solution, I will happily adopt.

comment:3 Changed 6 years ago by bill

It's unclear if vita is talking about the UTF-8 BOM, which exists but isn't recommended for usage, or his source files are UTF-16 with the UTF-16 BOM (FEFF sequence) you mentioned above, or if he somehow has a UTF-16 FEFF BOM in his UTF-8 files (which just doesn't make any sense).

Note: See TracTickets for help on using tickets.