Opened 8 years ago

Last modified 3 years ago

#15260 assigned enhancement

dojo/i18n module does not allow excluding bundles...

Reported by: Karl Tiedt Owned by: Rawld Gill
Priority: low Milestone: 1.15
Component: BuildSystem Version: 1.7.2
Keywords: Cc: Patrick Ruzand
Blocked By: Blocking:

Description (last modified by Karl Tiedt)

Example: we do not support but a subset of the locales supported by CLDR (~20) and currently our app uses an alternate means of NLS for *our* modules. We do not need the full CLDR directory yet if its removed and a module reports that it supports some locale thats been deleted... i18n module will die at the require() for that missing module

I cant help but think there is some missing magic bullet for this that just isnt well documented but... we havent found it.

We currently have patched i18n.js (see attached patch) so that the bundle generator will ONLY return bundles that not only have NLS bundles for that locale BUT are in our locales array used at build time...

We also still cannot find a way to get i18n.js to load from our layer (its required to be left on our server despite being in our built files)

By "delete from source root" you mean modify the NLS file's definition of locales supported?

Ultimately what we are trying to do (and this is how it worked in 1.6) is if a module supports "wx-yz" and we do not... we want "wx-yz" to fall back to "root" - How we achieved this in 1.6 was we deleted the CLDR folder because that 1+Mb of NLS entries for many locales that we never supported were not needed. The flattened builds took care of anything we did support.

However, (and you may have fixed this with one of your latest check-ins) it 1.7.x this behavior would kill the loading process and hang on the 404's for the CLDR nls files. And the reason this happened would be because a generic locale was hit that a CLDR module supported "zh" or "en" but our flattened bundles were "zh-cn" or "us-en" so the loader would try to pull in an "en" only flattened bundle.

If that still sounds like what the Transform would be a good fit for... I am certainly interested in hearing more :)

Attachments (1)

i18n.patch (1.1 KB) - added by Karl Tiedt 8 years ago.
not only check if locale is supported, but if its included in the locales list used to build

Download all attachments as: .zip

Change History (13)

Changed 8 years ago by Karl Tiedt

Attachment: i18n.patch added

not only check if locale is supported, but if its included in the locales list used to build

comment:1 Changed 8 years ago by Patrick Ruzand

Cc: Patrick Ruzand added

comment:2 Changed 8 years ago by Rawld Gill

Milestone: tbd1.8
Priority: undecidedhigh
Status: newassigned

comment:3 Changed 8 years ago by Rawld Gill

I don't fully understand the issue here. Was this feature available in 1.6 or is this a RFE?

If you're loading without preloads (i.e., unbuilt code) and the root bundle is an AMD-style root bundle, then the solution is to ensure that bundle only reports existing locales; if the root bundle is a legacy bundle, then you must be in sync mode, the localized bundles will be loaded via xhr, and you'll get 404 errors as has always been the case for each localized bundle that doesn't exist.

If you're loading with preloads (i.e., built code), then the preloads specify exactly what is available and the dojo/i18n module will load the best available. For example if the localeList profile switch is "ab,"ab-cd-ef" during the build, then the following will be loaded:

"ab" => "ab"
"ab-xy" => "ab"
"ab-cd-ef" => "ab-cd-ef"
"ab-cd-xy", xy!=ef => "ab"
"ab'cd-ef-xy", => "ab-cd-ef"
"ab-cd-xy-uv", xy!=ef => "ab"
anything else => root

If you are looking for built code to restrict what's in the root bundle and not output preloads, then I can easily add a switch to trim the localized bundles that are output. But remember, as it stands, there are two loading strategies for built code: either rely or don't relay 100% on the preload locale list.

comment:4 Changed 8 years ago by Karl Tiedt

I guess we are unclear what i18n path we are getting taken down then -- how do you ensure you get "non legacy i18n"?

Basically what we found was that number.js for example lists ~25 locales, because one of these obscure locales exists in that mapping, i18n will rty to load it if the browser is set to it... if we dont support this obscure locale, and delete those locale folders, i18n barfs and halts page execution.

comment:5 Changed 8 years ago by Rawld Gill

In [28477]:

added check for missing localizations, improved legacy-to-AMD bundle conversion; refs #15260; !strict

comment:6 in reply to:  4 ; Changed 8 years ago by Rawld Gill

Priority: highlow

Replying to ktiedt:

I guess we are unclear what i18n path we are getting taken down then -- how do you ensure you get "non legacy i18n"?

My explanation at comment 3, though accurate, was somewhat incomplete/misleading. The bottom line is that the builder automatically converts legacy bundles to AMD. So if you delete localized files from the output and then try to load any of the deleted locales, the program will fail to load. That is normal, correct behavior.

Again, if you want to output a build with fewer locales than are in the source root, then the solution is to delete those locales from that source root. To my knowledge this was not a legacy option. It is currently not an option with the new builder and doesn't seem that generally useful. The problem could be easily solved by writing a custom transform to go before the writeAmd transform. Of course that's a fairly advanced option and we don't have good docs on how to do that yet. Please confirm this is your issue and I'll try to help out with that transform.

As mentioned in comment 3, none of this applies to flattened bundles. For those, you will get exactly what you ask for with localeList...no more, no less.

comment:7 in reply to:  6 Changed 8 years ago by Karl Tiedt

Description: modified (diff)

Replying to rcgill:

Replying to ktiedt:

I guess we are unclear what i18n path we are getting taken down then -- how do you ensure you get "non legacy i18n"?

My explanation at comment 3, though accurate, was somewhat incomplete/misleading. The bottom line is that the builder automatically converts legacy bundles to AMD. So if you delete localized files from the output and then try to load any of the deleted locales, the program will fail to load. That is normal, correct behavior.

Again, if you want to output a build with fewer locales than are in the source root, then the solution is to delete those locales from that source root. To my knowledge this was not a legacy option. It is currently not an option with the new builder and doesn't seem that generally useful. The problem could be easily solved by writing a custom transform to go before the writeAmd transform. Of course that's a fairly advanced option and we don't have good docs on how to do that yet. Please confirm this is your issue and I'll try to help out with that transform.

As mentioned in comment 3, none of this applies to flattened bundles. For those, you will get exactly what you ask for with localeList...no more, no less.

comment:8 Changed 7 years ago by Karl Tiedt

Component: LoaderBuildSystem
Type: defectenhancement

Updating to build system/enhancement ticket.

comment:9 Changed 7 years ago by bill

Milestone: 1.82.0

Since 1.8 is already beta this should be bumped to 2.0 (or future).

comment:10 Changed 7 years ago by jbest

It appears this changeset was leaked into 1.7.3. I am not sure if this was intentional, but it breaks the previous behavior that the builder had in 1.7.2 and prior. We have a case were we have a huge localList and out of the box we only translate 4 or 5 of those. The 1.7.2 behavior was OK - it would generate an empty combined nls file, and that would mix in over our root. The new behavior of treating these as an errors causes our continuous integration system to halt when it sees errors come in on the build output.

The only issue we had with 1.7.2 was in dev mode (not using the flattened nls built file), it would not fall back to the root correctly when a translation didn't exist. It would throw an xhr error and halt. I thought this was the issue ktiedt had originally described.

Going forward, can we treat error 354 as a warning instead, since in built cases it won't have a negative impact?

comment:11 Changed 4 years ago by dylan

Milestone: 2.01.12

I think we should have implemented jbest's suggestion a while ago, " Going forward, can we treat error 354 as a warning instead, since in built cases it won't have a negative impact?"

comment:12 Changed 3 years ago by dylan

Milestone: 1.131.15

Ticket planning... move current 1.13 tickets out to 1.15 to make it easier to move tickets into the 1.13 milestone.

Note: See TracTickets for help on using tickets.