Opened 2 years ago

Last modified 2 years ago

#18917 new enhancement

Back-porting dojo core and compatibility issues with legacy OS's SSD driver

Reported by: noop Owned by:
Priority: undecided Milestone: future
Component: Storage/Flash Version: 1.11.2
Keywords: Cc:
Blocked By: Blocking:

Description

The symptoms of this operating system related limitation are deep in the Windows driver layer. (If they ever get around to patching it before the next release.)

The general cases occur with a server latency slow down as the flash sectors are forced to re-buffer the FAT when a long-file-name unicode string is parsed (even if the string does not contain extended characters).

Note, simply ensuring a naming convention where the first 6 characters of the file/directory names are unique within the current path will solve most of the edge cases.

Examples:
actual 8.3 msdos file: /DIJIT/_WIDGE~1.JS
actual 8.3 msdos file: /DIJIT/_WIDGE~2.JS
3 IO op increase occurs when serving some files while resolving the prefix ambiguity in serving:
dijit/_WidgetsInTemplateMixin
dijit/_WidgetBase
The Worst case
Example 2 (forces the driver to flush and re-buffer the entire path index): Path to another dos file:
/DIJIT/THEMES/THEMET~1/BLACKB~2.GIF
dos file:
/DIJIT/THEMES/THEMET~2.HTM

There are roughly 570 files in dojo/digit/dojox that can assert the ambiguity issue.
I will be kludging patches of the framework for our internal needs, but it would be nice to see the main branch of dojo in the future releases follow legacy compatible naming conventions.

Attachments (1)

dupe_lfn.txt (22.2 KB) - added by noop 2 years ago.

Download all attachments as: .zip

Change History (5)

Changed 2 years ago by noop

Attachment: dupe_lfn.txt added

comment:1 Changed 2 years ago by bill

It looks like the file name changes you're requesting would break backwards compatibility: applications would need to update all their references to dojo files in require()/define() calls etc. In that case, we can't do it.

It also seems like you could workaround the problem with JS and CSS files by doing a build.

And finally, it's hard to believe the problem you describe still occurs on modern versions of Windows. Didn't FAT get replaced by NTFS around 20 years ago?

comment:2 in reply to:  1 Changed 2 years ago by noop

Replying to bill:

It looks like the file name changes you're requesting would break backwards compatibility: applications would need to update all their references to dojo files in require()/define() calls etc. In that case, we can't do it.

Indeed, we are focusing on patching a conditional define in the loader to auto-register the dependency with the dynamic loader. i.e. 100% backwards compatible as the performance hit only occurs once, as the dojo registry has already cached the ambiguous resource in most cases. However, the dozens of css/image resources that can't be routed still pose the biggest risk of a split to the main version.

It also seems like you could workaround the problem with JS and CSS files by doing a build.

Indeed, we may have to re-evaluate the framework choice in the next project.

And finally, it's hard to believe the problem you describe still occurs on modern versions of Windows. Didn't FAT get replaced by NTFS around 20 years ago?


Released in 2006: https://en.wikipedia.org/wiki/ExFAT
But, we don't expect MS to update the flash driver at all...

Indeed, the hardware was designed around a legacy file-system (it runs poorly even with proper sector alignment in linux), and the wonders of ext4 or ZFS are not an option on the platform (even +1TB disks are not supported).

Thanks for the reply,
I've been using dojo a few years since the two guys from IBM were around...
and wanted to try something odd with an Emscripten compiled port....
Sometimes cross breading an elephant and a pig just wont work. ;-)

Cheers,
J

Last edited 2 years ago by noop (previous) (diff)

comment:3 Changed 2 years ago by dylan

I think this is a scenario where fixing something for this use case would be very inconvenient for 99.99% of users. Descriptive file names are an important part of understanding code as our file names match our module names which match the intent of code.

The best suggestion I can give is to find a tool that can, post-build, rewrite all file names and references to those files within the build-optimized JS. The features within the Dojo build system should be able to support this as we do quite a few things like this within the build, but you'd need to write your own build transform plugin to manage this. I recognize that this will be a painful option for you, but less painful than changing the entire API footprint for all users. :)

comment:4 Changed 2 years ago by dylan

Milestone: tbdfuture
Note: See TracTickets for help on using tickets.