Opened 8 years ago

Closed 7 years ago

#12619 closed defect (fixed)

Builds fails when prefixes are not matching the prefixpath scheme "../prefix"

Reported by: Marko Reiprecht Owned by: Rawld Gill
Priority: low Milestone: 1.8
Component: BuildSystem Version: 1.5
Keywords: needsreview Cc: m.reiprecht@…
Blocked By: Blocking:

Description

This bug is related to the build system, to create an custom optimized build.

DESCRIPTION and REPRODUCTION


I have following build layout:

-/

bundles
|plugin1 (ns prefix ct.bundles.plugin1)

| \- plugin2 (ns prefix ext.myns

ct (ns prefix ct)

dijit (ns prefix dijit) dojo (ns prefix dojo)

\- dojox (ns prefix dojox)

The problematic parts are the source files in the directories unter the bundles path. In a browser environement this isn't a problem here for each plugin dojo.registerModulePath is called, e.g. dojo.registerModulePath("ct.bundles.plugin1","../bundles/plugin1") or dojo.registerModulePath("ext.myns","../bundles/plugin2").

For a optimized build I use folowing prefix definitions:

prefixes: [

[ "dijit", "../dijit" ], [ "dojox", "../dojox" ], [ "ct", "../ct", "../../ct/copyright.txt"], [ "ct.bundles.plugin1", "../bundles/plugin1"], [ "ext.myns", "../bundles/plugin2"]

]

EXPECTED BEHAVIOR


Start the build process, and source layout == target layout and all files minimized like specified in the build profile.

ACTUAL BEHAVIOR


If I execute the build with such prefixes, it ends that my source layout is changed during the copy process to this:

-/

ext

| \- myns (ns prefix ext.myns

ct (ns prefix ct)

|bundles |plugin1 (ns prefix ct.bundles.plugin1)

dijit (ns prefix dijit)

dojo (ns prefix dojo)

\- dojox (ns prefix dojox)

And later the resolving of some files by dojo.require fails because the internal calculated source locations are incorrect.

This means that the aliasing of source folder names by prefixes is supported by the API but not by the build system.

ANALYSIS


The build system can only correctly work with "prefix" = "../prefix" layouts, e.g.: "dojo" = "../dojo", "ct" = "../ct", or "a.b" = "../a/b" and the prefix must allways be unique between all prefix definitions.

The reason for this restriction is, that the build.js asumes the condition above. E.g. in the build.js on line 263 in the function _copyToRelease are following lines:

var prefixSlashName = prefixName.replace(/\./g, "/"); var releasePath = kwArgs.releaseDir + "/" + prefixSlashName;

As you see this code asumes that it can use the prefix name to create a release file path.

SOLUTION


I solved this problem by fixing following code places:

--- File: util/buildscripts/build.js

Method: release Line: 125 prefixes[i][1] = kwArgs.releaseDir + "/" + prefixes[i][0].replace(/\./g, "/"); prefixes[i][1] = kwArgs.releaseDir + "/" + prefixes[i][1].replace(/.*\/dojo$/,"dojo").replace(/\.\.\,"");

Method: _copyToRelease Line: 263 var prefixSlashName = prefixName.replace(/\./g, "/"); var prefixSlashName = prefixPath.replace(/\.\.\/\.\.\/dojo\/\.\.\,"").replace(/\.\.\/\.\.\/dojo$/,"dojo");

Method: _optimizeReleaseDirs Line: 349 var releasePath = kwArgs.releaseDir + "/" + prefixName.replace(/\./g, "/"); prefixPath is allready absolute! var releasePath = prefixPath;

-- File: util/buildscripts/jslib/buildUtil.js Line : 989 Method: mapPathToResourceName (complete replaced)

buildUtil.mapPathToResourceName = function(pathName, prefixes){

summary: converts a path name to the best fit for a resource name based on the available prefixes. Returns a value like "foo.bar" given an input of /some/path/to/foo/bar.js"

First, find best fit prefix. var bestPrefix = ""; var bestPrefixPath = "";

remove dojo sub path pathName = pathName.replace(/\/dojo\/\.\.\,"/"); for(var i = 0; i < prefixes.length; i++){

Prefix path must match at start (because prefix path contains full path name) the longest prefix path wins var currentIndex = pathName.indexOf(prefixes[i][1] + "/"); if(currentIndex === 0 && prefixes[i][1].length > bestPrefixPath.length){

bestPrefix = prefixes[i][0]; bestPrefixPath = prefixes[i][1];

}

}

if(!bestPrefix){

throw "Could not find a matching prefix for pathName: " + pathName;

}

Strip off first part of file name that is not relevant. var newPathName = pathName.substring(bestPrefixPath.length, pathName.length);

Remove file extensions and any front slash. newPathName = newPathName.replace(/\, "").replace(/\..*?$/, "");

return bestPrefix + "." + newPathName.replace(/\g, ".");

}

These 4 changes solved my to let my local build work as expected. But I didn't test it against an XD build.

Maybe you can use this information to fix this problem for custom builds.

Attachments (1)

dojo.build.patch.txt (1.9 KB) - added by Marko Reiprecht 8 years ago.
The fix steps (better readable)

Download all attachments as: .zip

Change History (8)

Changed 8 years ago by Marko Reiprecht

Attachment: dojo.build.patch.txt added

The fix steps (better readable)

comment:1 Changed 8 years ago by Marko Reiprecht

Sorry for the bad text layout.

Here again the Build Layout:

-/
 |- bundles 
 |    |- plugin1 (ns prefix ct.bundles.plugin1)
 |    \- plugin2 (ns prefix ext.myns)
 |- ct    (ns prefix ct) 
 |- dijit (ns prefix dijit)
 |- dojo  (ns prefix dojo) 
 \- dojox (ns prefix dojox)

And the unexpected target layout:

-/
 |- ext 
 |   \- myns (ns prefix ext.myns 
 |- ct (ns prefix ct) 
 |   |- bundles
 |       \- plugin1 (ns prefix ct.bundles.plugin1)
 |- dijit (ns prefix dijit) 
 |- dojo  (ns prefix dojo) 
 \- dojox (ns prefix dojox)

comment:2 Changed 8 years ago by Rawld Gill

Milestone: tbd1.7
Status: newassigned

comment:3 Changed 8 years ago by Rawld Gill

I'm not completely sure I understand your profile. However, the build system is being completely replaced in 1.7. Could you please reattempt with trunk and report your findings.

Thanks!

comment:4 in reply to:  3 Changed 8 years ago by Philippe May

I am not sure, but i think i face a related issue with prefixes.

Ideally i would like to put a custom package outside of the dojo tree, and have a prefix in the profile like: "../../../foo".

With the new build system, doing so triggers an error:

js: "../../dojo/dojo.js", line 809: exception from uncaught JavaScript throw: TypeError: Cannot call method "map" of null

A work around solution is create a symlink of the package inside of the dojo directory tree, and have the "../foo" prefix.

Note that with the current SVN repo, the build system throws an approaching error while building the "layers" profile:

js: "../../dojo/dojo.js", line 809: exception from uncaught JavaScript throw: TypeError: Cannot call method "split" of null

Hope that helps.

comment:5 Changed 8 years ago by Rawld Gill

Milestone: 1.71.8

Putting a tree outside dojo has historically (read v1.6-) been impossible. The new build system can do it, but it requires different build syntax than is currently published. Although technically, there's not a bug here, I'm keeping this ticket open until I get the docs out that explain how to do what you are describing...hopefully soon.

comment:6 Changed 7 years ago by ben hockey

Keywords: needsreview added
Priority: highlow

rawld,

do these docs exist yet? can the ticket be closed?

comment:7 Changed 7 years ago by Rawld Gill

Resolution: fixed
Status: assignedclosed

This is not a problem with 1.7+; please consult http://dojotoolkit.org/reference-guide/1.7/build/buildSystem.html#specifying-resources.

There are no plans to update the old builder.

Please rewrite your profile using the new format.

Note: See TracTickets for help on using tickets.