Opened 7 years ago

Closed 7 years ago

#15620 closed defect (fixed)

Dojo build fails when using node.js 0.8.x

Reported by: darth Owned by: Rawld Gill
Priority: high Milestone: 1.8
Component: BuildSystem Version: 1.8.0b1
Keywords: Cc:
Blocked By: Blocking:

Description

Running a build using node 0.8.x fails with the following error

starting writing resources... { [Error: spawn EMFILE] code: 'EMFILE', errno: 'EMFILE', syscall: 'spawn' }
Error: spawn EMFILE
  at errnoException (child_process.js:837:11)
  at ChildProcess.spawn (child_process.js:789:11)
  at exports.spawn (child_process.js:614:9)
  at exec (/dojo/util/build/node/process.js:27:15)
  at Object.release (/dojo/util/build/fileHandleThrottle.js:9:20)
  at ChildProcess.<anonymous> (/dojo/util/build/node/process.js:29:10)
  at ChildProcess.EventEmitter.emit (events.js:91:17)
  at Process._handle.onexit (child_process.js:674:10)
gmake: \**\* [builddojo] Error 255

using ulimit -n 1024 does get around the issue. This issue spans across prior version of Dojo too, so not just 1.8. The issue doesn't happen when using node 0.6.x

Attachments (1)

process.js (1.1 KB) - added by Rawld Gill 7 years ago.
modified util/build/node/process.js that seems to fix node 0.8

Download all attachments as: .zip

Change History (6)

comment:1 Changed 7 years ago by Rawld Gill

Resolution: worksforme
Status: newclosed

I just upgraded to node 0.8.1 and tried on my machine without any problems. Also, if you are *increasing* your file descriptions limit to 1024, then you have a fairly small number of fds to begin with.

Closing since I think we'd be seeing a lot of reports if this was a widespread problem.

comment:2 in reply to:  1 Changed 7 years ago by darth

Replying to rcgill:

I just upgraded to node 0.8.1 and tried on my machine without any problems. Also, if you are *increasing* your file descriptions limit to 1024, then you have a fairly small number of fds to begin with.

Closing since I think we'd be seeing a lot of reports if this was a widespread problem.

I filed the bug here after asking sitepen folks for some help, and they could reproduce this issue. Directly in their words :

This error isn't specific to Dojo 1.8 or to your modifications. Trying with a clean subversion export of Dojo 1.7.3 yields similar errors. The nature of the EMFILE error is that there are too many file handles open. In testing on a Mac with OS X 10.7, we see a clean build using node 0.6.19, but going to node 0.8.1 causes this same error -- most of the time. Not always. If we then run ulimit -n 1024 to increase the allowed number of file handles (or file descriptors, whatever you prefer to call them), the build with node 0.8.1 proceeds just fine. When a build does work with node 0.8, which it does on occasion (we had about a 20% success rate in execution with ulimit -n 256, and 100% success rate with ulimit -n 1024), its output is identical to a build from 0.6.19...

comment:3 Changed 7 years ago by Rawld Gill

Resolution: worksforme
Status: closedreopened

After fiddling with node for a while longer, I managed to duplicate the error on my machine. It appears that node has changed the semantics of the child process exit signal in the 0.8 branch. I'm attaching a modified util/build/node/process.js module that seems to solve the problem in my environment. Please place this in your util tree and let me know what happens.

Changed 7 years ago by Rawld Gill

Attachment: process.js added

modified util/build/node/process.js that seems to fix node 0.8

comment:4 Changed 7 years ago by Rawld Gill

Milestone: tbd1.8
Priority: undecidedhigh
Status: reopenedassigned

comment:5 Changed 7 years ago by Rawld Gill

Resolution: fixed
Status: assignedclosed

In [29276]:

release file handles iaw node.js's various/new behaviors firing process exit and close events; fixes #15620; !strict

Note: See TracTickets for help on using tickets.