Opened 11 years ago

Closed 11 years ago

#7698 closed defect (duplicate)

OutOfMemoryError with build stripConsole option

Reported by: afisk Owned by: James Burke
Priority: high Milestone: 1.3
Component: BuildSystem Version: 1.2beta
Keywords: Cc:
Blocked By: Blocking:

Description

I'm getting OOMEs very quickly with the new stripConsole option. Adding -Xmx400M to build.sh solved it for me.

Attachments (2)

igx_cms.profile.js (200 bytes) - added by msmith 11 years ago.
Build Profile that causes OutOfMemoryError?
buildOutput.txt (3.0 KB) - added by msmith 11 years ago.
Build command and output that cause OutOfMemoryError?

Download all attachments as: .zip

Change History (10)

comment:1 Changed 11 years ago by James Burke

Milestone: tbdfuture

Yeah, I am wary of adding this by default, not sure what the right amount would be. I was using 256M for the CDN builds. Maybe if we can catch the error in the script and let folks know how to fix it might be useful.

comment:2 Changed 11 years ago by andy

Any chance this could get looked at in 1.2? I get OOME with -Xmx1024M for our custom build. Could we move the stripConsole to a pre build stage? I can't really see why running a regex against a list of files to remove console statements should blow out the heap?

comment:3 in reply to:  2 Changed 11 years ago by andy

Replying to andy:

Any chance this could get looked at in 1.2? I get OOME with -Xmx1024M for our custom build. Could we move the stripConsole to a pre build stage? I can't really see why running a regex against a list of files to remove console statements should blow out the heap?

I see now that the buildUtil is performing the regex against the layer file. Some of my layers are 1MB after inlining so may be why the heap explodes. Moving stripComments to a pre-layer-creation step would be best - but is it possible?

comment:4 Changed 11 years ago by James Burke

andy: sorry I meant to respond sooner, but I forgot to keep the issue new for me.

Can you describe at what point the build blows up for you? If you can give a snippet of the build output before the OutOfMemoryError?, that would be useful. I am curious if it blows during a layer processing or when just processing files on disk.

If possible, giving a sample build profile would help too, to see how much I need to load up the system before running out of memory even with -Xmx1024M. If this is not possible because of a bunch of custom code, then that is fine.

The build command would also be good to know too. Like if you are setting optimize= to some value.

Also, what changed in your build: you mentioned in the original report 400M was good to get you going, but later then you mentioned even with 1024M it was failing. What happened between those comments?

Changed 11 years ago by msmith

Attachment: igx_cms.profile.js added

Build Profile that causes OutOfMemoryError?

Changed 11 years ago by msmith

Attachment: buildOutput.txt added

Build command and output that cause OutOfMemoryError?

comment:5 Changed 11 years ago by msmith

Hi james. I was wrestling with this error for about 4 hours yesterday, before finally searching for it today. I came across this ticket and removing the stripConsole parameter avoids the error for me.

The error that is output every time is "java.lang.OutOfMemoryError?: Java heap space". Interesting notes about the build profile and build command:

  • The layer is named "dojo.js", in order to get a single dojo.js file with the build baked in.
  • The layer includes a large number of dependencies (implicitly). The profile that I've attached has "dojox.grid.DataGrid?" as the only explicit dependency.
  • It may also have something to do with including files from different projects (i.e. both dojo and dojox), but I think it's more related to the number of dependencies.
  • The build command includes the "stripConsole=normal" flag.

The build profile and build command/output are both attached to this ticket. I stripped down both the profile and my build command to demonstrate how to reproduce the error.

Thanks, ~Michael

comment:6 Changed 11 years ago by James Burke

Hi Michael, if you use the raw java command like the following when doing the build does it avoid the issue (works for Dojo 1.2.x or earlier)?

java -Xms256m -Xmx256m -jar ../shrinksafe/custom_rhino.jar build.js profile=.......

where you can increase the -Xms and Xmx arguments to give the build more memory? The ones above are using 256MB.

It is not uncommon to run out of heap space depending on the type of build. For the CDN builds I have to run the java command above with the -Xms/x args for it to build.

comment:7 Changed 11 years ago by lance.spellman

FYI: For our Dojomino project which uses about 30 custom widgets, most extending dijit and dojox widgets, I needed to set this to 1024M to run the build successfully.

comment:8 Changed 11 years ago by dante

Milestone: future1.3
Resolution: duplicate
Status: newclosed

@all - I missed this ticket and opened #8563 with a patch to fix this issue. Works against trunk/1.3 since. Please test and report any breakage. I believe it should lower your memory requirements across the board.

closing this as a dup of #8563 because it has a patch.

Note: See TracTickets for help on using tickets.