DataGrid Performance Issue in IE7 - dojo-release-1.3.0

[Evan]

When tested with data of 3000 records in csv file, DataGrid? got IE7 frozen once single sort is triggered. This is tested on dojo-release-1.3.0.

To reproduce the issue, please: 1.put the test case("test_data_grid_performance.html") under dojox/grid/tests/ 2.put the testing data("music-for test.4.csv") under dojox/grid/tests/support/

Other comments: when large data involved, more time is required to perform sorting & scrolling, it would be better if the loading messages are also shown to replace the large blank area.

[Evan]

Issue snapshot

[Evan]

Test case

[Evan]

testing data

[Evan]

Found the root cause, caused by slow sorting in DataStore? in IE7, Grid get frozen once the following code is executed: dojo.data.util.simpleFetch.fetch() line 65 if(requestObject.sort){

items.sort(dojo.data.util.sorter.createSortFunction(requestObject.sort, self));

} Though it's weird that only in IE the above sorting is especially slow, this isn't problem of Grid, as normally large datset should be sorted on server side rather than in browser

[Nathan Toone]

Changing the component to Data - I'm guessing that this is a peller issue, too...or possibly jared_j?

[Jared Jurkiewicz]

Sorting large data sets in an in-memory store isn't going to be particularly fast in IE. IE has a lousy sort function in their array sort.

[Adam Peller]

Evan, I think this is the issue we discussed . For large datasets, in memory data models will have suboptimal performance.

[Jared Jurkiewicz]

That said ... I may be able to increase the performance some here. I did digging last night and found that specifically CsvStore? did some sub-optimal things in getValue(), which seriously hurts performance on IE (Doesn't help FF either, but the hit isn't as bad).

I'm looking at some performance tweaks to the creation of the sort function, which helps all browsers and any store that uses it. It's not a huge improvement in performance in the grand scheme of things, but it is better. It's all just about moving up closure searching for defined vars so that the final compare function only has to look up one level instead of 3. Knocked about 1.2 seconds of sort time off that CsvStore? example in FireFox? (And helped IE too).

The bigger hit was in tweaking CsvStore?'s getValue to not do some of the checking the way it was doing it. Using the hasAttribute function the way it was kills IE performance here. Just with a tweak here, I cut the runtime of the sort for that Csv dataset by 2/3.

You still won't be able to go much beyond 3K rows in IE and CsvStore?, their JS engine just sucks that much (Orders of magnitude slower than other browsers), but you'll be able to do a fair amount more in Safari, Chome (especially, it's crazy fast), and Firefox.

[Jared Jurkiewicz]

Okay, I've done some work on CsvStore? and the core dojo.data.util.sorter class to try and improve the performance. The area I hit was getValue of CsvStore? and just how the sort function is constructed. The end results are as follows:

All browsers, before getValue tweak, average:
Fire2ox 2: 0.0373671875ms.
FireFox? 3: 0.0298593750ms.
IE 7: 0.1041562500ms.
Chrome: 0.0095000000ms.
Safari 3.2.3: 0.0187421875ms.


All browsers, after getValue tweak, average:
FireFox? 2: 0.0259335938ms. 31% improvement.
FireFox? 3: 0.0113535156ms 62% improvement.
IE 7: 0.0519765625ms. 50% improvement.
Chrome: 0.0067324219ms. 29% improvement
Safari 3.2.3: 0.0097324219ms. 48% improvement.


============================================
Sort 500 rows (Before getValue and sorter tweaks:)
FireFox? 2: 299.3160000000ms.
FireFox? 3: 107.5880000000ms.
IE 7: 830.6520000000ms.
Chrome: 18.4080000000ms.
Safari 3.2.3: 77.8360000000ms.


Sort 500 rows (After getValue and sorter tweaks:)
FireFox? 2: 156.4960000000ms. 48% improvement.
FireFox? 3: 49.1240000000ms. 54% improvement.
IE 7: 342.6200000000ms. 59% improvement.
Chrome: 12.3820000000ms. 33% improvement.
Safari 3.2.3: 30.2400000000ms. 61% improvement.

So, by updating the getValue function of CsvStore? and just slightly tweaking sort function creation, I'm able to substantially boost the performance of CsvStore?. Even with the boosts, IE still sucks compared to other browsers, but oh well. I'm going to double-check my changes and commit them under this tracker.

[Jared Jurkiewicz]

Patch to optimize the sorter creation a bit. Reduces context lookups and bytes a bit.

[Jared Jurkiewicz]

Patch that improves CsvStore? getValue performance and adds in some performance tests for it.

[Jared Jurkiewicz]

Raw perf numbers from tests.

[Jared Jurkiewicz]

(In [17737]) Tweak to sort function generator to improve performance. refs #9099

[Jared Jurkiewicz]

(In [17738]) Missed a semicolon refs #9099

[Jared Jurkiewicz]

(In [17739]) Fixes to CsvStore? to make it process faster. refs #9099