Opened 11 years ago

Closed 8 years ago

Last modified 8 years ago

#7030 closed defect (fixed)

Store users can't determine how Dojo Data Notifications affect their query result sets

Reported by: kriszyp Owned by: Jared Jurkiewicz
Priority: high Milestone: 1.6
Component: DojoX Data Version: 1.1.1
Keywords: Notifications Cc:
Blocked By: Blocking:

Description

Widget/Store? users have no means of determining whether an onNew event should result in an addition to the current query result set that they are displaying using or not. In addition, onNew and onSet can also affect the order of items being displayed, and once again the store users have no means of determining this information other than doing a new query.

I propose the concept of live result sets. Data stores that choose to support live result sets can indicate their capability and when desired can maintain active changes to query result arrays based on notifications. Stores can properly update the arrays that have been returned from previous queries with correct internal knowledge of the affects of new item creation and item property changes.

To use live result sets, you do a fetch, and if the returned dojo.data.api.Request object has a "makeLive" function, that function can be called to register the result set as "live". Once a result set is registered as live, any onNew, onSet, and onDelete operation will cause the result set to be properly updated to reflect the change, with awareness of whether or not a new item should be added to a result set based on the query and properly placing the item in a result set to preserve sorting and even if items should be removed or added based on property changes. Widgets can use these result sets to determine how to react to notifications, and how to update their displayed results based on changes. When an onNew or onSet is called, a widget can scan the result set to see if and where the new object or changed object should be place, or alternately a widget can simply use the updated result set and rerender based on that. This approach would be a very convenient mechanism for simpler widgets, that could just register for all the notifications and simply react by rerendering based on their result set array (regardless of which notification was called).

When a widget is finished using a result set, it would be good practice to call the abort function on the Request object, this will signal that the result set is no longer "live", and it will no longer be maintained (and could be GC'ed). If fetch returns a Request object that does not have a "makeLive" function, this indicates that the result set can not be made to be live, and the widget will have to resort to other means to maintain a proper view of the data.

The goals were:

  • Minimal API addition - basically would add a single optional method to dojo.data.api.Request (makeLive)
  • Expose the correct effect of a notification event, whether new items should go in a query, where it should go in sorted lists, and removal and additions due to property changes.
  • Ability for stores to signal their ability to handle live result sets per fetch (some fetches may support this and some may not, even within a single store).
  • Ability to kill a live result set, so the monitoring can be ended and resources released.

I have put together an experimental mixin class to augment a store with this capability, to test it out. This module will use the best available information to maintain result sets, using query attribute objects to determine if items are in a result set, and using the sort arrays to maintain sort information.

Attachments (1)

updateable.diff (18.4 KB) - added by Kris Zyp 11 years ago.
Updateable base class

Download all attachments as: .zip

Change History (8)

comment:1 Changed 11 years ago by Jared Jurkiewicz

Kris,

I've started looking at this and have a few preliminary comments:

1.) I don't like dual-using the abort function. Since you talk about having a makeLive(), then there probably should be a complimentary function (endLive), or somesuch. Or maybe something like: setLive(true|false), that either marks the set as live or not. That seems to be a cleaner approach and still adds only one function.

2.) First concern is reuse of a request object. It's not uncommon that people would reuse a request object and just alter certain parameters on it, such as start and count, to shift a view while keeping the query itself the same. What would be recommended for live sets? Do not reuse the object?

3.) What happens when you have a dataset that is supposed to be start:0 and count of 10? (Fixed size), and an onNew would end up inserting into that array due to sort. Would an item drop out of the array at the bottom?

4.) You made a comment about filter not being very compact. I'd love to see the regular expression that can do the same thing as filter does. I'm no regular expression expert and the few permuations I tried with using them to morph the pattern didn't work in all cases that filter handled. If you can provide a regexp that does the same thing as efficiently (or more efficiently), that would be awesome.

comment:2 in reply to:  1 Changed 11 years ago by kriszyp

1.) I don't like dual-using the abort function. Since you talk about having a makeLive(), then there probably should be a complimentary function (endLive), or somesuch. Or maybe something like: setLive(true|false), that either marks the set as live or not. That seems to be a cleaner approach and still adds only one function.

That's fine with me, was just trying to minimize API additions, but EIBTI I suppose.

2.) First concern is reuse of a request object. It's not uncommon that people would reuse a request object and just alter certain parameters on it, such as start and count, to shift a view while keeping the query itself the same. What would be recommended for live sets? Do not reuse the object?

Good point, I hadn't considered this case. How is a store supposed to react to an abort call on a request object that has been reused. That is if the same request object has been used for two fetch requests and they are both pending, what should happen when abort is called? Would it better if stores made a new instance/copy (request = dojox.mixin({},request) of the request object to use/return? Or do widgets rely on the identity of the request object not changing?

3.) What happens when you have a dataset that is supposed to be start:0 and count of 10? (Fixed size), and an onNew would end up inserting into that array due to sort. Would an item drop out of the array at the bottom?

I had considered this case, but didn't come up with any definite solution. Dealing with paging is definitely tricky, although certainly doable. More of matter of just deciding on what convention to use, I think.

4.) You made a comment about filter not being very compact. I'd love to see the regular expression that can do the same thing as filter does. I'm no regular expression expert and the few permuations I tried with using them to morph the pattern didn't work in all cases that filter handled. If you can provide a regexp that does the same thing as efficiently (or more efficiently), that would be awesome.

Sorry for criticizing your code. This is kind of orthogonal to the LiveResultSets? issue, but I will see if I can create an improved/smaller version of filter when I get some time.

comment:3 Changed 11 years ago by Kris Zyp

After further thought, paging is really problematic :/. Let me propose an alternate approach. Data stores could optionally provide an updateResultSet method that will update an array of items for the caller:

updateResultSet(/*Array*/ resultSet, /*Object*/ request) /*Boolean*/

The resultSet is the array to be updated, and request object is the arguments used for a fetch that should be used for the resultSet. The function returns a true indicating if it was able to make the update, updating the result set with the last notification that took place. If it returns false it means it could not update the array, and the widget needs to resort to other means to update (either blindly updating like many widgets do now, or doing a re-fetch).

I think this may make more sense for managing the result sets, it does not require stores to keep track of which result sets are live. The responsibility is on widgets to remember what the query/request object and the result set that they want updated, and properly deal with providing a correct request object.

I think this is more useful for widgets that use paging; they will generally have their own array of results that is concatenation of the paged queries, and they can provide this concatenated result array to the store for updating (along with the correct corresponding start/count range). The store will keep the result set valid by the query, if a new item is added and it goes in the middle of the list and there is a count provided than the last item will get stripped to maintain the correct count. i.e it will drop out of the array at the bottom. There are most likely situations that a widget can not even submit a result for updating, such as in a sparsely loaded paging array (no way to describe that with our start/count), and once again it can just automatically fallback to old techniques.

I will work on a PoC.

Changed 11 years ago by Kris Zyp

Attachment: updateable.diff added

Updateable base class

comment:4 Changed 11 years ago by Jared Jurkiewicz

Milestone: 1.2future

Ongoing work with Kris on making an updateable resultset if possible. Targeting as future, will hopefully go into 1.3

comment:5 Changed 8 years ago by Kris Zyp

Resolution: fixed
Status: newclosed

This is fixed by the new object store API.

comment:6 Changed 8 years ago by bill

Milestone: future1.7

comment:7 Changed 8 years ago by Kris Zyp

Milestone: 1.71.6
Note: See TracTickets for help on using tickets.