Fixes hasMany records being duplicated after payload returns from server

[kurko]

There is an edge case where a hasMany record gets duplicated (same ID).
Basically, if we generate a record locally using UUID or any custom ID
format, then push it to production which will return the same ID
previously defined, the store's array that holds records to be resolved
is going to duplicate both items, even if they have the same ID.

This commit fixes that by not allowing duplicated IDs in that array.

I need this badly for my app.

[kurko]

Without this fix, post.get('comments.length') returns 2 here, which means you end up with two records with ID=2. Pretty weird.

[stefanpenner]

what triggers this edge case?... nm i read the test.

[stefanpenner]

this function should likely not be defined on the fly, as it doesn't close over anything.

[kurko]

Could you explain what "doesn't close over anything" means? I believe this is very specific to this function, not a reason for adding more complexity to the store.js scope. What do you think?

[kurko]

Could you explain what "doesn't close over anything" means? I believe this is very specific to this function, not a reason for adding more complexity to the store.js scope. What do you think?

[rwjblue]

I believe that by "close over anything" he is referring to the fact that you do not need a closure (which would allow variables in your scope to be available inside this function). As it is now every time addUnsavedRecords is called you would be creating a new function. I believe that this might be a performance issue. You could make notDuplicated a function up out of addUnsavedRecords with likely the same result as the current code, but without generating a new function very time it is called.

[stefanpenner]

i left some high level comments. I wil try to give it more time soon.

[igorT]

How do you run into this edge case? I agree that there is a problem here we should fix, but it seems to me that your test case has a lot of potential to leave you in a inconsistent state, where post has a foreign key which points to nothing if the comment doesn't end up being saved.

[kurko]

My scenario: I have two stores, 1 for offline data (the default) and another for online data. Records have UUIDs, so records never collide (theoretically), so I grab a record from RESTAdapter and do an usual store.push(onlineRecord). It's awesome that everything works (push() really does its job very well), except this tiny little thing where relationships with the same IDs get duplicated.

I wish addUnsavedRecords wasn't a private method inside store.js ((function(){ })) because it clearly needs to be tested. I'd have it just as an extracted object though, so we'd be able to unit test it directly to fix its inconsistencies, but that's for another moment.

@igorT I wrote the test above just to be able to reach into addUnsavedRecords's flaw. In my app, I have something a bit different, like so:

// cart_test.js
test("#save doesn't duplicate relationships", function() {
  stop();

  Em.run(function() {
    cart.save().then(function(cart) {
      equal(cart.get('cartItems.length'), 0, "cart has no items initialy");

      cart.get('cartItems').pushObject(cartItem);
      cart.set('customer', customer);

      return Ember.RSVP.resolve(cart);
    }).then(function(cart) {
      equal(cart.get('cartItems.length'), 1, "cart has one item once saved");
      cart.save().then(function(savedCart) {
        equal(savedCart.get('cartItems.length'), 1, "cart continues having one item after being saved again");
        start();
      });
    });
  });
});

[kurko]

Changed the loop as per @stefanpenner's suggestion.

[igorT]

You say that your problem appears when you push the record into the store, but in the test you calling createRecord instead of push? I think if you pushed it, you wouldn't have this problem?(I agree that this is a bug, but it seems like you shouldn't be doing createRecord for records you don't expect to have to POST to the server)

[kurko]

@stefanpenner updated the code :)

@igorT I agree with the part about push, however when you create a record locally using UUID, you do want to POST that record to the backend. Do you think we can improve this somehow?

[kurko]

Would there be anything I could do to get this merged?

[stefanpenner]

@igorT the client side UUID scenario is legit

let me think through the problem + solution a bit though

[kurko]

@stefanpenner thank you. My main interest is to be able to push an object without having it duplicated in the store. I believe there are architectural improvements to be made, but outside the scope of this PR. Given I've been working on a forked version of ED for 15 days, I'd love to have the solution for this particular problem merged and then, in a next PR, touch other related points.

Please, let me know if that makes sense.

[kurko]

Ping? This has been referenced multiple times. Maybe it's time for a merge? :)

[Bertg]

Yup, we would like to see this as well... +1
Retracting my up vote, we where wrongly using createRecord where we should have been using push.

[seifsallam]

Having the same issue here +1

[butsjoh]

+1

[ivpusic]

+1

[kurko]

I am about to have to stop using Ember if this isn't merged. No reason was given for not merging it. I understand we all lack time, but no one argumented reasonably against it. It's just making a part of the code more robust, not changing any behavior. Is there a reason for not clicking merge? Please, let us know.

Thanks.

[kurko]

@igorT I was coding in an entirely different problem and, coincidently, stumbled in this exact same duplication issue. Here's another failing test that makes no sense:

test("#save doesn't lose relationships from the store", function() {
  var phone, person;

  stop();
  Em.run(function() {
    person = store.createRecord('person', { name: "Clint", cool: true });
    phone = store.createRecord('phone', { number: 1234 });

    console.log("phone", phone.id); // => 'abc'
    console.log("person's total phones", person.get('phones.length')); // => 0
    person.get('phones').pushObject(phone);
    console.log("phone", phone.id); // => 'abc'

    console.log("person's total phones", person.get('phones.length')); // => 1
    person.save().then(function(person) {
      console.log("total phones", person.get('phones.length')); // => 2
      // WTF!
      console.log("phone1's id", person.get('phones.firstObject.id')); // => 'abc'
      console.log("phone2's id", person.get('phones.lastObject.id')); // => 'abc'
      // WTF!
      equal(person.get('phones.length'), 1, "person has 1 phone");

      // test would continue...
      start();
     });
   });
 });

With all due respect, this is a serious bug. After 5 months waiting and patching my own Ember Data, I can't think of any reason why this wasn't merged yet. What deep analysis is there to be done? Seriously, I could be sending PRs all the time, helping Ember Data reach its 1.0 milestone, but I simply decided to stop my efforts for the last 5 months because I simply lost faith.

Please, be kind and just click the green button. Or at least tell me what's wrong with this. All I want is to help.

[igorT]

I think this makes sense. It will be fully fixed nicely by SSOT, because relationships are going to be a set, so they are not going to be able to have the same record twice. Part of the code conflicts with #1939 , I am happy to merge this once we figure out whether we want to use Ember.A or not

[igorT]

Hey, @kurko we merged #1939 can you please rebase this against master? I think after that it should be good to merge

[kurko]

OMG OMG It's passing!

[igorT]

I made couple of cleanups and resubmitted your PR as #1976 I have kept you as the commit author though :) WIll double check the correctness of Ember.A and merge tomorrow

[igorT]

It's merged in master now!

[GetContented]

Awesomeness! :)

[kurko]

Omg omg @igott, you're the best!

[jonathan-thorpe]

Is anyone able to confirm whether or not this has been fixed as intended? I'm using the current master branch and find the same behaviour continues:

I have a controller action that does:
@get('new_post').get('comments').pushObject(
@store.createRecord('comment')
)

And in another controller action:
@get('new_post').save()

My comments (hasMany) relationship then shows two records of duplicated data with the exception of the id. 'comments.lastObject' has an id of null (I expect this is the original record added with the pushObject) and the second containing the id of the comment record returned by the Rails app.

I'm very new to Ember/Ember-data, so please excuse my ignorance if I've done something wrong, but I've spent the past 5 hours trying to figure this out.

[GerritSommer]

Same for me... I need to destroy and remove the duplicate object manually.
So for me its a nofix with Ember data 1.0beta.9

[stefanpenner]

@klopfer1 is your scenario the same as this issue, or is it different

[GerritSommer]

Not exactly, I have a parent, with many embedded childs. When I create a new child and save it, I have the new child twice. One with ID from the backend and one with ID: null. As far as I understood this PR should have solved this.

[stefanpenner]

@klopfer1 can you open a new issue then. One that tries to clearly outline the specific scenario.

Also, if you have the time a PR with a failing test is often the easiest way to fix bugs.

[kurko]

This PR solves two records with the same ID (e.g UUID). Basically, a record returned from backend with an ID of a record that already existed in the store would get duplicated.

I don't think this covers null IDs. 
—
Sent from Mailbox

On Wed, Aug 20, 2014 at 10:52 AM, Gerrit S. notifications@github.com
wrote:

Not exactly, I have a parent, with many embedded childs. When I create a new child and save it, I have the new child twice. One with ID from the backend and one with ID: null. As far as I understood this PR should have solved this.

Reply to this email directly or view it on GitHub:
#1656 (comment)

[GerritSommer]

you are right. I think its related to this #1656 (comment)
I can try to write a test. Just did that on my project yet, but I thinks its time for a first time.

[angusfretwell]

It seems like this issue has regressed and is present in 1.13.14

[Znegl]

Getting the same bug with null IDs:
Saving new embedded hasMany records results in duplicate content in the frontend. One if the items in the frontend is marked as isNew.

Ember : 2.3.0
Ember Data : 2.3.3

[kurko]

FYI @igorT fixed this in another branch 1,5h years ago. Given this is a PR, I suggest opening a new issue and linking back to here.