Bidirectional hasMany relationships are not loadable/saveable from the many side

[rsutphin]

The default behavior of DS.JSONSerializer is to not serialize the many side of bidirectional hasMany relationships. See emberjs/data#2494 for links to discussion about this.

The upshot for ember-pouch is that the contents of a bidirectional hasMany relationship isn't saved or loaded when working from the many side. E.g., say you have these models:

// app/models/person.js
export default DS.Model.extend({
  sandwiches: DS.hasMany('sandwich', { inverse: 'owner' })
});

// app/models/sandwich.js
export default DS.Model.extend({
  owner: DS.belongsTo('person')
});

In this case you won't be able to load or save a person's sandwiches from the person. (You will be able to if you remove owner from sandwich, though.)

The workaround for this that other similar adapters use (ember-localstorage-adapter, ember-indexeddb-adapter) is to provide a serializer which forces these associations to be serialized on the hasMany side (as well as the belongsTo side). I'm doing this now in my own application to work around this issue.

Unfortunately this requires copying a bunch of code from JSONSerializer (hence emberjs/data#2494). It also duplicates the information about the relationship, requiring both sides to be saved manually whenever the relationship changes.

I'm not sure if there's a practical better solution. It appears that the ember-data solution is to have the server use different serializations for reads vs. writes, but that doesn't work for a local database. One idea I had was that the adapter could look up the IDs for the hasMany relationship dynamically — since all the data is local this might work, but it also might be complex to get right.

[nolanlawson]

Sounds reasonable. I'd be happy with the copy-paste solution until a better solution gets cooked up by the Ember Data team.

[rsutphin]

I'm going to send in a PR for this soon, but since this came up for someone else (#19), I've made a gist with the application serializer I'm using in my app for the moment.

[backspace]

Eep, I did read this thread, but not closely enough, apparently. I think the “from the many side” confused me, but I should have recognised this as my problem.

I tried adding your serializer.

Now there’s the opposite problem: the ID of the parent model is now being stored as null. If you try out the modified example application, the features are properly pre-loaded, but the relation is lost when you click the Load features button. If you print the database, you can see the issue ID is missing from the feature.

Maybe it has to do with the way I’m constructing the models. It’s an artificial example because it’s what I’m using to set up tests, so it’s not UI-driven.

Or maybe a serializeBelongsTo method is also necessary. I’ll investigate this, but I feel pretty out of my depth.

Thanks for noticing my floundering in that other thread!

[backspace]

Never mind, I was able to get the child model to have the parent model ID by either passing in the parent when constructing the child, or saving the child after adding it to the parent. Still learning hehe

[nolanlawson]

This sounds reasonable. Any chance of a PR or even just a modification to the readme? :)

[tchak]

So, this is more of a pouch-relational issue then ember-data issue :) Patching serializer will require us to save from both ends. It will also require to delete on both ends in case of document removal. This is the exact reason why ember-data will not do it in the json serializer. I think we should try to fix this on pouch-relational side.

[nolanlawson]

So I admit I'm still confused: what exactly needs to change to satisfy Ember Data?

In relational-pouch, we recently added async relations, which I maybe naively assumed cohered to Ember Data's requirements.

What I'm hearing is that when you fetch a sandwich, a person should be attached, but when you fetch a person, their sandwiches should not be attached, which to me seems counter-intuitive. And I thought async was supposed to handle that?

[tchak]

So "async" is definitely the way to go. Sideloading is interesting in case of http backend (less requests), but for local db it only adds complexity. ember-data 1.0 will remove non async relationships (you still will be able to sideload in case of rest adapter).

The problem here is the fact that ember-relational currently needs to save both ends of the relationship.

What you saying is that ember-data should not need serialized hasMany side of relationship. True. It will probably get fixed at some point. Not sure how soon.

[nolanlawson]

Ah okay, so if I just make async default to true, it'll be consistent with ED 1.0?

[tchak]

@nolanlawson It seems so. But right now, options are passed down to pouch-relational, so you are fine for now on this. I will try to find out if we can hope for smarter bi-directional relationships in ember-data in the foreseeable future and will get back at you.

[nolanlawson]

Don't worry, ember-pouch can change separately from relational-pouch; that's why I broke them up. :)

I'll just wait for further instructions, and also anybody who wants to tackle this is more than welcome; I am an expert on PouchDB but very much the opposite when it comes to Ember. 😛

[rsutphin]

Patching serializer will require us to save from both ends. It will also require to delete on both ends in case of document removal.

Right. It's not a great solution, but it's 1) simple to implement and 2) the same as all the other client-side persistence options that I've examined.

Another intermediate step would be to both change the serializer and also have ember-pouch handle doing the saves on both ends. The data is still un-normalized, but then the client application wouldn't need to be aware of it. I could see that being surprising, though — changing an association could result in other changes to related objects being silently persisted.

In relational-pouch, we recently added async relations, which I maybe naively assumed cohered to Ember Data's requirements.

Async relations are part of ember-data's understanding of the world, but do not address this problem directly. They say "when you do a find for an object of this type, you should expect that the values for this relationship will not be included, but will require another find." You can have bidirectional relationships that are async: true or async: false.

From my outside-the-project perspective, I think the overall disconnect is that ember-data's design is based on doing object mapping for documents received from/sent to an API server. There's an assumption that you can have have different representations for serialization vs. deserialization because there's another layer (the API server) which is handling translations to and from a canonical data store.

If we want ember-pouch to behave transparently for bidirectional relationships, ember-pouch (or pouch-relational) will have to take on the relationship-management tasks that ember-data currently assumes the server is handling. This would mean that ember-pouch (or pouch-relational) would need to understand all the ember-data association types and handle doing the appropriate queries to resolve either the ids (for an async: true relationship) or the full records (for an async: false relationship).

What you saying is that ember-data should not need serialized hasMany side of relationship. True. It will probably get fixed at some point. Not sure how soon.

It would be great if ember-data would handle this stuff for us. I wonder how likely that is, though — for its primary use case of data-loaded-from-a-server, requiring a separate query to determine the content of a hasMany seems like a negative.

[nolanlawson]

@rsutphin Another thing to keep in mind is that, unlike a remote REST API, PouchDB should give you roughly the same performance whether you do async: true or async: false. Everything is local, so the additional gets don't add up to much of a performance tax, in the average app.

[rsutphin]

Agreed, @nolanlawson. I think the possibly-noticeable difference for ember-pouch would be whether the next level of relationships are loaded also.

[fsmanuel]

I'm not sure when ember-data added _shouldSerializeHasMany but in 1.13.x it is available
We can add that line to the serializer and add a section to the readme that ppl should import the serializer.

  _shouldSerializeHasMany: function() { return true; }

[broerse]

Or use an initializer to import it? Example: https://github.com/firebase/emberfire/blob/master/addon/initializers/emberfire.js

[fsmanuel]

I'm more in favor of an explicit import.
We already expose the serializer:

import {Model, Adapter, Serializer} from 'ember-pouch';

[rsutphin]

I'm in favor of an explicit import also. An initializer is harder for a user of the library to discover & override.

[broerse]

If it is well documented for beginners lets do the explicit import.

[fsmanuel]

@patuku-roy I'm not 100% on this but you should use a serializer per type that extends your pouch serializer (it's just the RESTSerializer) and declare the attrs hash.
Something linke:

export default DS.RESTSerializer.extend(DS.EmbeddedRecordsMixin, {
  attrs: {
    comments: {serialize: true}
  }
});

See the docs for more infos about DS.EmbeddedRecordsMixin

[patuku-roy]

@fsmanuel Thanks, I did per type serializer and work like a charm, except I can not use serialize: true, I'm using serialize: "ids". The good news, I don't have a problem using ED Beta 18 and I don't have to use application level serializer also.

Another problem coming, every time I saved the model, I can not fetch the record until I refresh the route for couple times, but I suspect this is Ember runloop issue when reload the model rather than ember-pouch issue.

// app/models/post.js
export default DS.Model.extend({
    rev: DS.attr("string"),
    comments: DS.hasMany("comment", {async: true})
});

// app/serializers/post.js
export default DS.RESTSerializer.extend(DS.EmbeddedRecordsMixin, {
    attrs: {
        comments: {serialize: "ids"}
    }
});

and I save the model like this

// app/routes/post.js
this.store.createRecord("comment", {
    // ... put the data here ...
}).save().then(function(comment) {
    // save relation to each side
    post.get("comments").get("content").pushObject(comment);
    post.save();
});

[fsmanuel]

post.get("comments").pushObject(comment); should work.

what do you mean by "every time I saved the model, I can not fetch the record until I refresh the route for couple times"?
Is it the comment record or the post record that you can not fetch?

[patuku-roy]

@fsmanuel My bad, I use transitionTo before the record saved, after I'm waiting the record saved, the model can show up without any problem.

This is the working version,

// app/routes/post.js
var self = this;

this.store.createRecord("comment", {
    // ... put the data here ...
}).save().then(function(comment) {
    // save relation to each side
    post.get("comments").get("content").pushObject(comment);
    post.save();
    self.set("didSave", true);
});

and add new method to use transitionTo when the record saved.

// app/routes/post.js

didSave: false,
_transitionAfterSave: function(transitionTo) {
    var self    = this;

    this.addObserver("didSave", function() {
        if(self.get("didSave"))
        {
            self.set("didSave", false);
            self.transitionTo(transitionTo);
        }
    });
},

Thanks anyway ...

[broerse]

@fsmanuel Do you have time to write a PR for _shouldSerializeHasMany: function() { return true; } in the serializer and document it?

[MSchmidt]

Is there any progress on this one? After reading through the issue I'm still not sure how to do this the correct way.

[openhouse]

@MSchmidt I got this to work by adding an application serializer that extends ember-pouch's serializer:

//  app/serializers/application.js
import Ember from 'ember';
import { Serializer } from 'ember-pouch';

export default Serializer.extend({
  _shouldSerializeHasMany: function() { return true; }
});

Now the ids for the manyToOne relationship are stored on the hasMany side as well.

Here is the private method we are overriding
https://github.com/emberjs/data/blob/v2.2.1/packages/ember-data/lib/serializers/json-serializer.js#L814-L830

Ember: 2.2.0
Ember Data: 2.2.1

[wooandoo]

@openhouse thanks for your simple solution!

[nolanlawson]

@openhouse could you open a PR? If your method is successful, it should be added to the repo. :)

[backspace]

I made a PR with the suggested fix but it doesn’t seem like it works, to me… though maybe I’m missing something?

[backspace]

Maybe it is working in my application, but I don’t know why the test in my PR is failing.

This situation where the child record needs to be saved, inserted into the parent collection, and then the parent saved seems unwieldy and confusing for people who are familiar with Ember Data. Is anyone knowledgable enough about Ember Data to know whether there’s a way to cause a parent model to be automatically saved when a child model is? I’ve poked around the code and it seems like maybe it’s not possible, but maybe someone has a better understanding.

[bitmorse]

@openhouse thanks a lot! It's a lifesaver!

[backspace]

Hey, so I submitted another PR that incorporates the _shouldSerializeHasMany fix with a test that passes in all but the Ember 1.13 environment. The problem with my previous PR was that the dummy application wasn’t using the addon’s serialiser.

I’ll have to come back to see why that test is failing; it’s a seemingly-unrelated one.

[nolanlawson]

fixed by 3cb2231 and #111