Problems with Datastore

[siwatpru]

I am trying to get data from Datastore using the code below. However, it seems like something is not working properly.

var gcloud = require('gcloud')({
    projectId: myProjectId
})
var dataset = gcloud.datastore.dataset()
var employeeQuery = dataset.createQuery('Employee')
dataset.runQuery(employeeQuery, function(err, entities) {
    console.log(err)
    console.log(entities)
})

And here is the result.

{ [Error: Error during request.]
  errors: undefined,
  code: undefined,
  message: 'Error during request.',
  response: undefined }
undefined

I am running this on an Compute Engine instance, so there's should not be any problems with the authentication. I have tried using the Python library for Gcloud and there's no problems with it.

Also, I can use the code below to create a new entity but the 'err' in callback still give the same error, although the entity is already created.

var dataset = gcloud.datastore.dataset()
var key = dataset.key('Employee')
dataset.save({
  key: key,
  data: {
    name: 'Bob',
    age: 20
  }
}, function(err) {
  console.log(err)
})

Am I doing something wrong here ?

[boshido]

I found the same problem while I was following the code example in readme
I tried to search for any help with the error message but it didn't cause the error message
didn't give me any hints.

{ [Error: Error during request.] errors: undefined, code: undefined, message: 'Error during request.', response: undefined }

[stephenplusplus]

@boshido were you also running on GCE?

What version of gcloud are you running?

[boshido]

@stephenplusplus Yes, I was running my code on GCE.
First I used gcloud version 0.23.0 and saw the error so I reverted back to v0.22.0. The problem remained. The problem still occurs for both v0.21.0 and v0.20.0 as well.

[jgeewax]

Are you able to view data in Cloud Datastore from cloud.google.com/console ?

[jgeewax]

(Either way, "Error during request." is about as bad as it gets for an error message. We either need to get Google to fix this on their side for whatever the true problem is, or we intercept it and send a link off to another page that helps sort out the error.)

[siwatpru]

@jgeewax Yes, I am able to view data from the web console.

[electricFeel]

I think I saw something similar to this yesterday evening but didn't have time to fully investigate. In my case I was doing an insert when I ran into this issue. Saving seems to work...when the request is processed in util.js it comes back with a status 200 but in parseApiResp this check is called and the ApiError object is created

if (parsedResp.body && parsedResp.body.error) {
    // Error from JSON API.
    parsedResp.err = new util.ApiError(parsedResp.body.error);
  }

The implemented datastore API isn't a JSON API and the returned body is a protobuf is a binary string so that if condition should be false, for whatever reason, it sometimes isn't. I'm creating a Waterline adapter for the datastore and have instances that work flawlessly and instances where this fails. I haven't established yet if I'm not configuring something correctly but the parseApiResp could better handle this situation either way.

The simple solution is to check the response headers before doing the JSON.parse before doing the parse error check. Alternatively, moving that condition into the try block would also solve it albeit not as explicitly.

Again this is a parsing of the response, so the data is inserted (no problems on the datastore side) but parsing the response errors out.

[stephenplusplus]

@electricFeel thanks for catching this. That was a big whoops, so I'm working on splitting those parse methods apart and making sure we use them appropriately. After I think it's improved, I'll open a PR and ping everyone so that we can see if that resolves the issue here. If it does, we'll put a patch release out right away.

Thanks everyone!

[stephenplusplus]

@siwatpru @boshido @electricFeel - I put a PR together in #913. Please see if it resolves any of the issues you've been having:

$ npm install --save stephenplusplus/gcloud-node#spp--datastore-response-parsing

[siwatpru]

@stephenplusplus The issue is resolved. Everything is now working properly. Thanks :)

[boshido]

Thanks @stephenplusplus.
My problem have been resolved.
It seems to be ok now ;)

[electricFeel]

Sorry I've been a weekend-warrior with the code I've been working on. I confirmed that this resolves the issue I was seeing and, looking at the update, you came up with a much more elegant and durable solution.

Thanks and thanks for the shout out in the release notes!

[stephenplusplus]

Of course, and good luck with your project. Feel free to come back and let us know about it!