PubSub topic.create method should raise a Conflict error instead of RetryError/GaxError when topic exists

[lucmult]

Following the documentation from PubSub site if you run the example 2x it will generate the error:

GaxError(Exception occurred in retry method that was not classified as transient, caused by <_Rendezvous of RPC that terminated with (StatusCode.ALREADY_EXISTS, Resource already exists in the project (resource=my-new-topic).)>) <class 'google.gax.errors.RetryError'>

However it should raise Conflict Error (google.cloud.exceptions.Conflict), because the API returns a 409 response.
You can see the status code using API Explorer:
https://developers.google.com/apis-explorer/#search/pubsub/pubsub/v1/pubsub.projects.topics.create

409  
{
 "error": {
  "code": 409,
  "message": "Resource already exists in the project (resource=my-new-topic).",
  "status": "ALREADY_EXISTS",
...
  }
}

Full traceback on Python:

>>> topic.create()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/cloud/pubsub/topic.py", line 155, in create
    api.topic_create(topic_path=self.full_name)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/cloud/pubsub/_gax.py", line 104, in topic_create
    topic_pb = self._gax_api.create_topic(topic_path)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/cloud/gapic/pubsub/v1/publisher_client.py", line 278, in create_topic
    return self._create_topic(request, options)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 419, in inner
    return api_caller(api_call, this_settings, request)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 407, in base_caller
    return api_call(*args)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/api_callable.py", line 368, in inner
    return a_func(*args, **kwargs)
  File "/usr/local/google/home/lucmult/.virtualenvs/pubsub/local/lib/python2.7/site-packages/google/gax/retry.py", line 126, in inner
    ' classified as transient', exception)
google.gax.errors.RetryError: GaxError(Exception occurred in retry method that was not classified as transient, caused by <_Rendezvous of RPC that terminated with (StatusCode.ALREADY_EXISTS, Resource already exists in the project (resource=my-new-topic).)>)

[dhermes]

@lukesneeringer This is part of a more general question: how / where should we re-wrap GaxError?

[lucmult]

For anybody affected by this, to handle ALREADY_EXISTS exception you can use the following scheme:

try:
  # create topic
except Conflict as error:
   # handle the current conflict exception
except RetryException as error:
   status_code = error.cause.code()
   if status_code == status_code.ALREADY_EXISTS:
      # handle the already exists
   else:
      # nothing that we want just re-raise the exception
      raise

[lukesneeringer]

@lukesneeringer This is part of a more general question: how / where should we re-wrap GaxError?

The correct answer is to have GAX raise these different kinds of errors, rather than the catch-all GaxError. It could import them from google.cloud.exceptions.

[dhermes]

@lukesneeringer That would be much preferred to the current ad-hoc approach (only in use in one subpackage, though some other remappings also exist).

[marcboissonneaultjda]

@dhermes While your team is fixing the issue, can you to fix the subscription_create issue at the same time?
It's exactly the same issue but for subscription and I completely forgot to mention it in my support case with @lucmult

[tseaver]

@lukesneeringer Generalizing the solution from datastore requires moving datastore._gax._catch_remap_gax_error into core, most obviously core.exceptions. Doing that requires adding a from google.cloud.core to google.gax. We have unconditional dependencies to google.gax in other packages: would that be a problem for core?

[tseaver]

The issue within pubsub seems to be that the back-end API shifted from returning FAILED_PRECONDITION to ALREADY_EXISTS: we have handling which raises Conflict for the older code.

We should add a system test (or extend and existing one) to ensure that we don't regress whatever fix we apply now.

[Saurav-Jain16]

Error is coming in below code when starting the server "(node:6924) UnhandledPromiseRejectionWarning: Error: 6 ALREADY_EXISTS: Resource already exists in the project (resource=serverone)."

Any idea why this issue is coming. Same code is working on other server

'use strict';

const http = require('http');

// Imports the Google Cloud client library
const PubSub = require('@google-cloud/pubsub');

// Your Google Cloud Platform project ID
const projectId = 'projectId';

// Instantiates a client
const pubsub = PubSub({
projectId: projectId,
keyFilename:'keyFilename.json'
});

// Reference a topic that has been previously created.
var topic = pubsub.topic('dashboardConnector');

createFlowControlledSubscription('dashboardConnector','serverone',1,10000);

function createFlowControlledSubscription (topicName, subscriptionName, maxInProgress, maxBytes) {
// Instantiates a client
// const pubsub = PubSub();

// References an existing topic, e.g. "my-topic"
const topic = pubsub.topic(topicName);

// Creates a new subscription, e.g. "my-new-subscription"
// Note that flow control configurations are not persistent
return topic.createSubscription(subscriptionName, {
flowControl: {
maxBytes: maxBytes,
maxMessages: maxInProgress
}
})
.then((results) => {
const subscription = results[0];

  subscription.on('message', function(message) {
    var attrs=message.attributes;
    var data =JSON.parse(message.data.toString('utf8'));

     
	
       message.ack();
    });

  console.log(`Subscription ${subscription.name} created with a maximum of ${maxInProgress} unprocessed messages.`);
  return subscription;
});

}