Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TINKERPOP-2555 - Remote Transaction Support (g.tx()) for gremlin-python #1515

Merged
merged 5 commits into from
Dec 16, 2021
Merged

TINKERPOP-2555 - Remote Transaction Support (g.tx()) for gremlin-python #1515

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6f45069
Source:
lyndonbauto Dec 8, 2021
dd796be
[1] Fixed TEST_TRANSACTION environment variable
lyndonbauto Dec 8, 2021
2594cf5
[1] Added session support to string messages. This was unintentionall…
lyndonbauto Dec 8, 2021
d1e3abd
Added submitAsync in Client and DriverRemoteConnection with deprecate…
lyndonbauto Dec 10, 2021
5a0a835
Changed logic for disabling transactions within tests.
lyndonbauto Dec 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ jobs:
run: |
touch gremlin-python/.glv
mvn clean install -pl -:gremlin-javascript,-gremlin-dotnet,-:gremlin-dotnet-source,-:gremlin-dotnet-tests,-:gremlint -q -DskipTests -Dci
mvn verify -pl gremlin-python
mvn verify -pl gremlin-python -DincludeNeo4j
dotnet:
name: .NET
timeout-minutes: 1200 # 20 minutes
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
[[release-3-5-2]]
=== TinkerPop 3.5.2 (Release Date: NOT OFFICIALLY RELEASED YET)

* Added support for `g.Tx()` in Python.
* Added logging in in Python.
* Fixed shutdown cleanup issue in Python aiohttp transport layer.
* Added a `NoSugarTranslator` translator to `PythonTranslator` which translates Gremlin queries to Python without syntactic sugar (ex `g.V().limit(1)` instead of `g.V()[0:1]`)
* Added support for `g.Tx()` in .NET.
* Added support for `with()` constant options to `io()`.
Expand Down
31 changes: 31 additions & 0 deletions docs/src/reference/gremlin-variants.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -913,6 +913,37 @@ Apache TinkerPop's JVM-based Gremlin traversal machine. As such, their `apply(Tr
the strategy is encoded in the Gremlin-Python bytecode and transmitted to the Gremlin traversal machine for
re-construction machine-side.

[[gremlin-python-transactions]]
=== Transactions

To get a full understanding of this section, it would be good to start by reading the <<transactions,Transactions>>
section of this documentation, which discusses transactions in the general context of TinkerPop itself. This section
builds on that content by demonstrating the transactional syntax for Javascript.

[source,python]
----
g = traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin'))

# Create a Transaction.
tx = g.tx()

# Spawn a new GraphTraversalSource, binding all traversals established from it to tx.
gtx = tx.begin()

try:
# Execute a traversal within the transaction.
gtx.addV("person").property("name", "Lyndon").iterate(),

# Commit the transaction. The transaction can no longer be used and cannot be re-used.
# A new transaction can be spawned through g.tx().
# The context of g remains sessionless throughout the process.
gtx.commit()
except Exception e:
# Rollback the transaction if an error occurs.
gtx.rollback()

----

[[gremlin-python-lambda]]
=== The Lambda Solution

Expand Down
17 changes: 12 additions & 5 deletions docs/src/upgrade/release-3.5.x.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,20 @@ complete list of all the modifications that are part of this release.

=== Upgrading for Users

==== tx() in .NET
==== Tx() in .NET and tx() in Python

After Javascript, .NET is now the second non-JVM variant of Gremlin to get support for
link:https://tinkerpop.apache.org/docs/3.5.2/reference/#transactions[remote transactions]. An example of the `Tx()`
syntax can be found in the .NET link:https://tinkerpop.apache.org/docs/3.5.2/reference/#gremlin-dotnet-transactions[Transaction Section].
After Javascript, .NET and Python are now the second and third non-JVM variants of Gremlin to get support for
link:https://tinkerpop.apache.org/docs/3.5.2/reference/#transactions[remote transactions].

See: link:https://issues.apache.org/jira/browse/TINKERPOP-2556[TINKERPOP-2556]
An example of the .NET `Tx()` syntax can be found in the
link:https://tinkerpop.apache.org/docs/3.5.2/reference/#gremlin-dotnet-transactions[.NET Transaction Section].

An example of the Python `tx()` syntax can be found in the
link:https://tinkerpop.apache.org/docs/3.5.2/reference/#gremlin-dotnet-transactions[Python Transaction Section]


See: link:https://issues.apache.org/jira/browse/TINKERPOP-2556[TINKERPOP-2556] for .NET and
link:https://issues.apache.org/jira/browse/TINKERPOP-2555[TINKERPOP-2555] for Python

==== datetime()

Expand Down
122 changes: 122 additions & 0 deletions gremlin-python/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ limitations under the License.
<!-- provides a way to convert maven.test.skip value to skipTests for use in skipping python tests -->
<maven.test.skip>false</maven.test.skip>
<skipTests>${maven.test.skip}</skipTests>
<TEST_TRANSACTIONS>false</TEST_TRANSACTIONS>
<gremlin.server.dir>${project.parent.basedir}/gremlin-server</gremlin.server.dir>
</properties>
<build>
Expand Down Expand Up @@ -216,6 +217,7 @@ limitations under the License.
<target>
<exec executable="env/bin/python" dir="${project.build.directory}/python3"
failonerror="true">
<env key="TEST_TRANSACTIONS" value="${TEST_TRANSACTIONS}"/>
<env key="PYTHONPATH" value=""/>
<env key="KRB5_CONFIG" value="${project.build.directory}/kdc/krb5.conf"/>
<env key="KRB5CCNAME" value="${project.build.directory}/kdc/test-tkt.cc"/>
Expand All @@ -224,19 +226,22 @@ limitations under the License.
<!-- radish seems to like all dependencies in place -->
<exec executable="env/bin/python" dir="${project.build.directory}/python3"
failonerror="true">
<env key="TEST_TRANSACTIONS" value="${TEST_TRANSACTIONS}"/>
<env key="PYTHONPATH" value=""/>
<arg line="setup.py install"/>
</exec>
<!-- run for graphson 3.0 -->
<exec executable="env/bin/radish" dir="${project.build.directory}/python3"
failonerror="true">
<env key="TEST_TRANSACTIONS" value="${TEST_TRANSACTIONS}"/>
<env key="PYTHONPATH" value=""/>
<env key="PYTHONIOENCODING" value="utf-8:surrogateescape"/>
<arg line="-f dots -e -t -b ${project.build.directory}/python3/radish ${project.basedir}/../gremlin-test/features/ --user-data=&quot;serializer=application/vnd.gremlin-v3.0+json&quot;"/> <!-- -no-line-jump -->
</exec>
<!-- run for graphbinary 1.0 -->
<exec executable="env/bin/radish" dir="${project.build.directory}/python3"
failonerror="true">
<env key="TEST_TRANSACTIONS" value="${TEST_TRANSACTIONS}"/>
<env key="PYTHONPATH" value=""/>
<env key="PYTHONIOENCODING" value="utf-8:surrogateescape"/>
<arg line="-f dots -e -t -b ${project.build.directory}/python3/radish ${project.basedir}/../gremlin-test/features/ --user-data=&quot;serializer=application/vnd.graphbinary-v1.0&quot;"/> <!-- -no-line-jump -->
Expand All @@ -260,6 +265,11 @@ limitations under the License.
<artifactId>gremlin-test</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.tinkerpop</groupId>
<artifactId>neo4j-gremlin</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
Expand Down Expand Up @@ -354,6 +364,118 @@ limitations under the License.
</plugins>
</build>
</profile>
<!--
This profile will include neo4j for purposes of transactional testing within Gremlin Server.
Tests that require neo4j specifically will be "ignored" if this profile is not turned on.
-->
<profile>
<id>include-neo4j</id>
<activation>
<activeByDefault>false</activeByDefault>
<property>
<name>includeNeo4j</name>
</property>
</activation>
<properties>
<TEST_TRANSACTIONS>true</TEST_TRANSACTIONS>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<dependencies>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-tinkerpop-api-impl</artifactId>
<version>0.9-3.4.0</version>
<exclusions>
<exclusion>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-text</artifactId>
</exclusion>
<exclusion>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</exclusion>
<exclusion>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
</exclusion>
<exclusion>
<groupId>org.scala-lang</groupId>
<artifactId>scala-reflect</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-nop</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
</exclusion>
<exclusion>
<groupId>io.dropwizard.metrics</groupId>
<artifactId>metrics-core</artifactId>
</exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
</exclusion>
<exclusion>
<groupId>org.ow2.asm</groupId>
<artifactId>asm</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.8</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-reflect</artifactId>
<version>2.11.8</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>5.5.0</version>
</dependency>
<dependency>
<groupId>io.dropwizard.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>4.0.2</version>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-kernel</artifactId>
<version>3.4.11</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>
<!--
Provides a way to deploy the gremlinpython GLV to pypi. This cannot be part of the standard maven execution
because pypi does not have a staging environment like sonatype for releases. As soon as the release is
Expand Down
15 changes: 11 additions & 4 deletions gremlin-python/src/main/python/gremlin_python/driver/aiohttp/transport.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,11 @@
# specific language governing permissions and limitations
# under the License.
#


import aiohttp
import asyncio
import async_timeout
from aiohttp import ClientResponseError
import logging
lyndonbauto marked this conversation as resolved.
Show resolved Hide resolved

from gremlin_python.driver.transport import AbstractBaseTransport

Expand Down Expand Up @@ -58,6 +57,11 @@ def __init__(self, call_from_event_loop=None, read_timeout=None, write_timeout=N
if "ssl_options" in self._aiohttp_kwargs:
self._aiohttp_kwargs["ssl"] = self._aiohttp_kwargs.pop("ssl_options")

def __del__(self):
# Close will only actually close if things are left open, so this is safe to call.
# Clean up any connection resources and close the event loop.
self.close()

def connect(self, url, headers=None):
# Inner function to perform async connect.
async def async_connect():
Expand Down Expand Up @@ -116,10 +120,13 @@ async def async_read():
def close(self):
# Inner function to perform async close.
async def async_close():
if not self._websocket.closed:
if self._websocket is not None and not self._websocket.closed:
await self._websocket.close()
if not self._client_session.closed:
self._websocket = None

if self._client_session is not None and not self._client_session.closed:
await self._client_session.close()
self._client_session = None

# If the loop is not closed (connection hasn't already been closed)
if not self._loop.is_closed():
Expand Down
Loading