Releases · redis/lettuce

This release is a major release that introduces numerous changes like stateful connections, the reactive API and many more. Lettuce 4.0 includes all features from lettuce 3.3.

Highlights of lettuce 4.0.Final

Reactive API
Stateful connections
Cross-slot command execution
Node Selection API/Execution of commands on multiple cluster nodes
ReadFrom Settings/Redis Cluster slave reads
Custom commands

This release contains some breaking changes. You may want to consult the wiki at Migration from 3.x to 4.x to check the migration guide.

All Redis...Connection and Redis...AsyncConnection interfaces are deprecated and replaced by new ...Commands interfaces.

The cluster API was extended to run a command on multiple nodes and invocation to multi-key commands DEL, MGET, MSET and MSETNX perform automatic pipelining if the keys belong to different slots/masters.

A couple of changes are breaking changes, and you need most likely to adopt your code to use lettuce 4.0. lettuce 4.0 dropped Java 6 and 7 support and requires Java 8 to run. lettuce 4.0 needs Java 8 and cannot be used with Java 6 or 7.

Reactive API

Lettuce provides since its genesis two API's: Sync and async.
Now comes a third one: Reactive API.
The reactive API uses rx-java and every command on the reactive API returns an Observable.

StatefulRedisConnection<String, String> connection = client.connect();
RedisReactiveCommands<String, String> reactive = stateful.reactive();

Observable<String> observable = reactiveApi.keys("*");
observable.subscribe();

// more sophisticated

reactive.keys("*").flatMap(reactive::del).subscribe();

The command is only executed on a subscription to the Observable.
Commands returning a collection, such as Lists or Sets, return Observables that provide the element type. The reactive API covers the same commands as the synchronous/asynchronous API's.

Rearchitecting the API

Before 4.0, connection resources (sockets, events) were bound to the particular API.
If one wanted to use a different API, he had to create another connection to Redis. This coupling is loosed now. By calling connect() you will no longer get a synchronous connection, you will get a StatefulRedisConnection with the access to the synchronous, asynchronous and reactive API.
All commands are executed using netty and it does not matter, from which API you come.

3.x code:

RedisConnection connection = client.connect();

4.x code:

StatefulRedisConnection stateful = client.connect();

RedisCommands commands = stateful.sync();

// still working
RedisAsyncConnection connection = stateful.connectAsync();

The other connect methods like connectAsync and connectSentinelAsync are
deprecated but remain unchanged.

Breaking changes in connect methods are:

RedisClient.connect (provides a StatefulRedisConnection)
RedisClient.connectPubSub (provides a StatefulRedisPubSubConnection)
RedisClusterClient.connectCluster (provides a StatefulRedisClusterConnection)

New connect methods:

RedisClient.connectSentinel (provides a StatefulRedisSentinelConnection)

New segregated command interfaces

Starting with reactive API's, another 13 interfaces will be provided with lettuce.
This change increases the count of types within the com.lambdaworks.redis package and the package gets messier again. In combination with the stateful connection the original ...Connection or ...AsyncConnection interfaces no longer reflect the real purpose. New Commands interfaces provide the same functionality and are located in api.sync and api.async packages (respective cluster.api.sync, cluster.api.async and so on for the Redis Cluster client and PubSub).

The following interfaces are deprecated and substituted by new ...Commands interfaces:

RedisHashesAsyncConnection
RedisHashesConnection
RedisHLLAsyncConnection
RedisHLLConnection
RedisKeysAsyncConnection
RedisKeysConnection
RedisListsAsyncConnection
RedisListsConnection
RedisScriptingAsyncConnection
RedisScriptingConnection
RedisSentinelAsyncConnection
RedisServerAsyncConnection
RedisServerConnection
RedisSetsAsyncConnection
RedisSetsConnection
RedisSortedSetsAsyncConnection
RedisSortedSetsConnection
RedisStringsAsyncConnection
RedisStringsConnection
RedisClusterConnection
RedisConnection
RedisClusterAsyncConnection
RedisAsyncConnection
BaseRedisConnection
BaseRedisAsyncConnection

See Migration from 3.x to 4.x for the migration matrix.

New API's

StatefulClusterConnection
StatefulRedisPubSubConnection
StatefulClusterConnection
StatefulRedisSentinelConnection
RedisPubSubAsyncConnection and RedisPubSubConnection

API Changes

readOnly and readWrite changed from String return type to RedisFuture<String>. The connection state is maintained by the future completion.
Moved CommandOutput from com.lambdaworks.redis.protocol to com.lambdaworks.redis.output
Moved SetArgs from com.lambdaworks.redis.protocol to com.lambdaworks.redis
All connections are AutoCloseable so you can handle connections using try-with-resources.
RedisFutures are based on CompleteableFuture and throw now any occurred exception when accessing the value using get().

Exceptions are passed down the CompletionStages.

Client factory methods

RedisClient and RedisClusterClient provide factory methods (create) to create new instances. This is, to get flexibility by reducing the number of constructors and in preparation for reusable ClientResources which will be added with lettuce 4.1.

The constructors of RedisClient and RedisClusterClient are deprecated and will be removed in future versions.

RedisClient client = RedisClient.create();

RedisClient client = RedisClient.create("redis://localhost/");

RedisClient client = RedisClient.create(RedisURI.Builder.redis("localhost", 6379).build());


RedisClusterClient client = RedisClusterClient.create("redis://localhost/");

RedisClusterClient client = RedisClusterClient.create(RedisURI.Builder.redis("localhost", 6379).build());

Cross-slot command execution

Regular Redis Cluster commands are limited to single-slot keys, basically either single key commands or multi-key commands that share the same hash slot of their keys.

The cross slot limitation can be mitigated by using the advanced cluster API for some multi-key commands. Commands that operate on keys with different slots are decomposed into multiple commands.
The single commands are fired in a fork/join fashion. The commands are issued concurrently to avoid
synchronous chaining. Results are synchronized before the command is completed (from a user perspective).

Following commands are supported for cross-slot command execution:

DEL: Delete the KEYs from the affected cluster. Returns the number of keys that were removed
MGET`: Get the values of all given KEYs. Returns the values in the order of the keys.
MSET: Set multiple key/value pairs for all given KEYs. Returns always OK.

Cross-slot command execution is available on the following APIs:

RedisAdvancedClusterCommands
RedisAdvancedClusterAsyncCommands
RedisAdvancedClusterReactiveCommands

Node Selection API/Execution of commands on multiple cluster nodes

The advanced cluster API allows to select nodes and run commands on the node selection. This API is subject to incompatible changes in a future release. The API is exempt from any compatibility guarantees made by lettuce. The current state implies nothing about the quality or performance of the API in question, only the fact that it is not "API-frozen". All commands are sent concurrently to the Redis nodes, meaning you do not have to wait until the commands have finished to trigger the next command. That behavior is independent of the API you're using. The Node Selection API is available from the synchronous and asynchronous command interfaces:

Asynchronous

RedisAdvancedClusterAsyncCommands<String, String> async = clusterClient.connect().async();
AsyncNodeSelection<String, String> slaves = async.slaves();

AsyncExecutions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(result -> result.thenAccept(keys -> System.out.println(keys)));

Synchronous

RedisAdvancedClusterCommands<String, String> sync = clusterClient.connect().sync();
NodeSelection<String, String> slaves = sync.slaves();

Executions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(value -> System.out.println(value));

Commands are dispatched to the nodes within the selection, the result (CompletionStage or the value) is available from the AsyncExecutions/Executions result holders.
This API is a technical preview, so your feedback is highly appreciated.

ReadFrom Settings/Redis Cluster slave reads

The ReadFrom setting describes how lettuce routes read operations to the members of a Redis Cluster.

By default, lettuce routes its read operations to the master node. Reading from the master returns the most recent version of the data because write operations are issued to the single master node. Reading from masters guarantees strong consistency.

The ReadFrom setting can be set to one of the following presets:

MASTER Default mode. Read from the current master node.
MASTER_PREFERRED Read from the master, but if it is unavailable, read from slave nodes.
SLAVE Read from slave nodes.
NEAREST Read from any node of the cluster w...

Assets 10

02 Oct 06:03

mp911de

3.3.1.Final

1b873fa

3.3.1.Final

lettuce 3.3.1.Final is a maintenance release that fixes several problems reported
by the community.

Fixes

GEOADD passes long/lat parameters in the wrong order to Redis #134 (thanks to @IdanFridman)
Strip username from URI userinfo when creating a RedisURI with a username #131 (thanks to @jsiebens)
GeoArgs not evaluated when calling georadiusbymember(...) #142 (thanks to @codeparity)
Guava's CacheBuilder missing in shaded jar #143

API Docs: http://redis.paluch.biz/docs/api/releases/3.3.1.Final/

Assets 10

03 Sep 08:36

mp911de

4.0.Beta2

df8badd

4.0.Beta2 Pre-release

Pre-release

This release is a major release that introduces numerous changes like stateful connections, the reactive API and many more. Lettuce 4.0 includes all features from lettuce 3.3.

Highlights of lettuce 4.0.Beta2

Reactive API
Stateful connections
Cross-slot command execution
Node Selection API/Execution of commands on multiple cluster nodes
ReadFrom Settings/Redis Cluster slave reads
Custom commands

This release contains some breaking changes. You may want to consult the wiki at Migration from 3.x to 4.x to check the migration guide.

All Redis...Connection and Redis...AsyncConnection interfaces are deprecated and replaced by new ...Commands interfaces.

The cluster API was extended to run a command on multiple nodes and invocation to multi-key commands DEL, MGET, MSET and MSETNX perform automatic pipelining if the keys belong to different slots/masters.

A couple of changes are breaking changes, and you need most likely to adopt your code to use lettuce 4.0. lettuce 4.0 dropped Java 6 and 7 support and requires Java 8 to run.

lettuce 4.0 needs Java 8 and cannot be used with Java 6 or 7.

If you need any support, meet lettuce at:

Google Group: https://groups.google.com/d/forum/lettuce-redis-client-users
Gitter: https://gitter.im/mp911de/lettuce
Github Issues: https://github.com/mp911de/lettuce/issues

Reactive API

Lettuce provides since its genesis two API's: Sync and async. Now comes a third one: Reactive.
The reactive API uses rx-java and every command on the reactive API returns an Observable.

StatefulRedisConnection<String, String> connection = client.connect();
RedisReactiveCommands<String, String> reactive = stateful.reactive();

Observable<String> observable = reactiveApi.keys("*");
observable.subscribe();

// more sophisticated

reactive.keys("*").flatMap(reactive::del).subscribe();

The command is only executed on a subscription to the Observable. Commands returning a collection, such as Lists or Sets, return Observables that provide the element type. The reactive API covers the same commands as the synchronous/asynchronous API's.

Rearchitecting the API

Before 4.0, connection resources (sockets, events) were bound to the particular API. If one wanted to use a different API, he had to create another connection to Redis. This coupling is loosed now. By calling connect() you will no longer get a synchronous connection, you will get a StatefulRedisConnection with the access to the synchronous, asynchronous and reactive API.
All commands are executed using netty and it does not matter, from which API you come.

3.x code:

RedisConnection connection = client.connect();

4.x code:

StatefulRedisConnection stateful = client.connect();

RedisCommands commands = stateful.sync();

// still working
RedisConnection connection = stateful.sync();

The other connect methods like connectAsync and connectSentinelAsync are deprecated but remain unchanged.

Breaking changes in connect methods are:

RedisClient.connect (provides a StatefulRedisConnection)
RedisClient.connectPubSub (provides a StatefulRedisPubSubConnection)
RedisClusterClient.connectCluster (provides a StatefulRedisClusterConnection)

New connect methods:

RedisClient.connectSentinel (provides a StatefulRedisSentinelConnection)

New segregated command interfaces

Starting with reactive API's, another 13 interfaces will be provided with lettuce. This change increases the count of types within the com.lambdaworks.redis package and the package gets messier again. In combination with the stateful connection the original ...Connection or ...AsyncConnection interfaces no longer reflect the real purpose. New Commands interfaces provide the same functionality and are located in api.sync and api.async packages (respective cluster.api.sync, cluster.api.async and so on for the Redis Cluster client and PubSub).

The following interfaces are deprecated and substituted by new ...Commands interfaces:

RedisHashesAsyncConnection
RedisHashesConnection
RedisHLLAsyncConnection
RedisHLLConnection
RedisKeysAsyncConnection
RedisKeysConnection
RedisListsAsyncConnection
RedisListsConnection
RedisScriptingAsyncConnection
RedisScriptingConnection
RedisSentinelAsyncConnection
RedisServerAsyncConnection
RedisServerConnection
RedisSetsAsyncConnection
RedisSetsConnection
RedisSortedSetsAsyncConnection
RedisSortedSetsConnection
RedisStringsAsyncConnection
RedisStringsConnection
RedisClusterConnection
RedisConnection
RedisClusterAsyncConnection
RedisAsyncConnection
BaseRedisConnection
BaseRedisAsyncConnection

See Migration guide for the migration matrix.

New API's

StatefulClusterConnection
StatefulRedisPubSubConnection
StatefulClusterConnection
StatefulRedisSentinelConnection
RedisPubSubAsyncConnection and RedisPubSubConnection

API Changes

readOnly and readWrite changed from String return type to RedisFuture<String>. The connection state is maintained by the future completion.
Moved CommandOutput from com.lambdaworks.redis.protocol to com.lambdaworks.redis.output
Moved SetArgs from com.lambdaworks.redis.protocol to com.lambdaworks.redis
All connections are AutoCloseable so you can handle connections using try-with-resources.
RedisFutures are based on CompleteableFuture and throw now any occurred exception when accessing the value using get().
Exceptions are passed down the CompletionStages.

Cross-slot command execution

Regular Redis Cluster commands are limited to single-slot keys, basically either single key commands or multi-key commands that share the same hash slot of their keys.

The cross slot limitation can be mitigated by using the advanced cluster API for some multi-key commands. Commands that operate on keys with different slots are decomposed into multiple commands. The single commands are fired in a fork/join fashion. The commands are issued concurrently to avoid synchronous chaining. Results are synchronized before the command is completed (from a user perspective).

Following commands are supported for cross-slot command execution:

DEL: Delete the KEYs from the affected cluster. Returns the number of keys that were removed
MGET: Get the values of all given KEYs. Returns the values in the order of the keys.
MSET: Set multiple key/value pairs for all given KEYs. Returns always OK.

Cross-slot command execution is available on the following APIs:

RedisAdvancedClusterCommands
RedisAdvancedClusterAsyncCommands
RedisAdvancedClusterReactiveCommands

Node Selection API/Execution of commands on multiple cluster nodes

Asynchronous

RedisAdvancedClusterAsyncCommands<String, String> async = clusterClient.connect().async();
AsyncNodeSelection<String, String> slaves = connection.slaves();

AsyncExecutions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(result -> result.thenAccept(keys -> System.out.println(keys)));

Synchronous

RedisAdvancedClusterCommands<String, String> async = clusterClient.connect().sync();
NodeSelection<String, String> slaves = connection.slaves();

Executions<List<String>> executions = slaves.commands().keys("*");
executions.stream().forEach(value -> System.out.println(value));

Commands are dispatched to the nodes within the selection, the result (CompletionStage or the value) is available from the AsyncExecutions/Executions result holders. This API is a technical preview, so your feedback is highly appreciated.

ReadFrom Settings/Redis Cluster slave reads

The ReadFrom setting describes how lettuce routes read operations to the members of a Redis Cluster.

The ReadFrom setting can be set to one of the following presets:

MASTER Default mode. Read from the current master node.
MASTER_PREFERRED Read from the master, but if it is unavailable, read from slave nodes.
SLAVE Read from slave nodes.
NEAREST Read from any node of the cluster with the lowest latency.

Custom read settings can be implemented by extending the com.lambdaworks.redis.ReadFrom class.

Custom commands

Lettuce covers nearly all Redis commands. Redis development is an ongoing process, and some commands are not covered by lettuce meaning there are use cases that require invocation of custom commands or custom outputs. lettuce 4.x allows you to trigger own commands. That API is used by lettuce itself to dispatch commands and requires some knowledge of how commands are constructed and dispatched within lettuce.

StatefulRe...

Assets 10

27 Aug 15:05

mp911de

3.3.Final

8679420

3.3.Final

lettuce 3.3 introduces a variety of changes: It features changes to cluster support, new Geo commands for the upcoming Redis 3.2 release and allows batching, an extension to pipelining.

The documentation within the lettuce wiki was overhauled.
Meet the very new wiki at https://github.com/mp911de/lettuce/wiki

The test stability issue was addressed by running infinite testing jobs. Instabilities were caused due to missing synchronization and relying too much on timing. However, the real life teaches us, not to rely on time, rather on hard facts whether a cluster node is available, or a command is written out to the transport. The tests are improved and fail less often.

I'm very excited to present you lettuce 3.3.Final. Read on for the details or jump to the end to find the summary and the download links.

Redis Cluster: Node connections

lettuce's cluster support is enhanced by a couple of features. Users can obtain a connection to the particular cluster nodes by specifying either the nodeId or host and port. The new methods are available on two new interfaces:

RedisAdvancedClusterConnection
RedisAdvancedClusterAsyncConnection

Example code:

RedisAdvancedClusterAsyncConnection<String, String> connection = clusterClient.connectClusterAsync();
RedisClusterAsyncConnection<String, String> nodeConnection = connection
                           .getConnection("adf7f86efa42d903bcd93c5bce72397fe52e91bb");

...

RedisClusterAsyncConnection<String, String> nodeConnection = connection.getConnection("localhost", 7379);

You are free to operate on these connections. Connections can be bound to specific hosts or nodeId's. Connections bound to a nodeId will always stick to the nodeId, even if the nodeId is handled by a different host. Requests to unknown nodeId's or host/ports that are not part of the cluster are rejected.
Do not close the connections. Otherwise, unpredictable behavior will occur. Keep also in mind, that the node connections are used by the cluster connection itself to perform cluster operations: If you block one connection all other users of the cluster connection might be affected.

The cluster client handles ASK redirection the same way as MOVED redirection. It is transparent to the client. Dispatching the commands between the particular cluster connections was optimized, and the performance was improved.

Cluster topology view refreshing

Another cluster-related feature is the cluster topology view refresh. Reloading the partitions was possible since lettuce 3.0. Version 3.3 enabled the reloading on a regular basis in the background. The background update will close connections to nodeId's/hosts that are no longer part of the cluster. You can enable the refresh job (disabled by default) and specify the interval (defaults to 60 seconds). The refresh can be configured in the ClusterClientOptions.

Example code:

clusterClient.setOptions(new ClusterClientOptions.Builder()
                                 .refreshClusterView(true)
                                 .refreshPeriod(5, TimeUnit.SECONDS)
                                 .build());

RedisAdvancedClusterAsyncConnection<String, String> clusterConnection = clusterClient.connectClusterAsync();

Pipelining/Batching

Lettuce operates using pipelining described in http://redis.io/topics/pipelining.

What is different now? lettuce performs a flush after each command invocation on the transport. This is fine for the most use cases, but flushing can become a limiting factor when bulk loading, or you need batching.

Asynchronous connections allow you to disable the auto-flush behavior and give control over flushing the queued commands:

Example code:

RedisAsyncConnection<String, String> connection = client.connectAsync();
connection.setAutoFlushCommands(false);

connection.set("key", "value");
connection.set("key2", "value2");

connection.flushCommands(); // send the two SET commands out to the transport

connection.setAutoFlushCommands(true);

Why on the asynchronous connections only? The asynchronous connections return already a handle to the result, the synchronous API does not. Adding another API would require to duplicate all interfaces and increase complexity. Pipelining/Batching can improve your throughput. Pipelining also works with Redis Cluster and is not available on pooled connections.

Codecs

lettuce 3.3 uses a dedicated String codec instance for each String-encoded connection (Standalone) instead of sharing one String codec for all connections. Cluster connections share a String codec between the internal connections.

A new ByteArrayCodec ships with lettuce 3.3 that allows byte array connections
without the need to create an own codec.

Geo commands

This release supports the Geo-commands of the upcoming Redis 3.2 release. The Geo-API allows to maintain a set (backed by a Redis sorted set) of Geo locations described by WGS84 coordinates. You can add and query set members using the new Geo-API. Use ZREM to remove members from the Geo set until redis/redis#2674 is resolved.

The design of the Geo-API within lettuce, differs from other APIs in lettuce. The response structures of GEORADIUS depend on the command input, and there are other languages that fit better into the Redis response structure patterns. The static type checking within Java would only allow a List<Object> (or Object)
which contains nested Lists and Maps carrying the data. You would have to cast the elements to maps or lists and access then again the nested elements. Working with Lists and Maps in Java is less convenient compared to JavaScript or Ruby.

The Geo-API provides GeoCoordinates and GeoWithin types that allow direct access to the response values such as distance or the coordinate points.

Example code:

redis.geoadd(key, 8.6638775, 49.5282537, "Weinheim",
                  8.3796281, 48.9978127, "Office tower",
                  8.665351, 49.553302, "Train station");

Set<String> georadius = redis.georadius(key, 8.6582861, 49.5285695,
                                               5, GeoArgs.Unit.km);

// georadius contains "Weinheim" and "Train station"

Double distance = redis.geodist(key, "Weinheim", "Train station", GeoArgs.Unit.km);

// distance ≈ 2.78km

GeoArgs geoArgs = new GeoArgs().withHash()
                               .withCoordinates()
                               .withDistance()
                               .withCount(1)
                               .desc();

List<GeoWithin<String>> georadiusWithArgs = redis.georadius(key,
                                                        8.665351, 49.553302,
                                                        5, GeoArgs.Unit.km,
                                                        geoArgs);

// georadiusWithArgs contains "Weinheim" and "Train station"
// ordered descending by distance and containing distance/coordinates

Command execution reliability

A new document describes Command execution reliability in the context of lettuce and reconnects. In general, lettuce supports at-least-once and at-most-once semantics. The mode of operations is bound to the auto-reconnect flag in the client options.

If auto-reconnect is enabled, at-least-once semantics applies, at-most-once if auto-reconnect is disabled.

At-least-once and at-most-once are not new to lettuce. The documentation just explains how these semantics apply to lettuce.

Enhancements

Provide access to cluster connection using the advanced cluster API #71
Cluster connection failover when cluster topology changes #97
NodeId-bound cluster connections enhancement #104
Implement at-least-once for cluster connections #105
Pipelining for lettuce (or: flush after n commands) #92
Decouple ConnectionWatchdog reconnect from timer thread #100
Improve performance in 3.3 #90
Add checks for arguments #69
Use dedicated string codecs on String connections and add ByteArrayCodec #70
Tests for command reliability docs #98
Expose RedisClient.connect(RedisCodec, RedisURI) and RedisClient.connectAsync(RedisCodec, RedisURI) #108
Add close stale connections and strict cluster member check flags to ClusterClientOptions #109

Commands

Adopt variadic EXISTS and DEBUG HTSTATS #103
Support geo commands in lettuce 3.3 #86
Support NX|XX|CH|INCR options in ZADD #74
Add support for COUNTKEYSINSLOT #107
Add support for HSTRLEN #117

Fixes

Synchronization/cross thread visibility of variables #94
Check channel state before calling Channel.close.syncUninterruptibly #113
Stop the HashedWheelTimer before shutting down EventLoopGroups #123

Other

Rework of the wiki

http://redis.paluch.biz/docs/api/releases/3.3.Final/

Assets 10

06 Aug 12:42

mp911de

4.0.Beta1

b164f5f

4.0.Beta1 Pre-release

Pre-release

This release is a major release that introduces numerous changes like stateful connections, the reactive API and many more. Lettuce 4.0 includes all features from lettuce 3.3.

This release contains some breaking changes. You may want to consult the wiki to check the migration guide.

All Redis...Connection and Redis...AsyncConnection interfaces are deprecated and replaced by new ...Commands interfaces.

The cluster API was extended to run a command on multiple nodes and invocation to multi-key commands DEL, MGET, and MSET perform automatic pipelining if the keys belong to different slots/masters.

A couple of changes are breaking changes, and you need most likely to adopt your code to use lettuce 4.0. lettuce 4.0 dropped Java 6 and 7 support and requires Java 8 to run.

lettuce 4.0 needs Java 8 and cannot be used with Java 6 or 7.

Reactive API

Lettuce provides since its genesis two API's: Sync and async. Now comes a third one: Reactive.
The reactive API uses rx-java and every command on the reactive API returns an Observable.

StatefulRedisConnection<String, String> connection = client.connect();
RedisReactiveCommands<String, String> reactive = stateful.reactive();

Observable<String> observable = reactiveApi.keys("*");
observable.subscribe();

// chaining

reactive.keys("*").flatMap(reactive::del).subscribe();

The command is only executed on a subscription to the Observable.Commands returning a collection, such as Lists or Sets, return Observables that provide the element type. The reactive API covers the same commands as the synchronous/asynchronous API's.

Rearchitecting the API

Before 4.0, connection resources (sockets, events) were bound to the particular API. If one wanted to use a different API, he had to create another connection to Redis. This coupling is loosed now. By calling connect() you will no longer get a synchronous connection, you will get a StatefulRedisConnection with the access to the synchronous, asynchronous and reactive API. All commands are executed using netty and it does not matter, from which API you come.

3.x code:

RedisConnection connection = client.connect();

4.x code:

StatefulRedisConnection stateful = client.connect();

RedisCommands commands = stateful.sync();

// still working
RedisConnection connection = stateful.sync();

The other connect methods like connectAsync and connectSentinelAsync are deprecated but remain unchanged.

Breaking changes in connect methods are:

RedisClient.connect (provides a StatefulRedisConnection)
RedisClient.connectPubSub (provides a StatefulRedisPubSubConnection)
RedisClusterClient.connectCluster (provides a StatefulRedisClusterConnection)

New connect methods:

RedisClient.connectSentinel (provides a StatefulRedisSentinelConnection)

New segregated command interfaces

Starting with reactive API's, another 13 interfaces will be provided with lettuce. This change increases the count of types within the com.lambdaworks.redis package and the package gets messier again. In combination with the stateful connection the original ...Connection or ...AsyncConnection interfaces no longer reflect the real purpose. New Commands interfaces provide the same functionality and are located in api.sync and api.async packages (respective cluster.api.sync, cluster.api.async and so on for the Redis Cluster client and PubSub).

The following interfaces are deprecated and substituted by new ...Commands interfaces:

RedisHashesAsyncConnection
RedisHashesConnection
RedisHLLAsyncConnection
RedisHLLConnection
RedisKeysAsyncConnection
RedisKeysConnection
RedisListsAsyncConnection
RedisListsConnection
RedisScriptingAsyncConnection
RedisScriptingConnection
RedisSentinelAsyncConnection
RedisServerAsyncConnection
RedisServerConnection
RedisSetsAsyncConnection
RedisSetsConnection
RedisSortedSetsAsyncConnection
RedisSortedSetsConnection
RedisStringsAsyncConnection
RedisStringsConnection
RedisClusterConnection
RedisConnection
RedisClusterAsyncConnection
RedisAsyncConnection
BaseRedisConnection
BaseRedisAsyncConnection

See the wiki for the migration matrix.

New API's

StatefulClusterConnection
StatefulRedisPubSubConnection
StatefulClusterConnection
StatefulRedisSentinelConnection
RedisPubSubAsyncConnection and RedisPubSubConnection

API Changes

readOnly and readWrite changed from String return type to RedisFuture<String>. The connection state is maintained by the future completion.
Moved CommandOutput from com.lambdaworks.redis.protocol to com.lambdaworks.redis.output
Moved SetArgs from com.lambdaworks.redis.protocol to com.lambdaworks.redis
All connections are AutoCloseable so you can handle connections using try-with-resources.
RedisFutures are based on CompleteableFuture and throw now any occurred exception when accessing the value using get(). Exceptions are passed down the CompletionStages.

Cross-slot command execution

Regular Redis Cluster commands are limited to single-slot keys, basically either single key commands or multi-key commands that share the same hash slot of their keys.

The cross slot limitation can be mitigated by using the advanced cluster API for some
multi-key commands. Commands that operate on keys with different slots are decomposed into multiple commands. The single commands are fired in a fork/join fashion. The commands are issued concurrently to avoid synchronous chaining. Results are synchronized before the command is completed (from a user perspective).

Following commands are supported for cross-slot command execution:

DEL: Delete the KEYs from the affected cluster. Returns the number of keys that were removed
MGET: Get the values of all given KEYs. Returns the values in the order of the keys.
MSET: Set multiple key/value pairs for all given KEYs. Returns always OK.

Cross-slot command execution is available on the following APIs:

RedisAdvancedClusterCommands
RedisAdvancedClusterAsyncCommands
RedisAdvancedClusterReactiveCommands

Node Selection API/Execution of commands on multiple cluster nodes

The advanced cluster API allows to select nodes and run commands on the node selection:

RedisAdvancedClusterAsyncCommands<String, String> async = clusterClient.connect().async();
AsyncNodeSelection<String, String> slaves = connection.slaves();

AsyncExecutions<List<String>> executions = slaves.commands().keys("*");
executions.forEach(result -> result.thenAccept(keys -> System.out.println(keys)));

Commands are dispatched to the nodes within the selection, the result (CompletionStage) is available through AsyncExecutions. This API is currently only available for async commands and a technical preview so your feedback is highly appreciated.

Updated dependencies

rxjava 1.0.13 (new)
Google Guava 17.0 -> 18.0
netty 4.0.28.Final 4.0.30.Final
commons-pool2 2.2 -> 2.4.2

Enhancements

Advanced Cluster API (async) #78
Improve HLL command interface #77
Move segregated API interfaces to own packages #76
Provide a stateful Redis connection and decouple sync/async API from connection resources #75
Reactive support #68
Pipelining for certain cluster commands #66
Drop support for Java 6 and Java 7 #50
Migrate RedisFuture to CompletionStage #48

lettuce requires a minimum of Java 8 to build and run. It is tested continuously against the latest Redis source-build.

Javadoc: http://redis.paluch.biz/docs/api/snapshots/4.0.Beta1/

Assets 10

06 Aug 13:11

mp911de

3.3.Beta1

8a67c6e

3.3.Beta1 Pre-release

Pre-release

lettuce 3.3 introduces a variety of changes: It features changes to cluster support, new GEO commands for the upcoming Redis 3.2 release and allows batching, an extension to pipelining.

The documentation within the lettuce wiki was overhauled.
Meet the very new wiki at https://github.com/mp911de/lettuce/wiki

Redis Cluster: Node connections

RedisAdvancedClusterConnection
RedisAdvancedClusterAsyncConnection

Example code:

RedisAdvancedClusterAsyncConnection<String, String> connection = clusterClient.connectClusterAsync();
RedisClusterAsyncConnection<String, String> nodeConnection = connection
                           .getConnection("adf7f86efa42d903bcd93c5bce72397fe52e91bb");

...

RedisClusterAsyncConnection<String, String> nodeConnection = connection.getConnection("localhost", 7379);

The cluster client handles now ASK redirecting the same way as MOVED redirection works. It is transparent for the client. Dispatching the commands between the particular cluster connections was optimized, and the performance was improved.

Cluster topology view refreshing

Another cluster-related item is the cluster topology view refreshing. Reloading the partitions was possible since lettuce 3.0. Version 3.3 enabled the reloading on a regular basis in the background. The background update will close connections to nodeIds/hosts that are no longer part of the cluster. You are able to enable the refresh job (disabled by default) and specify the interval (defaults to 60 seconds). The refresh is enabled by using ClusterClientOptions.

Example code:

clusterClient.setOptions(new ClusterClientOptions.Builder()
                                 .refreshClusterView(true)
                                 .refreshPeriod(5, TimeUnit.SECONDS)
                                 .build());

RedisAdvancedClusterAsyncConnection<String, String> clusterConnection = clusterClient.connectClusterAsync();

Pipelining

Lettuce operates using pipelining described in http://redis.io/topics/pipelining. What is different now? lettuce performs a flush after each command invocation on the transport. This is fine for the most use cases, but flushing can become expensive and a limiting factor when you are bulk loading, or you need batching.
The async connections allow you to disable the auto-flush behavior and flush the queued commands at the moment you want to send them over the transport:

Example code:

RedisAsyncConnection<String, String> connection = client.connectAsync();
connection.setAutoFlushCommands(false);

connection.set("key", "value");
connection.set("key2", "value2");

connection.flushCommands();

connection.setAutoFlushCommands(true);

Why on the async connections only? The async connections return already a handle to the result, the sync API does not. Adding another API would require to duplicate all interfaces and cause the code to be much more complex. Pipelining/Batching can improve your throughput. Pipelining also works with Redis Cluster and is disabled on pooled connections.

Codecs

lettuce 3.3 uses a dedicated String codec instance for each String connection (Standalone) instead of sharing one String codec for all connections. Cluster connections share a String codec between the internal connections.

A new ByteArrayCodec ships with lettuce 3.3 that allows byte array connections without the need to create an own codec.

Geo commands

This release supports the Geo-commands of the upcoming Redis 3.2 release. The Geo-API allows to maintain a set (backed by a Redis sorted set) of Geo points described by WGS84 coordinates. You can add and query set members using the new Geo-API. Use ZREM to remove members from the
until redis/redis#2674 is resolved.

The design of the Geo-API within lettuce, differs from other APIs in lettuce. The response structures of GEORADIUS depend on the command input, and there are other languages that fit better into the Redis response structure patterns. The static type checking within Java would only allow a List<Object> (or Object) which contains nested Lists and Maps carrying the data. You would have to cast the elements to maps or lists and access then again the nested elements. Working with Lists and Maps in Java is less convenient compared to JavaScript or Ruby.

The Geo-API features GeoCoordinates and GeoWithin types that allow direct access to the response values such as distance or the coordinate points.

Example code:

redis.geoadd(key, 8.6638775, 49.5282537, "Weinheim",
                  8.3796281, 48.9978127, "Office tower",
                  8.665351, 49.553302, "Train station");

Set<String> georadius = redis.georadius(key, 8.6582861, 49.5285695,
                                               5, GeoArgs.Unit.km);

// georadius contains "Weinheim" and "Train station"

Double distance = redis.geodist(key, "Weinheim", "Train station", GeoArgs.Unit.km);

// distance ≈ 2.78km

GeoArgs geoArgs = new GeoArgs().withHash()
                               .withCoordinates()
                               .withDistance()
                               .withCount(1)
                               .desc();

List<GeoWithin<String>> georadiusWithArgs = redis.georadius(key,
                                                        8.665351, 49.553302,
                                                        5, GeoArgs.Unit.km,
                                                        geoArgs);

// georadiusWithArgs contains "Weinheim" and "Train station"
// ordered descending by distance and containing distance/coordinates

Command execution reliability

A new doc describes Command execution reliability in the context of lettuce and reconnects. In general, lettuce supports at-least-once and at-most-once semantics. The mode of operations is bound to the auto-reconnect flag in the client options.

If auto-reconnect is enabled, at-least-once semantics applies, at-most-once if auto-reconnect is disabled.

At-least-once and at-most-once are not new to lettuce. The documentation just explains how these semantics apply to lettuce.

Enhancements

Provide access to cluster connection using the advanced cluster API #71
Cluster connection failover when cluster topology changes #97
NodeId-bound cluster connections enhancement #104
Implement at-least-once for cluster connections #105
Pipelining for lettuce (or: flush after n commands) #92
Decouple ConnectionWatchdog reconnect from timer thread #100
Improve performance in 3.3 #90
Add checks for arguments #69
Use dedicated string codecs on String connections and add ByteArrayCodec #70
Tests for command reliability docs #98
Expose RedisClient.connect(RedisCodec, RedisURI) and RedisClient.connectAsync(RedisCodec, RedisURI) #108
Add close stale connections and strict cluster member check flags to ClusterClientOptions #109

Commands

Adopt variadic EXISTS and DEBUG HTSTATS #103
Support geo commands in lettuce 3.3 #86
Support NX|XX|CH|INCR options in ZADD #74
Add support for COUNTKEYSINSLOT #107

Fixes

Synchronization/cross thread visibility of variables #94
Check channel state before calling Channel.close.syncUninterruptibly #113

Other

Rework of the wiki

Javadoc: http://redis.paluch.biz/docs/api/snapshots/3.3.Beta1/

Assets 10

28 May 16:09

mp911de

3.2.Final

f754e5a

3.2.Final

This release features Unix domain sockets on linux-x86_64 systems. Local connections to a Redis instance are now possible without the use of the network. A second feature with a huge impact is client options.
Client options allow to control behavior on a fine-grained base. It is now possible to turn off auto-reconnect, validate the connection with a PING before activating the connection for client usage and many more.

We tested lettuce using YourKit for performance. Subtracting all IO, the most time is spent on constructing the return futures and command objects. The async API can issue about 300Kops/sec whereas the sync API performs about 13Kops/sec (GET's, PING's) on a single thread.
Sync performance depends on the Redis performance (tested on a Mac, 2,7 GHz Intel Core i7).

lettuce provides now also base functionality for building clients using the Redis protocol.
An example is spinach (https://github.com/mp911de/spinach).

lettuce is built with Java 8 but can be used with Java 6, 7 and 8. Java 6 and 7 support will be
dropped with lettuce 4.0. The new lettuce-testcontainer project ensures until then.
You can find the build results at https://travis-ci.org/mp911de/lettuce-testcontainer

If you need any support, meet lettuce at https://gitter.im/mp911de/lettuce

Unix Domain Sockets

Unix Domain Sockets are inter-process communication channels on POSIX compliant systems. They allow to exchange data between processes on the same host operating system. When using Redis, which is usually a network service, Unix Domain Sockets are usable only if connecting locally to a single instance. Redis Sentinel and Redis Cluster,maintain tables of remote or local nodes and act therefore as registry. Unix Domain Sockets are not beneficial with Redis Sentinel and Redis Cluster.
Using RedisClusterClient with Unix Domain Sockets would connect to the local node using a socket and then open TCP connection to all the other hosts. A good example is connecting locally to a standalone or a single cluster node to gain performance.

Unix Domain Socket connections are currently only available to Linux x86_64 systems with a minimum netty version of 4.0.26.Final. You can, however, port the native library to your system. The netty guys would appreciate.

See https://github.com/mp911de/lettuce/wiki/Unix-Domain-Sockets

Enhancements

Provide an option to connect to Redis via Unix Domain Sockets #52
Add new cluster command MYID #53 (Thanks to @taer)
Expose futures of re-subscription on reconnect #62 (Thanks to @kichik)
Ping before connect #61 (Thanks to @kichik)
Introduce ClientOptions to control specific behavior #58
Enhance stability when reconnecting #56
Extract and use interfaces for improved extensibility
Upgrade netty to 4.0.28.Final

Fixes

RedisPubSubConnectionImpl.activated does not call super.activated() #55
RedisPubSubConnectionImpl.activated() deadlock #57
Fix zrevrangebyscoreWithScores signature bug #65
Handle absence of optional epoll library in a reasonable way bug #64

Other

Inspect lettuce using YourKit #54

lettuce requires a minimum of Java 8 to build and Java 6 run. It is tested continuously against Redis 3.0.

For complete information on lettuce see the websites:

Assets 8

Releases: redis/lettuce

4.0.2.Final

Fixes

3.3.2.Final

Fixes

4.0.1.Final

Fixes

4.0.Final

Reactive API

Rearchitecting the API

New segregated command interfaces

New API's

API Changes

Client factory methods

Cross-slot command execution

Node Selection API/Execution of commands on multiple cluster nodes

ReadFrom Settings/Redis Cluster slave reads

3.3.1.Final

Fixes

4.0.Beta2

Reactive API

Rearchitecting the API

New segregated command interfaces

New API's

API Changes

Cross-slot command execution

Node Selection API/Execution of commands on multiple cluster nodes

ReadFrom Settings/Redis Cluster slave reads

Custom commands

3.3.Final

Redis Cluster: Node connections

Cluster topology view refreshing

Pipelining/Batching

Codecs

Geo commands

Command execution reliability

Enhancements

Commands

Fixes

Other

4.0.Beta1

Reactive API

Rearchitecting the API

New segregated command interfaces

New API's

API Changes

Cross-slot command execution

Node Selection API/Execution of commands on multiple cluster nodes

Updated dependencies

Enhancements

3.3.Beta1

Redis Cluster: Node connections

Cluster topology view refreshing

Pipelining

Codecs

Geo commands

Command execution reliability

Enhancements

Commands

Fixes

Other

3.2.Final

Unix Domain Sockets

Enhancements

Fixes

Other