Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extension for redis cluster mode #173

Merged
merged 11 commits into from
Jul 18, 2020
Merged
14 changes: 14 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,7 @@ The following configuration values exist for Flask-Caching:
* **filesystem**: FileSystemCache
* **redis**: RedisCache (redis required)
* **redissentinel**: RedisSentinelCache (redis required)
* **rediscluster**: RedisClusterCache (redis required)
* **uwsgi**: UWSGICache (uwsgi required)
* **memcached**: MemcachedCache (pylibmc or memcache required)
* **gaememcached**: same as memcached (for backwards compatibility)
Expand Down Expand Up @@ -389,6 +390,8 @@ The following configuration values exist for Flask-Caching:
RedisSentinelCache.
``CACHE_REDIS_SENTINEL_MASTER`` The name of the master server in a sentinel configuration. Used
only for RedisSentinelCache.
``CACHE_REDIS_CLUSTER`` A string of comma-separated Redis cluster node addresses.
e.g. host1:port1,host2:port2,host3:port3 . Used only for RedisClusterCache.
``CACHE_DIR`` Directory to store cache. Used only for
FileSystemCache.
``CACHE_REDIS_URL`` URL to connect to Redis server.
Expand Down Expand Up @@ -472,6 +475,17 @@ Set ``CACHE_TYPE`` to ``redissentinel`` to use this type.

Entries in CACHE_OPTIONS are passed to the redis client as ``**kwargs``

RedisClusterCache
``````````````````

Set ``CACHE_TYPE`` to ``rediscluster`` to use this type.

- CACHE_KEY_PREFIX
- CACHE_REDIS_CLUSTER
- CACHE_REDIS_PASSWORD

Entries in CACHE_OPTIONS are passed to the redis client as ``**kwargs``

MemcachedCache
``````````````

Expand Down
12 changes: 11 additions & 1 deletion flask_caching/backends/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from flask_caching.backends.nullcache import NullCache

# TODO: Rename to "redis" when python2 support is removed
from flask_caching.backends.rediscache import RedisCache, RedisSentinelCache
from flask_caching.backends.rediscache import RedisCache, RedisSentinelCache, RedisClusterCache
from flask_caching.backends.simplecache import SimpleCache

try:
Expand All @@ -35,6 +35,7 @@
"filesystem",
"redis",
"redissentinel",
"rediscluster",
"uwsgi",
"memcached",
"gaememcached",
Expand Down Expand Up @@ -116,6 +117,15 @@ def redissentinel(app, config, args, kwargs):
return RedisSentinelCache(*args, **kwargs)


def rediscluster(app, config, args, kwargs):
kwargs.update(
dict(cluster=config.get("CACHE_REDIS_CLUSTER", ""),
password=config.get("CACHE_REDIS_PASSWORD", ""),
default_timeout=config.get("CACHE_DEFAULT_TIMEOUT", 300),
key_prefix=config.get("CACHE_KEY_PREFIX", "")))
return RedisClusterCache(*args, **kwargs)


def uwsgi(app, config, args, kwargs):
if not has_UWSGICache:
raise NotImplementedError(
Expand Down
58 changes: 58 additions & 0 deletions flask_caching/backends/rediscache.py
Original file line number Diff line number Diff line change
Expand Up @@ -273,3 +273,61 @@ def __init__(
self._read_clients = sentinel.slave_for(master)

self.key_prefix = key_prefix or ""

class RedisClusterCache(RedisCache):
"""Uses the Redis key-value store as a cache backend.

The first argument can be either a string denoting address of the Redis
server or an object resembling an instance of a rediscluster.RedisCluster class.

Note: Python Redis API already takes care of encoding unicode strings on
the fly.


:param cluster: The redis cluster nodes address separated by comma.
e.g. host1:port1,host2:port2,host3:port3 .
:param password: password authentication for the Redis server.
:param default_timeout: the default timeout that is used if no timeout is
specified on :meth:`~BaseCache.set`. A timeout of
0 indicates that the cache never expires.
:param key_prefix: A prefix that should be added to all keys.

Any additional keyword arguments will be passed to
``rediscluster.RedisCluster``.
"""
def __init__(self,
cluster="",
password="",
default_timeout=300,
key_prefix="",
**kwargs):
super().__init__(default_timeout=default_timeout)

if kwargs.get("decode_responses", None):
raise ValueError("decode_responses is not supported by "
"RedisCache.")

try:
from rediscluster import RedisCluster
except ImportError:
raise RuntimeError("no rediscluster module found")

try:
nodes = [(node.split(':')) for node in cluster.split(',')]
startup_nodes = [{
'host': node[0].strip(),
'port': node[1].strip()
} for node in nodes]
except IndexError:
raise ValueError("Please give the correct cluster argument "
"e.g. host1:port1,host2:port2,host3:port3")
# Skips the check of cluster-require-full-coverage config,
# useful for clusters without the CONFIG command (like aws)
skip_full_coverage_check = kwargs.pop('skip_full_coverage_check', True)

cluster = RedisCluster(startup_nodes=startup_nodes,
password=password,
skip_full_coverage_check=skip_full_coverage_check,
**kwargs)
self._write_client = self._read_clients = cluster
self.key_prefix = key_prefix