Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Server-assisted Client-side Caching #3757

Merged
merged 71 commits into from
Sep 27, 2024
Merged

Support Server-assisted Client-side Caching #3757

merged 71 commits into from
Sep 27, 2024

Conversation

sazzad16
Copy link
Collaborator

@sazzad16 sazzad16 commented Mar 6, 2024

Resolves #2104

* Code re-use?

* Stop forcing to read push notifications before checking cache and remove BCAST

* Rename variable

* Remove ensureFillSafe()

* Refactor peeking and reading push notifications

* Cleanup comments
Now we have to mock Protocol#read(RedisInputStream, ClientSideCache) instead of Protocol#read(RedisInputStream).
* Support client side caching from UnifiedJedis

* Support client side caching as a separate parameter

* format imports

* Support CSC in sentinel mode

* undo change
* Support TTL in client side caching (using Caffeine library)

* Also Guava cache

* format pom.xml

* Client-side caching by command arguments

TODO: Compute hash code.

* send keys

* todo comment for clean-up

* rename method to invalidate

* Client-side caching by hashing command arguments

* Hash command arguments for CaffeineCSC using OpenHFT hashing

* Clean-up keyHashes map

* added javadoc

* rename method

* remove lock

* descriptive name

* descriptive names and fix

* common default values in base class
* Support Client-side caching through URI/URL

* check idx of '=' sign

* nicer exception

* edit/fix condition

* rename param

* Throw IllegalArgumentException at all such cases
* Create csc package

* Create csc.util package

* Create a config interface for client-side caching

* Default isCacheable

* Config to WhiteList/BlackList commands and String keys

* Create csc test package(s)

* Test white-list/black-list commands and keys

* Merge fix

* Remove csc.util package

* Fix javadoc links

* Added ClientSideCacheable interface and removed ClientSideCacheConfig interface

* Format imports

* Re-create csc.util package

* Rename to allow/deny instead of white/black
@sazzad16 sazzad16 requested review from chayim and uglide March 6, 2024 08:59
@codecov-commenter
Copy link

codecov-commenter commented Mar 6, 2024

Codecov Report

Attention: Patch coverage is 81.69717% with 110 lines in your changes are missing coverage. Please review.

Project coverage is 81.84%. Comparing base (2d42338) to head (3bd45a4).

Files Patch % Lines
.../java/redis/clients/jedis/util/JedisURIHelper.java 8.00% 42 Missing and 4 partials ⚠️
.../redis/clients/jedis/csc/GuavaClientSideCache.java 73.33% 11 Missing and 1 partial ⚠️
...dis/clients/jedis/csc/CaffeineClientSideCache.java 71.42% 10 Missing ⚠️
.../java/redis/clients/jedis/csc/ClientSideCache.java 82.60% 3 Missing and 5 partials ⚠️
...rc/main/java/redis/clients/jedis/JedisCluster.java 72.72% 6 Missing ⚠️
...jedis/csc/util/AllowAndDenyListWithStringKeys.java 61.53% 0 Missing and 5 partials ⚠️
src/main/java/redis/clients/jedis/Connection.java 77.77% 2 Missing and 2 partials ⚠️
...nts/jedis/providers/ClusterConnectionProvider.java 66.66% 4 Missing ⚠️
...main/java/redis/clients/jedis/JedisSentineled.java 50.00% 3 Missing ⚠️
...rc/main/java/redis/clients/jedis/UnifiedJedis.java 98.77% 2 Missing and 1 partial ⚠️
... and 6 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3757      +/-   ##
============================================
- Coverage     82.13%   81.84%   -0.29%     
- Complexity     6016     6112      +96     
============================================
  Files           301      310       +9     
  Lines         15273    15616     +343     
  Branches       1190     1229      +39     
============================================
+ Hits          12544    12781     +237     
- Misses         2162     2249      +87     
- Partials        567      586      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sazzad16 sazzad16 closed this Mar 6, 2024
@sazzad16 sazzad16 reopened this Mar 6, 2024
@sazzad16 sazzad16 added this to the 5.2.0 milestone Mar 6, 2024
sazzad16 and others added 9 commits July 15, 2024 14:50
* Separate CacheConnection

* Introduce CacheKey and CacheEntry

* Little tweak maximumSize test in CaffeineClientSideCacheTest

* Remove resetting timeout; we'll PING instead
* adding a DataProvider to access connection from cache

* resolve keys from commandarguments

* clean up in unifiiedjedis and add csc test with ssl

* - fix readtimeout exception with sockets for consuming invalidations pending in buffer
- apply a default list of cacheable commands to DefaultClientSideCacheable
- fix failing unit tests with cacheable / non-cacheable keys
- remove formatting changes

* - add serialization for cache instances
- add unit test with UnifiedJedis
- add benchmark for CSC execution
- clean unused imports

* - added 'Cache' interface and 'DefaultCache' implementation in regard to design doc
- added 'EvictionPolicy' interface and LRU implementation
- move cache object validation and cache control stuf from 'ClientSideCache' into 'CacheConnection'
- make guava and caffeine caches experimental

* - added SSLSocketWrapper and plug it to use 'available'
- handle exceptions properly
- fix some issues with unit tests

* implementing thread safety

* - fix eviction issue and add related test
- fix consuming invalidation messages on a response read
- introduce cachestats
- fix potential issue with cacheKeysRelatedtoRedisKey cleanup
- tests for sequential access, concurrent acces and maxsize

* - renmae abstract cache class
- add test case for returning new instance of cache object

* - change order of execution in sequential acces test

* -  flush the cache on any disconnect
- replace LRU policy references with EvictionPolicy interface
-  add some constructor overloads to enable custom eviction policies on cache

* fix testcache

* fix javadoc issue

* - fix multithreaded eviction policy issue
- update guava and caffeine implementations according to abstract cache
sazzad16 and others added 10 commits August 15, 2024 14:12
* initial changes

* cover tests for JedisPooled and functionality

* fix javadoc

* cover new tests for JedisCluster and JedisSentineled

* Fix CSC allow-and-deny-list and rename Cacheable interface

* Tag CommandArguments#getKeys() as Internal

* cover lruEvictionTest

* Address code reviews and more updates

* fix format and more minor changes

* format Connection

* modify WeakReference usage
* Use ExecutorService.shutdownNow()

* More ExecutorService.shutdownNow() and other changes
* caching null results

* add more assertion
* add cacheconfig

* remove empty file

* -modify constructors with cache as public
- trim guava caffeine

* remove cachetype

* - add getCache to UnifiedJedis
- add builder method to CacheConfig

* add evictionpolicy to cacheconfig

* - unifiedjedis constructor with cacheconfig
-  wrap IOException on protocol read error

* fix merge issue

---------

Co-authored-by: M Sazzadul Hoque <7600764+sazzad16@users.noreply.github.com>
Polish #3919 - address some pending change requests

- Swap contructor placements
- Fix grammar in exception message
* adding cacheclass to cacheconfig

* - add cachefactory test

* - revert connection ctors to public
- udpate some tests with UnifiedJedis.getCache
- add ping to flaky tests

* remove unnecessary anonymous types

* change ctor access modifiers

* fix test name

* make cachefactory methods static

* removing pings  due to still flaky with inv messages

* - drop CustomCache in tests and use TestCache
- check null cacheable issue with defaultcache
- support both ctors in custom cache classes regarding to value of cacheconfig.cacheable

* remove unncessary maxsize

* - remove inline anonymious
atakavci and others added 3 commits September 18, 2024 11:25
* checking server version for CSC

* fix format change

* fix noauth hello exception in integration tests

* fix version check

* remove redundant check

* remove unused imports

* 'toString' for Version

* rename to RedisVersion

* moving RedisVersion package
@uglide uglide marked this pull request as ready for review September 27, 2024 11:23
@uglide uglide merged commit 5442642 into master Sep 27, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for localcache (Client-side caching)
5 participants