Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRIVERS-2533: add RunCursorCommand API #1412

Merged
merged 82 commits into from
May 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
2ba230e
DRIVERS-2533: add RunCursorCommand API
nbbeeken Apr 24, 2023
8d46217
feat: add cleanup req
nbbeeken Apr 25, 2023
b688572
feat: add runCursorCommand UTF operation
nbbeeken Apr 25, 2023
eff1d85
chore: changelog
nbbeeken Apr 25, 2023
be59aa8
feat: add UTF operations and first test
nbbeeken Apr 25, 2023
155677b
feat: lb mode, and more csot
nbbeeken Apr 26, 2023
c437e53
docs: add tailable section
nbbeeken Apr 26, 2023
03779a1
feat: add clusterTime gossip
nbbeeken Apr 27, 2023
85e032d
apply suggestions
nbbeeken Apr 28, 2023
508990d
Add links to specs, monospacing, high level steps cleanup
nbbeeken Apr 28, 2023
3420d3e
Add improvements to Find, getMore and killCursors commands
nbbeeken Apr 28, 2023
3dfcd77
comments
nbbeeken May 1, 2023
679e14a
add invalid tests
nbbeeken May 1, 2023
0aab062
Merge branch 'master' into DRIVERS-2533-runCursorCommand-api
nbbeeken May 1, 2023
6745d71
rename invalid test file
nbbeeken May 1, 2023
1db918b
add clusterTime assertion
nbbeeken May 1, 2023
4890269
add valid pass
nbbeeken May 1, 2023
b39df09
add more test cases
nbbeeken May 1, 2023
31eebfc
fix server version
nbbeeken May 2, 2023
eb7e34a
add tailable cursor test
nbbeeken May 3, 2023
90cb9f0
fix schema issues
nbbeeken May 3, 2023
f159ebd
gen json
nbbeeken May 3, 2023
dc73ff8
fix: title
nbbeeken May 4, 2023
f623935
fix: title
nbbeeken May 4, 2023
576d914
fix: title
nbbeeken May 4, 2023
fa95fc2
gen: json
nbbeeken May 4, 2023
6f0ca16
test for errors from non-cursor responses
nbbeeken May 4, 2023
b7962b7
test: breakout clusterTime to its own test
nbbeeken May 4, 2023
b9e6fdb
update optionality language
nbbeeken May 4, 2023
cab4837
update unified test format test titles
nbbeeken May 4, 2023
a724120
Merge branch 'master' into DRIVERS-2533-runCursorCommand-api
nbbeeken May 9, 2023
052f399
test: add misconfiguration tests
nbbeeken May 9, 2023
eff3eef
Merge branch 'master' into DRIVERS-2533-runCursorCommand-api
nbbeeken May 10, 2023
386ef6b
find_getmore_killcursors comments
nbbeeken May 10, 2023
3393849
run-command comments
nbbeeken May 10, 2023
aadfd9d
remove extra assertions for runCommand.yml
nbbeeken May 10, 2023
661060a
reduce lsid strictness
nbbeeken May 10, 2023
2d206a5
rm empty filter
nbbeeken May 10, 2023
34de634
fix getMore value bitwidth
nbbeeken May 10, 2023
0994612
add interface for getMore controls
nbbeeken May 10, 2023
a0ca7bd
ignore result
nbbeeken May 10, 2023
d522f27
ignore result - correctly
nbbeeken May 10, 2023
850f26d
add comment to find
nbbeeken May 10, 2023
8f13d54
move CSOT, remove invalid test
nbbeeken May 10, 2023
66c7199
getmore and lsid, name fix
nbbeeken May 10, 2023
3117133
killCursors array type
nbbeeken May 10, 2023
4a7958e
fix getMore settings, pos int mention
nbbeeken May 10, 2023
330aead
entity and cursor operations
nbbeeken May 10, 2023
f30d110
must->should
nbbeeken May 10, 2023
7d172b1
changelogs
nbbeeken May 10, 2023
80fd8f7
fix rst
nbbeeken May 10, 2023
55e5cd8
Merge branch 'master' into DRIVERS-2533-runCursorCommand-api
nbbeeken May 11, 2023
2fe6fa8
add option to configure getMore fields and mention of testing
nbbeeken May 11, 2023
86a5ee8
remove check for result
nbbeeken May 11, 2023
699328f
Add a comment about ignoring extra events
nbbeeken May 11, 2023
7391ea2
non-negative
nbbeeken May 11, 2023
933c237
update numeric types
nbbeeken May 11, 2023
ce7631d
suggestions
nbbeeken May 11, 2023
259583c
Merge branch 'DRIVERS-2533-runCursorCommand-api' of github.com:mongod…
nbbeeken May 11, 2023
f0497fe
rm R/W concern sentence, update RP
nbbeeken May 11, 2023
0876609
Merge branch 'master' into DRIVERS-2533-runCursorCommand-api
jmikola May 12, 2023
09195d5
test: add more CSOT tests
nbbeeken May 16, 2023
b113dde
Merge branch 'DRIVERS-2533-runCursorCommand-api' of github.com:mongod…
nbbeeken May 16, 2023
e055387
test: add tailable awaitdata case
nbbeeken May 16, 2023
e612efe
comments
nbbeeken May 17, 2023
dc500f5
fix operation name
nbbeeken May 17, 2023
7dd20be
fix format
nbbeeken May 17, 2023
73703b3
fix: db
nbbeeken May 18, 2023
4e50497
docs: cursorType
nbbeeken May 18, 2023
5b4651d
fix: clusterTime test
nbbeeken May 18, 2023
8a87576
comments
nbbeeken May 18, 2023
90f12a6
fix: add documents to insert
nbbeeken May 19, 2023
0adbd22
fmt
nbbeeken May 19, 2023
c50a20d
fix LB test
nbbeeken May 19, 2023
d4e60c0
descriptive comments
nbbeeken May 19, 2023
299705d
LB fix session
nbbeeken May 19, 2023
2ea4e3d
remove inequality tests
nbbeeken May 22, 2023
d3f73c7
add serverless forbid to capped collection test
nbbeeken May 22, 2023
f498f9a
forbid serverless for csot capped collection tests
nbbeeken May 22, 2023
5c8d733
flaky timeouts on serverless
nbbeeken May 22, 2023
942549c
update timeouts
nbbeeken May 23, 2023
0a42153
fix: comments and timeout value
nbbeeken May 23, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
583 changes: 583 additions & 0 deletions source/client-side-operations-timeout/tests/runCursorCommand.json

Large diffs are not rendered by default.

304 changes: 304 additions & 0 deletions source/client-side-operations-timeout/tests/runCursorCommand.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,304 @@
description: runCursorCommand

schemaVersion: '1.9'

runOnRequirements:
- minServerVersion: "4.4"

createEntities:
- client:
id: &failPointClient failPointClient
useMultipleMongoses: false
- client:
id: &commandClient commandClient
useMultipleMongoses: false
observeEvents: [commandStartedEvent, commandSucceededEvent]
- client:
id: &client client
useMultipleMongoses: false
observeEvents: [commandStartedEvent]
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
ignoreCommandMonitoringEvents: [killCursors]
- database: # For tests that need success event assertions
id: &commandDb commandDb
client: *commandClient
databaseName: *commandDb
- database:
id: &db db
client: *client
databaseName: *db
- collection:
id: &collection collection
database: *db
collectionName: *collection

initialData:
- collectionName: *collection
databaseName: *db
documents: &documents
- { _id: 1, x: 11 }
- { _id: 2, x: 22 }
- { _id: 3, x: 33 }
- { _id: 4, x: 44 }
- { _id: 5, x: 55 }

tests:
- description: errors if timeoutMode is set without timeoutMS
operations:
- name: runCursorCommand
object: *db
arguments:
commandName: find
command: { find: *collection }
timeoutMode: cursorLifetime
expectError:
isClientError: true

- description: error if timeoutMode is cursorLifetime and cursorType is tailableAwait
operations:
- name: runCursorCommand
object: *db
arguments:
commandName: find
command: { find: *collection }
timeoutMode: cursorLifetime
cursorType: tailableAwait
expectError:
isClientError: true

# If timeoutMode is unset, it should default to CURSOR_LIFETIME and the time remaining after the find succeeds should be applied to the getMore
- description: Non-tailable cursor lifetime remaining timeoutMS applied to getMore if timeoutMode is unset
runOnRequirements:
- serverless: forbid
operations:
# Block find/getMore for 15ms.
- name: failPoint
object: testRunner
arguments:
client: *failPointClient
failPoint:
configureFailPoint: failCommand
mode: { times: 2 }
data:
failCommands: [find, getMore]
blockConnection: true
blockTimeMS: 60
# Run a find with timeoutMS less than double our failPoint blockTimeMS and
# batchSize less than the total document count will cause a find and a getMore to be sent.
# Both will block for 60ms so together they will go over the timeout.
- name: runCursorCommand
object: *db
arguments:
commandName: find
timeoutMS: 100
command: { find: *collection, batchSize: 2 }
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
expectError:
isTimeoutError: true
expectEvents:
- client: *client
events:
- commandStartedEvent:
commandName: find
command:
find: *collection
maxTimeMS: { $$type: [int, long] }
- commandStartedEvent:
commandName: getMore
command:
getMore: { $$type: [int, long] }
collection: *collection
maxTimeMS: { $$exists: true }

# If timeoutMode=ITERATION, timeoutMS applies separately to the initial find and the getMore on the cursor. Neither
# command should have a maxTimeMS field. This is a failure test. The "find" inherits timeoutMS=100 and "getMore"
# commands are blocked for 60ms, causing iteration to fail with a timeout error.
- description: Non=tailable cursor iteration timeoutMS is refreshed for getMore if timeoutMode is iteration - failure
runOnRequirements:
- serverless: forbid
operations:
- name: failPoint
object: testRunner
arguments:
client: *failPointClient
failPoint:
configureFailPoint: failCommand
mode: { times: 1 }
data:
failCommands: ["getMore"]
blockConnection: true
blockTimeMS: 60
- name: runCursorCommand
object: *db
arguments:
commandName: find
command: { find: *collection, batchSize: 2 }
timeoutMode: iteration
timeoutMS: 100
batchSize: 2
expectError:
isTimeoutError: true
expectEvents:
- client: *client
events:
- commandStartedEvent:
commandName: find
databaseName: *db
command:
find: *collection
maxTimeMS: { $$exists: false }
- commandStartedEvent:
commandName: getMore
databaseName: *db
command:
getMore: { $$type: ["int", "long"] }
collection: *collection
maxTimeMS: { $$exists: false }

# The timeoutMS option should apply separately to the initial "find" and each getMore. This is a failure test. The
# find inherits timeoutMS=100 from the collection and the getMore command blocks for 60ms, causing iteration to fail
# with a timeout error.
- description: Tailable cursor iteration timeoutMS is refreshed for getMore - failure
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
runOnRequirements:
- serverless: forbid
operations:
- name: failPoint
object: testRunner
arguments:
client: *failPointClient
failPoint:
configureFailPoint: failCommand
mode: { times: 1 }
data:
failCommands: ["getMore"]
blockConnection: true
blockTimeMS: 60
- name: dropCollection
object: *db
arguments:
collection: &cappedCollection cappedCollection
- name: createCollection
object: *db
arguments:
collection: *cappedCollection
capped: true
size: 4096
max: 3
saveResultAsEntity: *cappedCollection
- name: insertMany
object: *cappedCollection
arguments:
documents:
- { _id: 1, x: 11 }
- { _id: 2, x: 22 }
- name: createCommandCursor
object: *db
arguments:
commandName: find
command: { find: *cappedCollection, batchSize: 1, tailable: true }
timeoutMode: iteration
timeoutMS: 100
batchSize: 1
cursorType: tailable
saveResultAsEntity: &tailableCursor tailableCursor
# Iterate the cursor twice: the first iteration will return the document from the batch in the find and the
# second will do a getMore.
- name: iterateUntilDocumentOrError
object: *tailableCursor
- name: iterateUntilDocumentOrError
object: *tailableCursor
expectError:
isTimeoutError: true
expectEvents:
- client: *client
events:
- commandStartedEvent:
commandName: drop
- commandStartedEvent:
commandName: create
- commandStartedEvent:
commandName: insert
- commandStartedEvent:
commandName: find
databaseName: *db
command:
find: *cappedCollection
tailable: true
awaitData: { $$exists: false }
maxTimeMS: { $$exists: false }
- commandStartedEvent:
commandName: getMore
databaseName: *db
command:
getMore: { $$type: ["int", "long"] }
collection: *cappedCollection
maxTimeMS: { $$exists: false }

# The timeoutMS value should be refreshed for getMore's. This is a failure test. The find inherits timeoutMS=10 from
# the collection and the getMore blocks for 15ms, causing iteration to fail with a timeout error.
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
- description: Tailable cursor awaitData iteration timeoutMS is refreshed for getMore - failure
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
runOnRequirements:
- serverless: forbid
operations:
- name: failPoint
object: testRunner
arguments:
client: *failPointClient
failPoint:
configureFailPoint: failCommand
mode: { times: 1 }
data:
failCommands: ["getMore"]
blockConnection: true
blockTimeMS: 60
- name: dropCollection
object: *db
arguments:
collection: &cappedCollection cappedCollection
- name: createCollection
object: *db
arguments:
collection: *cappedCollection
capped: true
size: 4096
max: 3
saveResultAsEntity: *cappedCollection
- name: insertMany
prestonvasquez marked this conversation as resolved.
Show resolved Hide resolved
object: *cappedCollection
arguments:
documents: [ { foo: bar }, { fizz: buzz } ]
- name: createCommandCursor
object: *db
arguments:
command: { find: *cappedCollection, tailable: true, awaitData: true }
cursorType: tailableAwait
batchSize: 1
saveResultAsEntity: &tailableCursor tailableCursor
# Iterate twice to force a getMore.
- name: iterateUntilDocumentOrError
object: *tailableCursor
- name: iterateUntilDocumentOrError
object: *tailableCursor
expectError:
isTimeoutError: true
expectEvents:
- client: *client
events:
- commandStartedEvent:
commandName: drop
- commandStartedEvent:
commandName: create
- commandStartedEvent:
commandName: insert
- commandStartedEvent:
commandName: find
databaseName: *db
command:
find: *cappedCollection
tailable: true
awaitData: true
maxTimeMS: { $$exists: true }
- commandStartedEvent:
commandName: getMore
databaseName: *db
command:
getMore: { $$type: ["int", "long"] }
collection: *cappedCollection
Loading