Make connection pool configurable #158

louiseschmidtgen · 2024-08-14T08:08:32Z

Make Connection Pool configurable

We use a connection pool to talk with the database. This connection pool can be configured.
Existing connections are not actively closed by us, we lazily close them via the connection pool configuration.
Through this PR we can set the connection pool configurations via flags.

This PR attempts to mitigate an issue where the leader node is killed and connections are "hanging" on the old connection of the killed node.

Configuration

Sql allows us to configure these parameters:

Current Configuration

We currently use:

MaxIdleConns: 5
MaxOpenConns: 5
ConnMaxLifetime: 60s
ConnMaxIdleTime: undefined

Optimal default values

From a discussion with the dqlite team:
MaxConnections

lxd team allows only one connection
benchmarking shows that the best write throughput from dqlite shows you should use at least 2 connections, 5 seems reasonable
Not certain whether dqlite can take advantage of more than 5 simultaneous connections for reads, if so bumping this number for very large clusters would be advisable.
Max ConnLifetime
shorter connection lifetimes mean a quicker response to close connections to dead leader nodes, however this can be wasteful on a healthy cluster.

github-actions · 2024-08-14T08:18:45Z

Benchmark Result

goos: linux
goarch: amd64
pkg: github.com/canonical/k8s-dqlite/test
cpu: AMD EPYC 7763 64-Core Processor                
                                        │    Before    │                After                │
                                        │    sec/op    │    sec/op      vs base              │
Compaction/sqlite-4                       40.17µ ± 11%    39.42µ ± 11%       ~ (p=0.535 n=7)
Compaction/dqlite-4                       40.44µ ±  7%    41.45µ ±  8%       ~ (p=0.209 n=7)
Create/sqlite-4                           1.103m ±  2%    1.211m ±  4%  +9.88% (p=0.001 n=7)
Create/dqlite-4                           998.3µ ±  3%   1065.9µ ±  2%  +6.77% (p=0.001 n=7)
Delete/sqlite-4                           1.283m ±  3%    1.387m ±  5%  +8.07% (p=0.001 n=7)
Delete/dqlite-4                           1.289m ±  3%    1.298m ±  6%       ~ (p=0.902 n=7)
Get/sqlite-4                              264.4µ ±  1%    273.0µ ±  1%  +3.23% (p=0.001 n=7)
Get/dqlite-4                              353.8µ ±  1%    360.6µ ±  1%  +1.91% (p=0.001 n=7)
Lease/sqlite-4                            97.69µ ±  3%   101.00µ ±  1%  +3.38% (p=0.001 n=7)
Lease/dqlite-4                            100.7µ ±  3%    101.2µ ±  2%       ~ (p=0.073 n=7)
List/sqlite-tiny/all-4                    4.437µ ±  3%    4.475µ ±  1%       ~ (p=0.318 n=7)
List/sqlite-tiny/pagination-4             9.094µ ±  4%    9.004µ ±  4%       ~ (p=0.805 n=7)
List/sqlite-fits-in-page/all-4            7.282µ ±  7%    7.501µ ±  3%       ~ (p=0.318 n=7)
List/sqlite-fits-in-page/pagination-4     22.42µ ±  4%    23.05µ ±  2%  +2.83% (p=0.011 n=7)
List/sqlite-overflows-page/all-4          17.64µ ±  4%    17.73µ ±  5%       ~ (p=0.620 n=7)
List/sqlite-overflows-page/pagination-4   48.12µ ±  2%    48.15µ ±  3%       ~ (p=0.710 n=7)
List/dqlite-tiny/all-4                    2.770µ ±  3%    2.685µ ±  5%  -3.07% (p=0.038 n=7)
List/dqlite-tiny/pagination-4             5.911µ ±  2%    5.934µ ±  1%  +0.39% (p=0.034 n=7)
List/dqlite-fits-in-page/all-4            6.052µ ±  5%    6.140µ ±  2%       ~ (p=0.535 n=7)
List/dqlite-fits-in-page/pagination-4     16.88µ ±  2%    16.90µ ±  2%       ~ (p=0.710 n=7)
List/dqlite-overflows-page/all-4          18.82µ ±  2%    19.25µ ±  2%  +2.28% (p=0.007 n=7)
List/dqlite-overflows-page/pagination-4   43.01µ ±  2%    43.39µ ±  3%       ~ (p=0.456 n=7)
Update/sqlite-4                           1.283m ±  6%    1.285m ±  4%       ~ (p=0.620 n=7)
Update/dqlite-4                           1.237m ±  4%    1.307m ±  1%  +5.68% (p=0.001 n=7)
geomean                                   65.03µ          66.34µ        +2.01%

                                        │        Before        │                    After                     │
                                        │ page-cache-misses/op │ page-cache-misses/op  vs base                │
Compaction/sqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Compaction/dqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/sqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Lease/sqlite-4                                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Lease/dqlite-4                                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
geomean                                                      ²                         +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │        Before        │                    After                     │
                                        │ page-cache-spills/op │ page-cache-spills/op  vs base                │
Compaction/sqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Compaction/dqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/sqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Lease/sqlite-4                                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Lease/dqlite-4                                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite-4                                   0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
geomean                                                      ²                         +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │    Before     │                After                │
                                        │ page-reads/op │ page-reads/op  vs base              │
Compaction/sqlite-4                         12.20 ±  1%    12.21 ±   1%       ~ (p=0.830 n=7)
Compaction/dqlite-4                         12.20 ±  1%    12.20 ±   1%       ~ (p=0.371 n=7)
Create/sqlite-4                             14.24 ±  3%    14.11 ±   2%       ~ (p=0.150 n=7)
Create/dqlite-4                             13.14 ±  1%    13.16 ±   1%       ~ (p=0.640 n=7)
Delete/sqlite-4                             25.17 ±  2%    25.02 ±   2%       ~ (p=0.100 n=7)
Delete/dqlite-4                             23.69 ±  1%    23.81 ±   1%       ~ (p=0.334 n=7)
Get/sqlite-4                                14.96 ±  0%    14.96 ±   0%       ~ (p=0.408 n=7)
Get/dqlite-4                                11.96 ±  0%    11.95 ±   0%       ~ (p=0.292 n=7)
Lease/sqlite-4                             654.8µ ± 83%   800.0µ ± 100%       ~ (p=0.475 n=7)
Lease/dqlite-4                             800.0µ ± 50%   800.0µ ±  50%       ~ (p=0.126 n=7)
List/sqlite-tiny/all-4                     192.3m ±  0%   192.3m ±   0%       ~ (p=0.389 n=7)
List/sqlite-tiny/pagination-4               1.132 ±  3%    1.095 ±   3%       ~ (p=0.388 n=7)
List/sqlite-fits-in-page/all-4             165.1m ±  0%   165.1m ±   0%       ~ (p=0.991 n=7)
List/sqlite-fits-in-page/pagination-4      965.6m ±  0%   965.6m ±   0%       ~ (p=0.543 n=7)
List/sqlite-overflows-page/all-4           278.7m ± 22%   293.4m ±  29%       ~ (p=0.902 n=7)
List/sqlite-overflows-page/pagination-4     2.081 ±  0%    2.080 ±   2%       ~ (p=0.057 n=7)
List/dqlite-tiny/all-4                     191.1m ±  0%   190.4m ±   0%  -0.37% (p=0.006 n=7)
List/dqlite-tiny/pagination-4               1.125 ±  3%    1.126 ±   0%  +0.09% (p=0.049 n=7)
List/dqlite-fits-in-page/all-4             163.1m ± 59%   163.2m ±  59%       ~ (p=0.685 n=7)
List/dqlite-fits-in-page/pagination-4      954.2m ±  0%   954.5m ±   3%       ~ (p=0.360 n=7)
List/dqlite-overflows-page/all-4           182.8m ±  0%   182.9m ±   0%       ~ (p=0.720 n=7)
List/dqlite-overflows-page/pagination-4     1.071 ±  0%    1.071 ±   3%       ~ (p=0.462 n=7)
Update/sqlite-4                             25.66 ±  1%    25.59 ±   2%       ~ (p=0.535 n=7)
Update/dqlite-4                             24.73 ±  1%    24.59 ±   1%       ~ (p=0.194 n=7)
geomean                                     1.225          1.235         +0.84%

                                        │     Before     │                 After                  │
                                        │ page-writes/op │ page-writes/op  vs base                │
Compaction/sqlite-4                        207.8m ± 0%        207.7m ± 0%       ~ (p=1.000 n=7)
Compaction/dqlite-4                        187.3m ± 1%        187.6m ± 1%       ~ (p=0.436 n=7)
Create/sqlite-4                             5.205 ± 1%         5.131 ± 1%  -1.42% (p=0.002 n=7)
Create/dqlite-4                             4.281 ± 1%         4.242 ± 1%  -0.91% (p=0.050 n=7)
Delete/sqlite-4                             5.317 ± 1%         5.339 ± 1%       ~ (p=0.247 n=7)
Delete/dqlite-4                             4.389 ± 1%         4.375 ± 1%       ~ (p=0.433 n=7)
Get/sqlite-4                                0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                                0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Lease/sqlite-4                              0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Lease/dqlite-4                              0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                      0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4               0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4              0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4       0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4            0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4     0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                      0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4               0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4              0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4       0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4            0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4     0.000 ± 0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite-4                             5.162 ± 0%         5.142 ± 0%  -0.39% (p=0.047 n=7)
Update/dqlite-4                             4.169 ± 0%         4.177 ± 0%  +0.19% (p=0.009 n=7)
geomean                                                ²                   -0.10%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │     Before      │                  After                  │
                                        │ sec-reading/op  │ sec-reading/op  vs base                 │
Compaction/sqlite-4                       124.4µ ±   9%      118.6µ ±  19%        ~ (p=0.104 n=7)
Compaction/dqlite-4                       28.90µ ±  54%      29.20µ ±  60%        ~ (p=0.902 n=7)
Create/sqlite-4                           37.60µ ± 524%      53.40µ ± 178%        ~ (p=0.902 n=7)
Create/dqlite-4                           10.10µ ±  40%      10.20µ ±  19%        ~ (p=1.000 n=7)
Delete/sqlite-4                           159.2µ ± 101%      218.1µ ±  73%        ~ (p=0.128 n=7)
Delete/dqlite-4                           48.60µ ±  24%      50.50µ ±  21%        ~ (p=0.710 n=7)
Get/sqlite-4                              88.10µ ±   4%      88.80µ ±   7%        ~ (p=0.710 n=7)
Get/dqlite-4                              31.90µ ±  12%      33.50µ ±  19%        ~ (p=0.128 n=7)
Lease/sqlite-4                            200.0n ±    ?      300.0n ±    ?        ~ (p=0.487 n=7)
Lease/dqlite-4                             0.000 ±    ?       0.000 ±   0%        ~ (p=1.000 n=7)
List/sqlite-tiny/all-4                    14.20µ ±   2%      14.40µ ±   3%   +1.41% (p=0.045 n=7)
List/sqlite-tiny/pagination-4             19.90µ ±   5%      19.90µ ±   6%        ~ (p=0.974 n=7)
List/sqlite-fits-in-page/all-4            19.00µ ±  15%      18.20µ ±  10%        ~ (p=0.689 n=7)
List/sqlite-fits-in-page/pagination-4     62.80µ ±   4%      63.20µ ±   3%        ~ (p=0.782 n=7)
List/sqlite-overflows-page/all-4          63.50µ ±  22%      70.40µ ±  15%        ~ (p=0.097 n=7)
List/sqlite-overflows-page/pagination-4   178.9µ ±   2%      179.7µ ±   3%        ~ (p=0.710 n=7)
List/dqlite-tiny/all-4                    3.900µ ±   3%      3.500µ ±  11%  -10.26% (p=0.012 n=7)
List/dqlite-tiny/pagination-4             8.500µ ±   9%      8.700µ ±  11%        ~ (p=0.207 n=7)
List/dqlite-fits-in-page/all-4            10.10µ ±  16%      10.20µ ±  15%        ~ (p=0.687 n=7)
List/dqlite-fits-in-page/pagination-4     25.20µ ±   7%      26.40µ ±  13%        ~ (p=0.120 n=7)
List/dqlite-overflows-page/all-4          32.40µ ±  13%      32.80µ ±   2%   +1.23% (p=0.040 n=7)
List/dqlite-overflows-page/pagination-4   73.50µ ±  10%      72.60µ ±   7%        ~ (p=0.456 n=7)
Update/sqlite-4                           168.4µ ±  42%      176.8µ ±  66%        ~ (p=0.259 n=7)
Update/dqlite-4                           112.6µ ±  10%      110.4µ ±  14%        ~ (p=0.097 n=7)
geomean                                                 ¹                    +5.23%               ¹
¹ summaries must be >0 to compute geomean

                                        │     Before     │                  After                  │
                                        │ sec-writing/op │ sec-writing/op  vs base                 │
Compaction/sqlite-4                       10.30µ ± 13%       10.10µ ±  8%        ~ (p=0.827 n=7)
Compaction/dqlite-4                       7.000µ ±  7%       7.200µ ±  8%        ~ (p=0.264 n=7)
Create/sqlite-4                           841.2µ ±  4%       947.6µ ±  4%  +12.65% (p=0.001 n=7)
Create/dqlite-4                           31.10µ ± 26%       34.50µ ± 26%        ~ (p=0.300 n=7)
Delete/sqlite-4                           899.0µ ±  6%      1006.0µ ± 10%  +11.90% (p=0.002 n=7)
Delete/dqlite-4                           21.50µ ± 39%       22.30µ ± 35%        ~ (p=0.932 n=7)
Get/sqlite-4                               0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
Get/dqlite-4                               0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
Lease/sqlite-4                             0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
Lease/dqlite-4                             0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                     0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4              0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4             0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4      0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4           0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4    0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                     0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4              0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4             0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4      0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4           0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4    0.000 ±  0%        0.000 ±  0%        ~ (p=1.000 n=7) ¹
Update/sqlite-4                           844.7µ ±  9%       837.0µ ±  6%        ~ (p=1.000 n=7)
Update/dqlite-4                           20.80µ ± 28%       23.70µ ± 44%        ~ (p=0.244 n=7)
geomean                                                ²                    +2.11%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │    Before     │                After                │
                                        │     B/op      │     B/op       vs base              │
Compaction/sqlite-4                         982.0 ± 53%    1008.0 ± 68%       ~ (p=0.710 n=7)
Compaction/dqlite-4                       5.190Ki ± 75%   4.999Ki ± 79%       ~ (p=0.620 n=7)
Create/sqlite-4                           21.32Ki ±  3%   21.31Ki ±  0%       ~ (p=0.710 n=7)
Create/dqlite-4                           20.06Ki ±  0%   20.07Ki ±  0%       ~ (p=0.383 n=7)
Delete/sqlite-4                           32.64Ki ±  1%   32.52Ki ±  2%       ~ (p=0.318 n=7)
Delete/dqlite-4                           29.46Ki ±  0%   29.46Ki ±  0%       ~ (p=0.828 n=7)
Get/sqlite-4                              25.08Ki ±  0%   25.08Ki ±  0%       ~ (p=0.219 n=7)
Get/dqlite-4                              22.89Ki ±  0%   22.89Ki ±  0%       ~ (p=0.732 n=7)
Lease/sqlite-4                            12.33Ki ±  0%   12.33Ki ±  0%  +0.03% (p=0.015 n=7)
Lease/dqlite-4                            12.33Ki ±  0%   12.33Ki ±  0%       ~ (p=0.339 n=7)
List/sqlite-tiny/all-4                    1.261Ki ±  3%   1.261Ki ±  2%       ~ (p=0.710 n=7)
List/sqlite-tiny/pagination-4             1.771Ki ±  6%   1.789Ki ±  4%       ~ (p=1.000 n=7)
List/sqlite-fits-in-page/all-4            5.197Ki ±  4%   5.188Ki ±  3%       ~ (p=0.832 n=7)
List/sqlite-fits-in-page/pagination-4     10.25Ki ±  1%   10.24Ki ±  0%       ~ (p=0.274 n=7)
List/sqlite-overflows-page/all-4          26.35Ki ±  5%   26.74Ki ±  7%       ~ (p=0.805 n=7)
List/sqlite-overflows-page/pagination-4   46.14Ki ±  0%   46.12Ki ±  0%       ~ (p=0.535 n=7)
List/dqlite-tiny/all-4                      867.0 ±  1%     830.0 ±  7%  -4.27% (p=0.007 n=7)
List/dqlite-tiny/pagination-4               838.0 ±  1%     833.0 ±  2%       ~ (p=0.476 n=7)
List/dqlite-fits-in-page/all-4            4.289Ki ±  1%   4.292Ki ±  1%       ~ (p=0.927 n=7)
List/dqlite-fits-in-page/pagination-4     4.323Ki ±  2%   4.400Ki ±  6%       ~ (p=0.198 n=7)
List/dqlite-overflows-page/all-4          21.22Ki ±  1%   21.30Ki ±  1%       ~ (p=0.318 n=7)
List/dqlite-overflows-page/pagination-4   22.97Ki ±  1%   22.95Ki ±  6%       ~ (p=0.805 n=7)
Update/sqlite-4                           30.32Ki ±  1%   30.33Ki ±  1%       ~ (p=0.195 n=7)
Update/dqlite-4                           27.60Ki ±  0%   27.61Ki ±  0%  +0.04% (p=0.006 n=7)
geomean                                   9.201Ki         9.193Ki        -0.09%

                                        │   Before    │                After                │
                                        │  allocs/op  │  allocs/op   vs base                │
Compaction/sqlite-4                       38.00 ± 55%   39.00 ± 74%       ~ (p=0.738 n=7)
Compaction/dqlite-4                       125.0 ± 76%   121.0 ± 80%       ~ (p=0.645 n=7)
Create/sqlite-4                           455.0 ±  5%   453.0 ±  1%       ~ (p=0.977 n=7)
Create/dqlite-4                           372.0 ±  0%   372.0 ±  0%       ~ (p=1.000 n=7) ¹
Delete/sqlite-4                           728.0 ±  2%   724.0 ±  2%  -0.55% (p=0.047 n=7)
Delete/dqlite-4                           511.0 ±  0%   511.0 ±  0%       ~ (p=1.000 n=7)
Get/sqlite-4                              563.0 ±  0%   563.0 ±  0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                              407.0 ±  0%   407.0 ±  0%       ~ (p=1.000 n=7) ¹
Lease/sqlite-4                            224.0 ±  0%   224.0 ±  0%       ~ (p=1.000 n=7) ¹
Lease/dqlite-4                            224.0 ±  0%   224.0 ±  0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                    41.00 ±  5%   41.00 ±  2%       ~ (p=0.371 n=7)
List/sqlite-tiny/pagination-4             61.00 ±  7%   62.00 ±  5%       ~ (p=0.850 n=7)
List/sqlite-fits-in-page/all-4            48.00 ±  6%   48.00 ±  4%       ~ (p=0.956 n=7)
List/sqlite-fits-in-page/pagination-4     93.00 ±  0%   93.00 ±  0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4          56.00 ±  5%   57.00 ±  5%       ~ (p=0.704 n=7)
List/sqlite-overflows-page/pagination-4   93.00 ±  0%   93.00 ±  0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                    14.00 ±  0%   13.00 ±  8%  -7.14% (p=0.021 n=7)
List/dqlite-tiny/pagination-4             13.00 ±  0%   13.00 ±  0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4            18.00 ±  0%   18.00 ±  6%       ~ (p=1.000 n=7)
List/dqlite-fits-in-page/pagination-4     17.00 ±  6%   18.00 ±  6%       ~ (p=0.274 n=7)
List/dqlite-overflows-page/all-4          29.00 ±  3%   30.00 ±  3%       ~ (p=1.000 n=7)
List/dqlite-overflows-page/pagination-4   33.00 ±  3%   33.00 ± 12%       ~ (p=0.437 n=7)
Update/sqlite-4                           671.0 ±  1%   671.0 ±  2%       ~ (p=0.237 n=7)
Update/dqlite-4                           492.0 ±  0%   492.0 ±  0%       ~ (p=1.000 n=7) ¹
geomean                                   105.4         105.5        +0.14%
¹ all samples are equal

                                        │        Before         │                    After                    │
                                        │ network-bytes-read/op │ network-bytes-read/op  vs base              │
Compaction/dqlite-4                               1.891Ki ± 77%           1.818Ki ± 81%       ~ (p=0.620 n=7)
Create/dqlite-4                                     241.6 ±  0%             240.9 ±  0%  -0.29% (p=0.019 n=7)
Delete/dqlite-4                                   1.159Ki ±  0%           1.159Ki ±  0%       ~ (p=0.714 n=7)
Get/dqlite-4                                      1.133Ki ±  0%           1.133Ki ±  0%       ~ (p=0.735 n=7)
Lease/dqlite-4                                    0.03280 ± 27%           0.03280 ± 27%       ~ (p=0.288 n=7)
List/dqlite-tiny/all-4                              201.7 ±  1%             185.4 ± 11%  -8.08% (p=0.005 n=7)
List/dqlite-tiny/pagination-4                       189.2 ±  1%             189.3 ±  1%       ~ (p=0.930 n=7)
List/dqlite-fits-in-page/all-4                    1.227Ki ±  2%           1.228Ki ±  2%       ~ (p=0.926 n=7)
List/dqlite-fits-in-page/pagination-4             1.203Ki ±  3%           1.248Ki ±  8%       ~ (p=0.073 n=7)
List/dqlite-overflows-page/all-4                  6.231Ki ±  1%           6.268Ki ±  2%       ~ (p=0.333 n=7)
List/dqlite-overflows-page/pagination-4           6.946Ki ±  2%           6.965Ki ± 11%       ~ (p=1.000 n=7)
Update/dqlite-4                                     565.0 ±  0%             564.8 ±  0%  -0.04% (p=0.044 n=7)
geomean                                             422.1                   419.3        -0.66%

                                        │          Before          │                      After                      │
                                        │ network-bytes-written/op │ network-bytes-written/op  vs base               │
Compaction/dqlite-4                                   4.530 ±  13%                4.509 ± 14%        ~ (p=0.259 n=7)
Create/dqlite-4                                       116.2 ±   1%                115.1 ±  1%   -0.95% (p=0.029 n=7)
Delete/dqlite-4                                       424.0 ±   0%                424.0 ±  0%        ~ (p=0.824 n=7)
Get/dqlite-4                                          80.57 ±   0%                80.57 ±  0%        ~ (p=0.908 n=7)
Lease/dqlite-4                                      0.02969 ± 107%              0.05920 ± 77%        ~ (p=0.299 n=7)
List/dqlite-tiny/all-4                              0.04254 ±   3%              0.03583 ± 22%  -15.77% (p=0.004 n=7)
List/dqlite-tiny/pagination-4                       0.04449 ±   5%              0.04665 ±  9%        ~ (p=0.259 n=7)
List/dqlite-fits-in-page/all-4                      0.05230 ±   2%              0.05244 ±  4%        ~ (p=1.000 n=7)
List/dqlite-fits-in-page/pagination-4               0.07369 ±  10%              0.07469 ± 30%        ~ (p=1.000 n=7)
List/dqlite-overflows-page/all-4                    0.06472 ±   5%              0.06650 ±  4%        ~ (p=0.165 n=7)
List/dqlite-overflows-page/pagination-4              0.1148 ±  11%               0.1127 ± 27%        ~ (p=0.928 n=7)
Update/dqlite-4                                       263.7 ±   0%                263.6 ±  0%   -0.04% (p=0.012 n=7)
geomean                                               1.185                       1.243         +4.92%

Current status

goos: linux
goarch: amd64
pkg: github.com/canonical/k8s-dqlite/test
cpu: AMD EPYC 7763 64-Core Processor                
                                        │    After     │
                                        │    sec/op    │
Compaction/sqlite-4                       39.42µ ± 11%
Compaction/dqlite-4                       41.45µ ±  8%
Create/sqlite-4                           1.211m ±  4%
Create/dqlite-4                           1.066m ±  2%
Delete/sqlite-4                           1.387m ±  5%
Delete/dqlite-4                           1.298m ±  6%
Get/sqlite-4                              273.0µ ±  1%
Get/dqlite-4                              360.6µ ±  1%
Lease/sqlite-4                            101.0µ ±  1%
Lease/dqlite-4                            101.2µ ±  2%
List/sqlite-tiny/all-4                    4.475µ ±  1%
List/sqlite-tiny/pagination-4             9.004µ ±  4%
List/sqlite-fits-in-page/all-4            7.501µ ±  3%
List/sqlite-fits-in-page/pagination-4     23.05µ ±  2%
List/sqlite-overflows-page/all-4          17.73µ ±  5%
List/sqlite-overflows-page/pagination-4   48.15µ ±  3%
List/dqlite-tiny/all-4                    2.685µ ±  5%
List/dqlite-tiny/pagination-4             5.934µ ±  1%
List/dqlite-fits-in-page/all-4            6.140µ ±  2%
List/dqlite-fits-in-page/pagination-4     16.90µ ±  2%
List/dqlite-overflows-page/all-4          19.25µ ±  2%
List/dqlite-overflows-page/pagination-4   43.39µ ±  3%
Update/sqlite-4                           1.285m ±  4%
Update/dqlite-4                           1.307m ±  1%
geomean                                   66.34µ

                                        │        After         │
                                        │ page-cache-misses/op │
Compaction/sqlite-4                               0.000 ± 0%
Compaction/dqlite-4                               0.000 ± 0%
Create/sqlite-4                                   0.000 ± 0%
Create/dqlite-4                                   0.000 ± 0%
Delete/sqlite-4                                   0.000 ± 0%
Delete/dqlite-4                                   0.000 ± 0%
Get/sqlite-4                                      0.000 ± 0%
Get/dqlite-4                                      0.000 ± 0%
Lease/sqlite-4                                    0.000 ± 0%
Lease/dqlite-4                                    0.000 ± 0%
List/sqlite-tiny/all-4                            0.000 ± 0%
List/sqlite-tiny/pagination-4                     0.000 ± 0%
List/sqlite-fits-in-page/all-4                    0.000 ± 0%
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%
List/sqlite-overflows-page/all-4                  0.000 ± 0%
List/sqlite-overflows-page/pagination-4           0.000 ± 0%
List/dqlite-tiny/all-4                            0.000 ± 0%
List/dqlite-tiny/pagination-4                     0.000 ± 0%
List/dqlite-fits-in-page/all-4                    0.000 ± 0%
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%
List/dqlite-overflows-page/all-4                  0.000 ± 0%
List/dqlite-overflows-page/pagination-4           0.000 ± 0%
Update/sqlite-4                                   0.000 ± 0%
Update/dqlite-4                                   0.000 ± 0%
geomean                                                      ¹
¹ summaries must be >0 to compute geomean

                                        │        After         │
                                        │ page-cache-spills/op │
Compaction/sqlite-4                               0.000 ± 0%
Compaction/dqlite-4                               0.000 ± 0%
Create/sqlite-4                                   0.000 ± 0%
Create/dqlite-4                                   0.000 ± 0%
Delete/sqlite-4                                   0.000 ± 0%
Delete/dqlite-4                                   0.000 ± 0%
Get/sqlite-4                                      0.000 ± 0%
Get/dqlite-4                                      0.000 ± 0%
Lease/sqlite-4                                    0.000 ± 0%
Lease/dqlite-4                                    0.000 ± 0%
List/sqlite-tiny/all-4                            0.000 ± 0%
List/sqlite-tiny/pagination-4                     0.000 ± 0%
List/sqlite-fits-in-page/all-4                    0.000 ± 0%
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%
List/sqlite-overflows-page/all-4                  0.000 ± 0%
List/sqlite-overflows-page/pagination-4           0.000 ± 0%
List/dqlite-tiny/all-4                            0.000 ± 0%
List/dqlite-tiny/pagination-4                     0.000 ± 0%
List/dqlite-fits-in-page/all-4                    0.000 ± 0%
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%
List/dqlite-overflows-page/all-4                  0.000 ± 0%
List/dqlite-overflows-page/pagination-4           0.000 ± 0%
Update/sqlite-4                                   0.000 ± 0%
Update/dqlite-4                                   0.000 ± 0%
geomean                                                      ¹
¹ summaries must be >0 to compute geomean

                                        │     After     │
                                        │ page-reads/op │
Compaction/sqlite-4                        12.21 ±   1%
Compaction/dqlite-4                        12.20 ±   1%
Create/sqlite-4                            14.11 ±   2%
Create/dqlite-4                            13.16 ±   1%
Delete/sqlite-4                            25.02 ±   2%
Delete/dqlite-4                            23.81 ±   1%
Get/sqlite-4                               14.96 ±   0%
Get/dqlite-4                               11.95 ±   0%
Lease/sqlite-4                            800.0µ ± 100%
Lease/dqlite-4                            800.0µ ±  50%
List/sqlite-tiny/all-4                    192.3m ±   0%
List/sqlite-tiny/pagination-4              1.095 ±   3%
List/sqlite-fits-in-page/all-4            165.1m ±   0%
List/sqlite-fits-in-page/pagination-4     965.6m ±   0%
List/sqlite-overflows-page/all-4          293.4m ±  29%
List/sqlite-overflows-page/pagination-4    2.080 ±   2%
List/dqlite-tiny/all-4                    190.4m ±   0%
List/dqlite-tiny/pagination-4              1.126 ±   0%
List/dqlite-fits-in-page/all-4            163.2m ±  59%
List/dqlite-fits-in-page/pagination-4     954.5m ±   3%
List/dqlite-overflows-page/all-4          182.9m ±   0%
List/dqlite-overflows-page/pagination-4    1.071 ±   3%
Update/sqlite-4                            25.59 ±   2%
Update/dqlite-4                            24.59 ±   1%
geomean                                    1.235

                                        │     After      │
                                        │ page-writes/op │
Compaction/sqlite-4                        207.7m ± 0%
Compaction/dqlite-4                        187.6m ± 1%
Create/sqlite-4                             5.131 ± 1%
Create/dqlite-4                             4.242 ± 1%
Delete/sqlite-4                             5.339 ± 1%
Delete/dqlite-4                             4.375 ± 1%
Get/sqlite-4                                0.000 ± 0%
Get/dqlite-4                                0.000 ± 0%
Lease/sqlite-4                              0.000 ± 0%
Lease/dqlite-4                              0.000 ± 0%
List/sqlite-tiny/all-4                      0.000 ± 0%
List/sqlite-tiny/pagination-4               0.000 ± 0%
List/sqlite-fits-in-page/all-4              0.000 ± 0%
List/sqlite-fits-in-page/pagination-4       0.000 ± 0%
List/sqlite-overflows-page/all-4            0.000 ± 0%
List/sqlite-overflows-page/pagination-4     0.000 ± 0%
List/dqlite-tiny/all-4                      0.000 ± 0%
List/dqlite-tiny/pagination-4               0.000 ± 0%
List/dqlite-fits-in-page/all-4              0.000 ± 0%
List/dqlite-fits-in-page/pagination-4       0.000 ± 0%
List/dqlite-overflows-page/all-4            0.000 ± 0%
List/dqlite-overflows-page/pagination-4     0.000 ± 0%
Update/sqlite-4                             5.142 ± 0%
Update/dqlite-4                             4.177 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                        │      After      │
                                        │ sec-reading/op  │
Compaction/sqlite-4                       118.6µ ±  19%
Compaction/dqlite-4                       29.20µ ±  60%
Create/sqlite-4                           53.40µ ± 178%
Create/dqlite-4                           10.20µ ±  19%
Delete/sqlite-4                           218.1µ ±  73%
Delete/dqlite-4                           50.50µ ±  21%
Get/sqlite-4                              88.80µ ±   7%
Get/dqlite-4                              33.50µ ±  19%
Lease/sqlite-4                            300.0n ±    ?
Lease/dqlite-4                             0.000 ±   0%
List/sqlite-tiny/all-4                    14.40µ ±   3%
List/sqlite-tiny/pagination-4             19.90µ ±   6%
List/sqlite-fits-in-page/all-4            18.20µ ±  10%
List/sqlite-fits-in-page/pagination-4     63.20µ ±   3%
List/sqlite-overflows-page/all-4          70.40µ ±  15%
List/sqlite-overflows-page/pagination-4   179.7µ ±   3%
List/dqlite-tiny/all-4                    3.500µ ±  11%
List/dqlite-tiny/pagination-4             8.700µ ±  11%
List/dqlite-fits-in-page/all-4            10.20µ ±  15%
List/dqlite-fits-in-page/pagination-4     26.40µ ±  13%
List/dqlite-overflows-page/all-4          32.80µ ±   2%
List/dqlite-overflows-page/pagination-4   72.60µ ±   7%
Update/sqlite-4                           176.8µ ±  66%
Update/dqlite-4                           110.4µ ±  14%
geomean                                                 ¹
¹ summaries must be >0 to compute geomean

                                        │     After      │
                                        │ sec-writing/op │
Compaction/sqlite-4                       10.10µ ±  8%
Compaction/dqlite-4                       7.200µ ±  8%
Create/sqlite-4                           947.6µ ±  4%
Create/dqlite-4                           34.50µ ± 26%
Delete/sqlite-4                           1.006m ± 10%
Delete/dqlite-4                           22.30µ ± 35%
Get/sqlite-4                               0.000 ±  0%
Get/dqlite-4                               0.000 ±  0%
Lease/sqlite-4                             0.000 ±  0%
Lease/dqlite-4                             0.000 ±  0%
List/sqlite-tiny/all-4                     0.000 ±  0%
List/sqlite-tiny/pagination-4              0.000 ±  0%
List/sqlite-fits-in-page/all-4             0.000 ±  0%
List/sqlite-fits-in-page/pagination-4      0.000 ±  0%
List/sqlite-overflows-page/all-4           0.000 ±  0%
List/sqlite-overflows-page/pagination-4    0.000 ±  0%
List/dqlite-tiny/all-4                     0.000 ±  0%
List/dqlite-tiny/pagination-4              0.000 ±  0%
List/dqlite-fits-in-page/all-4             0.000 ±  0%
List/dqlite-fits-in-page/pagination-4      0.000 ±  0%
List/dqlite-overflows-page/all-4           0.000 ±  0%
List/dqlite-overflows-page/pagination-4    0.000 ±  0%
Update/sqlite-4                           837.0µ ±  6%
Update/dqlite-4                           23.70µ ± 44%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                        │     After     │
                                        │     B/op      │
Compaction/sqlite-4                        1008.0 ± 68%
Compaction/dqlite-4                       4.999Ki ± 79%
Create/sqlite-4                           21.31Ki ±  0%
Create/dqlite-4                           20.07Ki ±  0%
Delete/sqlite-4                           32.52Ki ±  2%
Delete/dqlite-4                           29.46Ki ±  0%
Get/sqlite-4                              25.08Ki ±  0%
Get/dqlite-4                              22.89Ki ±  0%
Lease/sqlite-4                            12.33Ki ±  0%
Lease/dqlite-4                            12.33Ki ±  0%
List/sqlite-tiny/all-4                    1.261Ki ±  2%
List/sqlite-tiny/pagination-4             1.789Ki ±  4%
List/sqlite-fits-in-page/all-4            5.188Ki ±  3%
List/sqlite-fits-in-page/pagination-4     10.24Ki ±  0%
List/sqlite-overflows-page/all-4          26.74Ki ±  7%
List/sqlite-overflows-page/pagination-4   46.12Ki ±  0%
List/dqlite-tiny/all-4                      830.0 ±  7%
List/dqlite-tiny/pagination-4               833.0 ±  2%
List/dqlite-fits-in-page/all-4            4.292Ki ±  1%
List/dqlite-fits-in-page/pagination-4     4.400Ki ±  6%
List/dqlite-overflows-page/all-4          21.30Ki ±  1%
List/dqlite-overflows-page/pagination-4   22.95Ki ±  6%
Update/sqlite-4                           30.33Ki ±  1%
Update/dqlite-4                           27.61Ki ±  0%
geomean                                   9.193Ki

                                        │    After    │
                                        │  allocs/op  │
Compaction/sqlite-4                       39.00 ± 74%
Compaction/dqlite-4                       121.0 ± 80%
Create/sqlite-4                           453.0 ±  1%
Create/dqlite-4                           372.0 ±  0%
Delete/sqlite-4                           724.0 ±  2%
Delete/dqlite-4                           511.0 ±  0%
Get/sqlite-4                              563.0 ±  0%
Get/dqlite-4                              407.0 ±  0%
Lease/sqlite-4                            224.0 ±  0%
Lease/dqlite-4                            224.0 ±  0%
List/sqlite-tiny/all-4                    41.00 ±  2%
List/sqlite-tiny/pagination-4             62.00 ±  5%
List/sqlite-fits-in-page/all-4            48.00 ±  4%
List/sqlite-fits-in-page/pagination-4     93.00 ±  0%
List/sqlite-overflows-page/all-4          57.00 ±  5%
List/sqlite-overflows-page/pagination-4   93.00 ±  0%
List/dqlite-tiny/all-4                    13.00 ±  8%
List/dqlite-tiny/pagination-4             13.00 ±  0%
List/dqlite-fits-in-page/all-4            18.00 ±  6%
List/dqlite-fits-in-page/pagination-4     18.00 ±  6%
List/dqlite-overflows-page/all-4          30.00 ±  3%
List/dqlite-overflows-page/pagination-4   33.00 ± 12%
Update/sqlite-4                           671.0 ±  2%
Update/dqlite-4                           492.0 ±  0%
geomean                                   105.5

                                        │         After         │
                                        │ network-bytes-read/op │
Compaction/dqlite-4                               1.818Ki ± 81%
Create/dqlite-4                                     240.9 ±  0%
Delete/dqlite-4                                   1.159Ki ±  0%
Get/dqlite-4                                      1.133Ki ±  0%
Lease/dqlite-4                                    0.03280 ± 27%
List/dqlite-tiny/all-4                              185.4 ± 11%
List/dqlite-tiny/pagination-4                       189.3 ±  1%
List/dqlite-fits-in-page/all-4                    1.228Ki ±  2%
List/dqlite-fits-in-page/pagination-4             1.248Ki ±  8%
List/dqlite-overflows-page/all-4                  6.268Ki ±  2%
List/dqlite-overflows-page/pagination-4           6.965Ki ± 11%
Update/dqlite-4                                     564.8 ±  0%
geomean                                             419.3

                                        │          After           │
                                        │ network-bytes-written/op │
Compaction/dqlite-4                                    4.509 ± 14%
Create/dqlite-4                                        115.1 ±  1%
Delete/dqlite-4                                        424.0 ±  0%
Get/dqlite-4                                           80.57 ±  0%
Lease/dqlite-4                                       0.05920 ± 77%
List/dqlite-tiny/all-4                               0.03583 ± 22%
List/dqlite-tiny/pagination-4                        0.04665 ±  9%
List/dqlite-fits-in-page/all-4                       0.05244 ±  4%
List/dqlite-fits-in-page/pagination-4                0.07469 ± 30%
List/dqlite-overflows-page/all-4                     0.06650 ±  4%
List/dqlite-overflows-page/pagination-4               0.1127 ± 27%
Update/dqlite-4                                        263.6 ±  0%
geomean                                                1.243

bschimke95

LGTM, just minor nits/questions

cmd/root.go

pkg/kine/drivers/generic/generic.go

marco6

Nice work on making this more configurable!

I see that this affects performance quite a bit, especially for leases. Can we try to understand why? I just want to make sure that a MaxLifetime does not impact with the correctness of the queries (i.e. the connection expires and we miss some updates while reconnecting).

Also, couple of nitpicks I noticed while reading.

pkg/kine/drivers/sqlite/sqlite_test.go

pkg/server/server.go

louiseschmidtgen · 2024-08-30T11:03:28Z

@marco6 Regarding the benchmark: The lease test calls leaseGrant which in return does no backend calls and only returns a response. I am not sure why the network-bytes-written/op is increased, however I think the total amount is negligible compared to the other operations calls' total network-bytes-written/op. What do you think?

marco6 · 2024-09-03T14:58:49Z

@marco6 Regarding the benchmark: The lease test calls leaseGrant which in return does no backend calls and only returns a response. I am not sure why the network-bytes-written/op is increased, however I think the total amount is negligible compared to the other operations calls' total network-bytes-written/op. What do you think?

Seems like the test was not on the database after all. Removing the test seems like way to go, but maybe in another PR.

marco6

Nice work! 🎉 There are still a couple of nitpicks, but I'm pre-approving this so you can merge as soon as they are fixed!

pkg/kine/drivers/generic/generic.go

louiseschmidtgen requested a review from a team as a code owner August 14, 2024 08:08

louiseschmidtgen force-pushed the KU-1282/query-connection branch from e7a7872 to 61b99e3 Compare August 14, 2024 14:35

louiseschmidtgen force-pushed the KU-1282/query-connection branch from 61b99e3 to 0d4d7ed Compare August 27, 2024 12:54

bschimke95 approved these changes Aug 29, 2024

View reviewed changes

cmd/root.go Outdated Show resolved Hide resolved

pkg/kine/drivers/generic/generic.go Show resolved Hide resolved

marco6 requested changes Aug 29, 2024

View reviewed changes

pkg/kine/drivers/sqlite/sqlite_test.go Outdated Show resolved Hide resolved

pkg/server/server.go Outdated Show resolved Hide resolved

louiseschmidtgen force-pushed the KU-1282/query-connection branch from 478dcc8 to 1c6af34 Compare August 29, 2024 13:10

louiseschmidtgen added 4 commits August 30, 2024 11:37

make connection pool configurable

0eb9d74

tweak idle time

e8c64b8

defaults

ac09e9c

pass conn pool opts better

bf66a62

louiseschmidtgen force-pushed the KU-1282/query-connection branch from f43a6d1 to bf66a62 Compare August 30, 2024 09:38

marco6 approved these changes Sep 3, 2024

View reviewed changes

pkg/kine/drivers/generic/generic.go Outdated Show resolved Hide resolved

pkg/kine/drivers/generic/generic.go Outdated Show resolved Hide resolved

louiseschmidtgen added 2 commits September 3, 2024 17:29

comments

841dce7

fix

faaeae2

louiseschmidtgen merged commit 358d1f0 into master Sep 4, 2024
7 checks passed

marco6 deleted the KU-1282/query-connection branch September 26, 2024 06:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make connection pool configurable #158

Make connection pool configurable #158

louiseschmidtgen commented Aug 14, 2024 •

edited

Loading

github-actions bot commented Aug 14, 2024 •

edited

Loading

bschimke95 left a comment

marco6 left a comment

louiseschmidtgen commented Aug 30, 2024

marco6 commented Sep 3, 2024

marco6 left a comment

Make connection pool configurable #158

Make connection pool configurable #158

Conversation

louiseschmidtgen commented Aug 14, 2024 • edited Loading