feat: Request-ID plugin add snowflake algorithm #4559
Conversation
dickens7
changed the title
|
Need to make the CI pass. |
|
CI help |
Rerun to see how it goes Good to go |
|
Hi @dickens7 do you mind also add documentations of this feature |
|
|
| end | ||
| return snowflake:next_id() | ||
| end | ||
|
|
There was a problem hiding this comment.
Could we put them under utils? Snowflake ID generating should also be used elsewhere.
There was a problem hiding this comment.
Let's implement this in the next pr
|
Please solve the conflict. |
docs/en/latest/plugins/request-id.md
Outdated
|
|
||
| | Name | Type | Requirement | Default | Valid | Description | | ||
| | ------------------- | ------- | ------------- | -------------- | ------- | ------------------------------ | | ||
| | enable | boolean | required | false | | When set it to true, enable the snowflake algorithm. | |
There was a problem hiding this comment.
| | enable | boolean | required | false | | When set it to true, enable the snowflake algorithm. | | |
| | enable | boolean | optional | false | | When set it to true, enable the snowflake algorithm. | |
There was a problem hiding this comment.
Modified jsonSchema required = {"enable", "snowflake_epoc "}
There was a problem hiding this comment.
Once you provide a default value, the field is actually optional, as users don't need to specify their values.
There was a problem hiding this comment.
Got it. I'll make it optional
| goto continue | ||
| end | ||
|
|
||
| local _, err1 = etcd_cli:setnx(prefix .. tostring(id), uuid) |
There was a problem hiding this comment.
So we have to write ETCD so that we can write the uuid?
There was a problem hiding this comment.
The purpose of writing etcd here is to generate DataMachineId for snowflake. The value is uuid because etcd_cli:setnx still returns succeeded even if the key exists. UUID is used for verification after etcd_cli:setnx.
| snowflake: | ||
| enable: false | ||
| snowflake_epoc: 1609459200000 # the starting timestamp is expressed in milliseconds | ||
| data_machine_bits: 12 # data machine bit, maximum 31, because Lua cannot do bit operations greater than 31 |
There was a problem hiding this comment.
should check data_machine_bits + sequence_bits = 22 always? may the test case need to cover it.
There was a problem hiding this comment.
data_machine_bits and sequence_bits are not fixed and can be configured according to different requirements
|
|
||
| - `snowflake_epoc` default start time is `2021-01-01T00:00:00Z`, and it can support `69 year` approximately to `2090-09-0715:47:35Z` according to the default configuration | ||
| - `data_machine_bits` corresponds to the set of workIDs and datacEnteridd in the snowflake definition. The plug-in aslocates a unique ID to each process. Maximum number of supported processes is `pow(2, data_machine_bits)`. The default number of `12 bits` is up to `4096`. | ||
| - `sequence_bits` defaults to `10 bits` and each process generates up to `1024` ID per second |
There was a problem hiding this comment.
The per second here and the per millisecond above confuse me
There was a problem hiding this comment.
This is the definition of two parts of the snowflake algorithm. Millisecond is the definition of the timestamp part, which means that the timestamp part is in milliseconds. Second is the definition of the sequence number part, which means that the number of id can be generated per second
| ngx.say("ids not unique") | ||
| return | ||
| end | ||
| ids[id] = true |
There was a problem hiding this comment.
Is there any way to check that the length of the ids has reached a fixed value? such as 100, or at least 2. I am worried that there is no data in the ids due to sync.
|
LGTM |
| @@ -53,7 +221,6 @@ function _M.rewrite(conf, ctx) | |||
| end | |||
| end | |||
|
|
|||
|
|
|||
There was a problem hiding this comment.
why remove this blank?
What this PR does / why we need it:
#4209
Pre-submission checklist: