Persisting entity information

[renegat4]

Hi,
after reading the docs and playing arround with flipt i can't find where flipt stores information about which variant an entity got.

Doing requests with the same entitiyId always returns the same value. Which is perfectly what i expected. Even after restarting flipt.
But there seems to be no information about this in the database.

[markphelps]

Hey @renegat4, great question!

You're correct that Flipt doesn't store any info about entityID or which variant an entity matches in the database. This is for two main reasons:

1. I want it to be fast since evaluation is the most common use case and depends on traffic to your app/site, so I don't want Flipt to be the bottleneck
2. Keeping track of entityIDs would likely result in a large amount of data being stored since they are basically infinite as they can be any value

So instead, Flipt uses a technique where it hashes the entityID with the flagKey using this function which uses the stdlib hash/crc32.ChecksumIEEE function to create the hash.

It then creates a slice of 'buckets' along with a sorted slice of distributions. The hashed value from above is then checked to see where it would 'fit' in these buckets. All of the code where this is done is here.

So, if distribution A has a 30% rollout, then it would 'take up' buckets 0 - 299 (out of 1k buckets).
Then distribution B would take up the remaining buckets 300-999.

The hashed value is a 32bit integer, which is then mod'ed (% numBuckets) or (% 1000) so that it is guaranteed to be a number between 0 - 999.

All of this combined (using consistent hashing of entityID and flagKey with the ordered search of 'buckets' which map to the distributions) allows Flipt to consistently return the correct distribution each time.

Hope that helps!

[sashayakovtseva]

I think you should add this to your docs :)

[markphelps]

great idea @sashayakovtseva . will do in the morning!

[markphelps]

flipt-io/docs#84