mirror of
https://github.com/fluencelabs/redis
synced 2025-03-17 16:10:50 +00:00
We have 24 total bits of space in each object in order to implement an LFU (Least Frequently Used) eviction policy. We split the 24 bits into two fields: 8 bits 16 bits +--------+----------------+ | LOG_C | Last decr time | +--------+----------------+ LOG_C is a logarithmic counter that provides an indication of the access frequency. However this field must also be deceremented otherwise what used to be a frequently accessed key in the past, will remain ranked like that forever, while we want the algorithm to adapt to access pattern changes. So the remaining 16 bits are used in order to store the "decrement time", a reduced-precision unix time (we take 16 bits of the time converted in minutes since we don't care about wrapping around) where the LOG_C counter is halved if it has an high value, or just decremented if it has a low value. New keys don't start at zero, in order to have the ability to collect some accesses before being trashed away, so they start at COUNTER_INIT_VAL. The logaritmic increment performed on LOG_C takes care of COUNTER_INIT_VAL when incrementing the key, so that keys starting at COUNTER_INIT_VAL (or having a smaller value) have a very high chance of being incremented on access. The simulation starts with a power-law access pattern, and later converts into a flat access pattern in order to see how the algorithm adapts. Currenty the decrement operation period is 1 minute, however note that it is not guaranteed that each key will be scanned 1 time every minute, so the actual frequency can be lower. However under high load, we access 3/5 keys every newly inserted key (because of how Redis eviction works). This is a work in progress at this point to evaluate if this works well.