mirror of
https://github.com/fluencelabs/redis
synced 2025-03-26 04:11:04 +00:00
09fcb00249
We have 24 total bits of space in each object in order to implement
an LFU (Least Frequently Used) eviction policy.
We split the 24 bits into two fields:
8 bits 16 bits
+--------+----------------+
| LOG_C | Last decr time |
+--------+----------------+
LOG_C is a logarithmic counter that provides an indication of the access
frequency. However this field must also be deceremented otherwise what used
to be a frequently accessed key in the past, will remain ranked like that
forever, while we want the algorithm to adapt to access pattern changes.
So the remaining 16 bits are used in order to store the "decrement time",
a reduced-precision unix time (we take 16 bits of the time converted
in minutes since we don't care about wrapping around) where the LOG_C
counter is halved if it has an high value, or just decremented if it
has a low value.
New keys don't start at zero, in order to have the ability to collect
some accesses before being trashed away, so they start at COUNTER_INIT_VAL.
The logaritmic increment performed on LOG_C takes care of COUNTER_INIT_VAL
when incrementing the key, so that keys starting at COUNTER_INIT_VAL
(or having a smaller value) have a very high chance of being incremented
on access.
The simulation starts with a power-law access pattern, and later converts
into a flat access pattern in order to see how the algorithm adapts.
Currenty the decrement operation period is 1 minute, however note that
it is not guaranteed that each key will be scanned 1 time every minute,
so the actual frequency can be lower. However under high load, we access
3/5 keys every newly inserted key (because of how Redis eviction works).
This is a work in progress at this point to evaluate if this works well.
The test-lru.rb program can be used in order to check the behavior of the
Redis approximated LRU algorithm against the theoretical output of true
LRU algorithm.
In order to use the program you need to recompile Redis setting the define
REDIS_LRU_CLOCK_RESOLUTION to 1, by editing the file server.h.
This allows to execute the program in a fast way since the 1 ms resolution
is enough for all the objects to have a different enough time stamp during
the test.
The program is executed like this:
ruby test-lru.rb /tmp/lru.html
You can optionally specify a number of times to run, so that the program
will output averages of different runs, by adding an additional argument.
For instance in order to run the test 10 times use:
ruby test-lru.rb /tmp/lru.html 10