fluencelabs/redis
1
0
Fork 0
mirror of https://github.com/fluencelabs/redis synced 2025-03-19 17:10:50 +00:00
Code Issues Projects Releases Wiki Activity

activeExpireCycle() smarter with many DBs and under expire pressure.

Browse Source 
activeExpireCycle() tries to test just a few DBs per iteration so that
it scales if there are many configured DBs in the Redis instance.
However this commit makes it a bit smarter when one a few of those DBs
are under expiration pressure and there are many many keys to expire.

What we do is to remember if in the last iteration had to return because
we ran out of time. In that case the next iteration we'll test all the
configured DBs so that we are sure we'll test again the DB under
pressure.

Before of this commit after some mass-expire in a given DB the function
tested just a few of the next DBs, possibly empty, a few per iteration,
so it took a long time for the function to reach again the DB under
pressure. This resulted in a lot of memory being used by already expired
keys and never accessed by clients.
This commit is contained in:
antirez 2013-03-11 11:10:33 +01:00
parent 08b107e405
commit 2d851333a6
1 changed files with 20 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
src/redis.c
View File

@ -648,19 +648,31 @@ void updateDictResizePolicy(void) {
* No more than REDIS_DBCRON_DBS_PER_CALL databases are tested at every * No more than REDIS_DBCRON_DBS_PER_CALL databases are tested at every
* iteration. */ * iteration. */
void activeExpireCycle(void) { void activeExpireCycle(void) {
static unsigned int current_db = 0; /* This function has some global state in order to continue the work
* incrementally across calls. */
static unsigned int current_db = 0; /* Last DB tested. */
static int timelimit_exit = 0; /* Time limit hit in previous call? */
unsigned int j, iteration = 0; unsigned int j, iteration = 0;
unsigned int dbs_per_call = REDIS_DBCRON_DBS_PER_CALL; unsigned int dbs_per_call = REDIS_DBCRON_DBS_PER_CALL;
long long start = ustime(), timelimit; long long start = ustime(), timelimit;
/* Don't test more DBs than we have. */ /* We usually should test REDIS_DBCRON_DBS_PER_CALL per iteration, with
if (dbs_per_call > server.dbnum) dbs_per_call = server.dbnum; * two exceptions:
*
* 1) Don't test more DBs than we have.
* 2) If last time we hit the time limit, we want to scan all DBs
* in this iteration, as there is work to do in some DB and we don't want
* expired keys to use memory for too much time. */
if (dbs_per_call > server.dbnum || timelimit_exit)
dbs_per_call = server.dbnum;
/* We can use at max REDIS_EXPIRELOOKUPS_TIME_PERC percentage of CPU time /* We can use at max REDIS_EXPIRELOOKUPS_TIME_PERC percentage of CPU time
* per iteration. Since this function gets called with a frequency of * per iteration. Since this function gets called with a frequency of
* server.hz times per second, the following is the max amount of * server.hz times per second, the following is the max amount of
* microseconds we can spend in this function. */ * microseconds we can spend in this function. */
timelimit = 1000000*REDIS_EXPIRELOOKUPS_TIME_PERC/server.hz/100; timelimit = 1000000*REDIS_EXPIRELOOKUPS_TIME_PERC/server.hz/100;
timelimit_exit = 0;
if (timelimit <= 0) timelimit = 1; if (timelimit <= 0) timelimit = 1;
for (j = 0; j < dbs_per_call; j++) { for (j = 0; j < dbs_per_call; j++) {
@ -718,7 +730,11 @@ void activeExpireCycle(void) {
* caller waiting for the other active expire cycle. */ * caller waiting for the other active expire cycle. */
iteration++; iteration++;
if ((iteration & 0xf) == 0 && /* check once every 16 iterations. */ if ((iteration & 0xf) == 0 && /* check once every 16 iterations. */
(ustime()-start) > timelimit) return; (ustime()-start) > timelimit)
{
timelimit_exit = 1;
return;
}
} while (expired > REDIS_EXPIRELOOKUPS_PER_CRON/4); } while (expired > REDIS_EXPIRELOOKUPS_PER_CRON/4);
} }
} }