From 5d373da96a7c2b38912cd925c456d719ea1a8933 Mon Sep 17 00:00:00 2001 From: antirez Date: Fri, 14 May 2010 18:58:37 +0200 Subject: [PATCH] ZUNION,ZINTER -> ZUNIONSTORE,ZINTERSTORE --- doc/CommandReference.html | 2 +- doc/SortCommand.html | 13 ++++++----- doc/SortedSetCommandsSidebar.html | 2 +- doc/SortedSets.html | 2 +- doc/ZrangebyscoreCommand.html | 36 +++++++++++++++++++++++++++++-- redis.c | 16 +++++++------- staticsymbols.h | 18 ++++++++++++++-- tests/unit/type/zset.tcl | 36 +++++++++++++++---------------- 8 files changed, 87 insertions(+), 38 deletions(-) diff --git a/doc/CommandReference.html b/doc/CommandReference.html index 4d433624..6541e3a4 100644 --- a/doc/CommandReference.html +++ b/doc/CommandReference.html @@ -31,7 +31,7 @@

Commands operating on string values

Commands operating on lists

Commands operating on sets

-

Commands operating on sorted sets (zsets, Redis version >

1.1) ==

+

Commands operating on sorted sets (zsets, Redis version >

1.1) ==

Commands operating on hashes

Sorting

Transactions

diff --git a/doc/SortCommand.html b/doc/SortCommand.html index 7ed3eb01..90c8ac7f 100644 --- a/doc/SortCommand.html +++ b/doc/SortCommand.html @@ -16,7 +16,7 @@
-SortCommand: Contents
    Sorting by external keys
    Retrieving external keys
    Storing the result of a SORT operation
    SORT and Hashes: BY and GET by hash field
    Return value +SortCommand: Contents
    Sorting by external keys
    Not Sorting at all
    Retrieving external keys
    Storing the result of a SORT operation
    SORT and Hashes: BY and GET by hash field
    Return value

SortCommand

@@ -46,19 +46,22 @@ SORT mylist LIMIT 0 10 ALPHA DESC SORT mylist BY weight_*
the BY option takes a pattern (weight_* in our example) that is usedin order to generate the key names of the weights used for sorting.Weight key names are obtained substituting the first occurrence of *with the actual value of the elements on the list (1,2,3,4 in our example).
Our previous example will return just the sorted IDs. Often it isneeded to get the actual objects sorted (object_1, ..., object_4 in theexample). We can do it with the following command:
-

Retrieving external keys

+
Not Sorting at all
+SORT mylist BY nosort
+
also the BY option can take a "nosort" specifier. This is useful if you want to retrieve a external key (using GET, read below) but you don't want the sorting overhead.

+
Retrieving external keys
 SORT mylist BY weight_* GET object_*
 
Note that GET can be used multiple times in order to get more keys forevery element of the original List, Set or Sorted Set sorted.

 
Since Redis >= 1.1 it's possible to also GET the list elements itselfusing the special # pattern:

-
+
 SORT mylist BY weight_* GET object_* GET #
 
Storing the result of a SORT operation
By default SORT returns the sorted elements as its return value.Using the STORE option instead to return the elements SORT willstore this elements as a Redis List in the specified key.An example:

-
+
 SORT mylist BY weight_* STORE resultkey
 
An interesting pattern using SORT ... STORE consists in associatingan EXPIRE timeout to the resulting key so that inapplications where the result of a sort operation can be cached forsome time other clients will use the cached list instead to call SORTfor every request. When the key will timeout an updated version ofthe cache can be created using SORT ... STORE again.

 
Note that implementing this pattern it is important to avoid that multipleclients will try to rebuild the cached version of the cacheat the same time, so some form of locking should be implemented(for instance using SETNX).

 
SORT and Hashes: BY and GET by hash field

-
It's possible to use BY and GET options against Hash fields using the following syntax:
+
It's possible to use BY and GET options against Hash fields using the following syntax:
 SORT mylist BY weight_*->fieldname
 SORT mylist GET object_*->fieldname
 

diff --git a/doc/SortedSetCommandsSidebar.html b/doc/SortedSetCommandsSidebar.html
index 8fb879d7..2534beb2 100644
--- a/doc/SortedSetCommandsSidebar.html
+++ b/doc/SortedSetCommandsSidebar.html
@@ -26,7 +26,7 @@
- == Sorted Set Commands ==

+ == Sorted Set Commands ==

diff --git a/doc/SortedSets.html b/doc/SortedSets.html index e31550b5..a9d5f8b3 100644 --- a/doc/SortedSets.html +++ b/doc/SortedSets.html @@ -26,7 +26,7 @@
- #sidebar SortedSetCommandsSidebar

Redis Sorted Set Type

Redis Sorted Sets are, similarly to Sets, collections of Redis Strings. The difference is that every member of a Sorted Set hash an associated score that is used in order to take this member in order.

The ZaddCommand command is used to add a new member to a Sorted Set, specifying the score of the element. Calling ZADD against a member already present in the sorted set but using a different score will update the score for the element, moving it to the right position in order to preserve ordering.

It's possible to get ranges of elements from Sorted Sets in a very similar way to what happens with Lists and the LRANGE command using the Sorted Sets ZRANGE command.

It's also possible to get or remove ranges of elements by score using the ZRANGEBYSCORE and ZREMRANGEBYSCORE commands.

The max number of members in a sorted set is 232-1 (4294967295, more than 4 billion of members per set).

Note that while Sorted Sets are already ordered, it is still possible to use the SORT command against sorted sets to get the elements in a different order.

Implementation details

Redis Sets are implemented using a dual-ported data structure containing a skip list and an hash table. When an element is added a map between the element and the score is added to the hash table (so that given the element we get the score in O(1)), and a map between the score and the element is added in the skip list so that elements are taken in order.

Redis uses a special skip list implementation that is doubly linked so that it's possible to traverse the sorted set from tail to head if needed (Check the ZREVRANGE command).

When ZADD is used in order to update the score of an element, Redis retrieve the score of the element using the hash table, so that it's fast to access the element inside the skip list (that's indexed by score) in order to update the position.

Like it happens for Sets the hash table resizing is a blocking operation performed synchronously so working with huge sorted sets (consisting of many millions of elements) care should be taken when mass-inserting a very big amount of elements in a Set while other clients are querying Redis at high speed.

It is possible that in the near future Redis will switch to skip lists even for the element => score map, so every Sorted Set will have two skip lists, one indexed by element and one indexed by score. + #sidebar SortedSetCommandsSidebar

Redis Sorted Set Type

Redis Sorted Sets are, similarly to Sets, collections of Redis Strings. The difference is that every member of a Sorted Set hash an associated score that is used in order to take this member in order.

The ZADD command is used to add a new member to a Sorted Set, specifying the score of the element. Calling ZADD against a member already present in the sorted set but using a different score will update the score for the element, moving it to the right position in order to preserve ordering.

It's possible to get ranges of elements from Sorted Sets in a very similar way to what happens with Lists and the LRANGE command using the Sorted Sets ZRANGE command.

It's also possible to get or remove ranges of elements by score using the ZRANGEBYSCORE and ZREMRANGEBYSCORE commands.

The max number of members in a sorted set is 232-1 (4294967295, more than 4 billion of members per set).

Note that while Sorted Sets are already ordered, it is still possible to use the SORT command against sorted sets to get the elements in a different order.

Implementation details

Redis Sets are implemented using a dual-ported data structure containing a skip list and an hash table. When an element is added a map between the element and the score is added to the hash table (so that given the element we get the score in O(1)), and a map between the score and the element is added in the skip list so that elements are taken in order.

Redis uses a special skip list implementation that is doubly linked so that it's possible to traverse the sorted set from tail to head if needed (Check the ZREVRANGE command).

When ZADD is used in order to update the score of an element, Redis retrieve the score of the element using the hash table, so that it's fast to access the element inside the skip list (that's indexed by score) in order to update the position.

Like it happens for Sets the hash table resizing is a blocking operation performed synchronously so working with huge sorted sets (consisting of many millions of elements) care should be taken when mass-inserting a very big amount of elements in a Set while other clients are querying Redis at high speed.

It is possible that in the near future Redis will switch to skip lists even for the element => score map, so every Sorted Set will have two skip lists, one indexed by element and one indexed by score.
diff --git a/doc/ZrangebyscoreCommand.html b/doc/ZrangebyscoreCommand.html index 42982251..583e9303 100644 --- a/doc/ZrangebyscoreCommand.html +++ b/doc/ZrangebyscoreCommand.html @@ -16,7 +16,7 @@
-ZrangebyscoreCommand: Contents
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` (Redis >
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` `[`WITHSCORES`]` (Redis >
    Return value +ZrangebyscoreCommand: Contents
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` (Redis >
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` `[`WITHSCORES`]` (Redis >
    Exclusive intervals and infinity
    Return value
    Examples

ZrangebyscoreCommand

@@ -30,8 +30,40 @@

ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` `[`WITHSCORES`]` (Redis >

1.3.4) = Time complexity: O(log(N))+O(M) with N being the number of elements in the sorted set and M the number of elements returned by the command, so if M is constant (for instance you always ask for the first ten elements with LIMIT) you can consider it O(log(N))
Return the all the elements in the sorted set at key with a score between_min_ and max (including elements with score equal to min or max).
The elements having the same score are returned sorted lexicographically asASCII strings (this follows from a property of Redis sorted sets and does notinvolve further computation).
-
Using the optional LIMIT it's possible to get only a range of the matchingelements in an SQL-alike way. Note that if offset is large the commandsneeds to traverse the list for offset elements and this adds up to theO(M) figure.
+
Using the optional LIMIT it's possible to get only a range of the matchingelements in an SQL-alike way. Note that if offset is large the commandsneeds to traverse the list for offset elements and this adds up to theO(M) figure.

Exclusive intervals and infinity

+min and max can be -inf and +inf, so that you are not required to know what's the greatest or smallest element in order to take, for instance, elements "up to a given value".

Also while the interval is for default closed (inclusive) it's possible to specify open intervals prefixing the score with a "(" character, so for instance: +
+ZRANGEBYSCORE zset (1.3 5
+
+Will return all the values with score > 1.3 and <= 5, while for instance: +
+ZRANGEBYSCORE zset (5 (10
+
+Will return all the values with score > 5 and < 10 (5 and 10 excluded).

Return value

Multi bulk reply, specifically a list of elements in the specified score range. +

Examples

+
+redis> zadd zset 1 foo
+(integer) 1
+redis> zadd zset 2 bar
+(integer) 1
+redis> zadd zset 3 biz
+(integer) 1
+redis> zadd zset 4 foz
+(integer) 1
+redis> zrangebyscore zset -inf +inf
+1. "foo"
+2. "bar"
+3. "biz"
+4. "foz"
+redis> zrangebyscore zset 1 2
+1. "foo"
+2. "bar"
+redis> zrangebyscore zset (1 2
+1. "bar"
+redis> zrangebyscore zset (1 (2
+(empty list or set)
+
diff --git a/redis.c b/redis.c index 99e1d72f..b5bbd04d 100644 --- a/redis.c +++ b/redis.c @@ -723,8 +723,8 @@ static void hmgetCommand(redisClient *c); static void hdelCommand(redisClient *c); static void hlenCommand(redisClient *c); static void zremrangebyrankCommand(redisClient *c); -static void zunionCommand(redisClient *c); -static void zinterCommand(redisClient *c); +static void zunionstoreCommand(redisClient *c); +static void zinterstoreCommand(redisClient *c); static void hkeysCommand(redisClient *c); static void hvalsCommand(redisClient *c); static void hgetallCommand(redisClient *c); @@ -785,8 +785,8 @@ static struct redisCommand cmdTable[] = { {"zrem",zremCommand,3,REDIS_CMD_BULK,NULL,1,1,1}, {"zremrangebyscore",zremrangebyscoreCommand,4,REDIS_CMD_INLINE,NULL,1,1,1}, {"zremrangebyrank",zremrangebyrankCommand,4,REDIS_CMD_INLINE,NULL,1,1,1}, - {"zunion",zunionCommand,-4,REDIS_CMD_INLINE|REDIS_CMD_DENYOOM,zunionInterBlockClientOnSwappedKeys,0,0,0}, - {"zinter",zinterCommand,-4,REDIS_CMD_INLINE|REDIS_CMD_DENYOOM,zunionInterBlockClientOnSwappedKeys,0,0,0}, + {"zunionstore",zunionstoreCommand,-4,REDIS_CMD_INLINE|REDIS_CMD_DENYOOM,zunionInterBlockClientOnSwappedKeys,0,0,0}, + {"zinterstore",zinterstoreCommand,-4,REDIS_CMD_INLINE|REDIS_CMD_DENYOOM,zunionInterBlockClientOnSwappedKeys,0,0,0}, {"zrange",zrangeCommand,-4,REDIS_CMD_INLINE,NULL,1,1,1}, {"zrangebyscore",zrangebyscoreCommand,-4,REDIS_CMD_INLINE,NULL,1,1,1}, {"zcount",zcountCommand,4,REDIS_CMD_INLINE,NULL,1,1,1}, @@ -5917,7 +5917,7 @@ static void zunionInterGenericCommand(redisClient *c, robj *dstkey, int op) { /* expect zsetnum input keys to be given */ zsetnum = atoi(c->argv[2]->ptr); if (zsetnum < 1) { - addReplySds(c,sdsnew("-ERR at least 1 input key is needed for ZUNION/ZINTER\r\n")); + addReplySds(c,sdsnew("-ERR at least 1 input key is needed for ZUNIONSTORE/ZINTERSTORE\r\n")); return; } @@ -6067,11 +6067,11 @@ static void zunionInterGenericCommand(redisClient *c, robj *dstkey, int op) { zfree(src); } -static void zunionCommand(redisClient *c) { +static void zunionstoreCommand(redisClient *c) { zunionInterGenericCommand(c,c->argv[1], REDIS_OP_UNION); } -static void zinterCommand(redisClient *c) { +static void zinterstoreCommand(redisClient *c) { zunionInterGenericCommand(c,c->argv[1], REDIS_OP_INTER); } @@ -9699,7 +9699,7 @@ static void waitForMultipleSwappedKeys(redisClient *c, struct redisCommand *cmd, } } -/* Preload keys needed for the ZUNION and ZINTER commands. +/* Preload keys needed for the ZUNIONSTORE and ZINTERSTORE commands. * Note that the number of keys to preload is user-defined, so we need to * apply a sanity check against argc. */ static void zunionInterBlockClientOnSwappedKeys(redisClient *c, struct redisCommand *cmd, int argc, robj **argv) { diff --git a/staticsymbols.h b/staticsymbols.h index aff7b3a3..96f200d5 100644 --- a/staticsymbols.h +++ b/staticsymbols.h @@ -26,9 +26,12 @@ static struct redisFunctionSym symsTable[] = { {"brpopCommand",(unsigned long)brpopCommand}, {"bytesToHuman",(unsigned long)bytesToHuman}, {"call",(unsigned long)call}, +{"catAppendOnlyExpireAtCommand",(unsigned long)catAppendOnlyExpireAtCommand}, +{"catAppendOnlyGenericCommand",(unsigned long)catAppendOnlyGenericCommand}, {"checkType",(unsigned long)checkType}, {"closeTimedoutClients",(unsigned long)closeTimedoutClients}, {"compareStringObjects",(unsigned long)compareStringObjects}, +{"computeDatasetDigest",(unsigned long)computeDatasetDigest}, {"computeObjectSwappability",(unsigned long)computeObjectSwappability}, {"configCommand",(unsigned long)configCommand}, {"configGetCommand",(unsigned long)configGetCommand}, @@ -64,6 +67,8 @@ static struct redisFunctionSym symsTable[] = { {"dupClientReplyValue",(unsigned long)dupClientReplyValue}, {"dupStringObject",(unsigned long)dupStringObject}, {"echoCommand",(unsigned long)echoCommand}, +{"equalStringObjects",(unsigned long)equalStringObjects}, +{"execBlockClientOnSwappedKeys",(unsigned long)execBlockClientOnSwappedKeys}, {"execCommand",(unsigned long)execCommand}, {"execCommandReplicateMulti",(unsigned long)execCommandReplicateMulti}, {"existsCommand",(unsigned long)existsCommand}, @@ -148,6 +153,7 @@ static struct redisFunctionSym symsTable[] = { {"lindexCommand",(unsigned long)lindexCommand}, {"listMatchObjects",(unsigned long)listMatchObjects}, {"listMatchPubsubPattern",(unsigned long)listMatchPubsubPattern}, +{"ll2string",(unsigned long)ll2string}, {"llenCommand",(unsigned long)llenCommand}, {"loadServerConfig",(unsigned long)loadServerConfig}, {"lockThreadedIO",(unsigned long)lockThreadedIO}, @@ -164,6 +170,8 @@ static struct redisFunctionSym symsTable[] = { {"lsetCommand",(unsigned long)lsetCommand}, {"ltrimCommand",(unsigned long)ltrimCommand}, {"mgetCommand",(unsigned long)mgetCommand}, +{"mixDigest",(unsigned long)mixDigest}, +{"mixObjectDigest",(unsigned long)mixObjectDigest}, {"monitorCommand",(unsigned long)monitorCommand}, {"moveCommand",(unsigned long)moveCommand}, {"msetCommand",(unsigned long)msetCommand}, @@ -191,8 +199,11 @@ static struct redisFunctionSym symsTable[] = { {"queueIOJob",(unsigned long)queueIOJob}, {"queueMultiCommand",(unsigned long)queueMultiCommand}, {"randomkeyCommand",(unsigned long)randomkeyCommand}, +{"rdbEncodeInteger",(unsigned long)rdbEncodeInteger}, +{"rdbGenericLoadStringObject",(unsigned long)rdbGenericLoadStringObject}, {"rdbLoad",(unsigned long)rdbLoad}, {"rdbLoadDoubleValue",(unsigned long)rdbLoadDoubleValue}, +{"rdbLoadEncodedStringObject",(unsigned long)rdbLoadEncodedStringObject}, {"rdbLoadIntegerObject",(unsigned long)rdbLoadIntegerObject}, {"rdbLoadLen",(unsigned long)rdbLoadLen}, {"rdbLoadLzfStringObject",(unsigned long)rdbLoadLzfStringObject}, @@ -309,14 +320,17 @@ static struct redisFunctionSym symsTable[] = { {"vmThreadedIOCompletedJob",(unsigned long)vmThreadedIOCompletedJob}, {"vmWriteObjectOnSwap",(unsigned long)vmWriteObjectOnSwap}, {"waitEmptyIOJobsQueue",(unsigned long)waitEmptyIOJobsQueue}, +{"waitForMultipleSwappedKeys",(unsigned long)waitForMultipleSwappedKeys}, {"waitForSwappedKey",(unsigned long)waitForSwappedKey}, +{"xorDigest",(unsigned long)xorDigest}, +{"xorObjectDigest",(unsigned long)xorObjectDigest}, {"yesnotoi",(unsigned long)yesnotoi}, {"zaddCommand",(unsigned long)zaddCommand}, {"zaddGenericCommand",(unsigned long)zaddGenericCommand}, {"zcardCommand",(unsigned long)zcardCommand}, {"zcountCommand",(unsigned long)zcountCommand}, {"zincrbyCommand",(unsigned long)zincrbyCommand}, -{"zinterCommand",(unsigned long)zinterCommand}, +{"zinterstoreCommand",(unsigned long)zinterstoreCommand}, {"zrangeCommand",(unsigned long)zrangeCommand}, {"zrangeGenericCommand",(unsigned long)zrangeGenericCommand}, {"zrangebyscoreCommand",(unsigned long)zrangebyscoreCommand}, @@ -336,8 +350,8 @@ static struct redisFunctionSym symsTable[] = { {"zslFreeNode",(unsigned long)zslFreeNode}, {"zslInsert",(unsigned long)zslInsert}, {"zslRandomLevel",(unsigned long)zslRandomLevel}, -{"zunionCommand",(unsigned long)zunionCommand}, {"zunionInterBlockClientOnSwappedKeys",(unsigned long)zunionInterBlockClientOnSwappedKeys}, {"zunionInterGenericCommand",(unsigned long)zunionInterGenericCommand}, +{"zunionstoreCommand",(unsigned long)zunionstoreCommand}, {NULL,0} }; diff --git a/tests/unit/type/zset.tcl b/tests/unit/type/zset.tcl index cbadb198..107bd6b1 100644 --- a/tests/unit/type/zset.tcl +++ b/tests/unit/type/zset.tcl @@ -296,12 +296,12 @@ start_server default.conf {} { list [r zremrangebyrank zset 1 3] [r zrange zset 0 -1] } {3 {a e}} - test {ZUNION against non-existing key doesn't set destination} { + test {ZUNIONSTORE against non-existing key doesn't set destination} { r del zseta - list [r zunion dst_key 1 zseta] [r exists dst_key] + list [r zunionstore dst_key 1 zseta] [r exists dst_key] } {0 0} - test {ZUNION basics} { + test {ZUNIONSTORE basics} { r del zseta zsetb zsetc r zadd zseta 1 a r zadd zseta 2 b @@ -309,35 +309,35 @@ start_server default.conf {} { r zadd zsetb 1 b r zadd zsetb 2 c r zadd zsetb 3 d - list [r zunion zsetc 2 zseta zsetb] [r zrange zsetc 0 -1 withscores] + list [r zunionstore zsetc 2 zseta zsetb] [r zrange zsetc 0 -1 withscores] } {4 {a 1 b 3 d 3 c 5}} - test {ZUNION with weights} { - list [r zunion zsetc 2 zseta zsetb weights 2 3] [r zrange zsetc 0 -1 withscores] + test {ZUNIONSTORE with weights} { + list [r zunionstore zsetc 2 zseta zsetb weights 2 3] [r zrange zsetc 0 -1 withscores] } {4 {a 2 b 7 d 9 c 12}} - test {ZUNION with AGGREGATE MIN} { - list [r zunion zsetc 2 zseta zsetb aggregate min] [r zrange zsetc 0 -1 withscores] + test {ZUNIONSTORE with AGGREGATE MIN} { + list [r zunionstore zsetc 2 zseta zsetb aggregate min] [r zrange zsetc 0 -1 withscores] } {4 {a 1 b 1 c 2 d 3}} - test {ZUNION with AGGREGATE MAX} { - list [r zunion zsetc 2 zseta zsetb aggregate max] [r zrange zsetc 0 -1 withscores] + test {ZUNIONSTORE with AGGREGATE MAX} { + list [r zunionstore zsetc 2 zseta zsetb aggregate max] [r zrange zsetc 0 -1 withscores] } {4 {a 1 b 2 c 3 d 3}} - test {ZINTER basics} { - list [r zinter zsetc 2 zseta zsetb] [r zrange zsetc 0 -1 withscores] + test {ZINTERSTORE basics} { + list [r zinterstore zsetc 2 zseta zsetb] [r zrange zsetc 0 -1 withscores] } {2 {b 3 c 5}} - test {ZINTER with weights} { - list [r zinter zsetc 2 zseta zsetb weights 2 3] [r zrange zsetc 0 -1 withscores] + test {ZINTERSTORE with weights} { + list [r zinterstore zsetc 2 zseta zsetb weights 2 3] [r zrange zsetc 0 -1 withscores] } {2 {b 7 c 12}} - test {ZINTER with AGGREGATE MIN} { - list [r zinter zsetc 2 zseta zsetb aggregate min] [r zrange zsetc 0 -1 withscores] + test {ZINTERSTORE with AGGREGATE MIN} { + list [r zinterstore zsetc 2 zseta zsetb aggregate min] [r zrange zsetc 0 -1 withscores] } {2 {b 1 c 2}} - test {ZINTER with AGGREGATE MAX} { - list [r zinter zsetc 2 zseta zsetb aggregate max] [r zrange zsetc 0 -1 withscores] + test {ZINTERSTORE with AGGREGATE MAX} { + list [r zinterstore zsetc 2 zseta zsetb aggregate max] [r zrange zsetc 0 -1 withscores] } {2 {b 2 c 3}} test {ZSETs skiplist implementation backlink consistency test} {