fluencelabs/redis
1
0
Fork 0
mirror of https://github.com/fluencelabs/redis synced 2025-05-04 15:02:13 +00:00
Code Issues Projects Releases Wiki Activity

Merge branch 'unstable' of github.com:antirez/redis into unstable

Browse Source
This commit is contained in:
Guy Korland 2019-10-20 09:59:23 +03:00
commit 1ac30300f0
163 changed files with 12184 additions and 2597 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
CONTRIBUTING
View File

@ -14,9 +14,7 @@ each source file that you contribute.
PLEASE DO NOT POST GENERAL QUESTIONS that are not about bugs or suspected
bugs in the Github issues system. We'll be very happy to help you and provide
all the support at the Reddit sub:
http://reddit.com/r/redis
all the support in the mailing list.
There is also an active community of Redis users at Stack Overflow:
@ -24,7 +22,12 @@ each source file that you contribute.
# How to provide a patch for a new feature
1. If it is a major feature or a semantical change, please post it as a new submission in r/redis on Reddit at http://reddit.com/r/redis. Try to be passionate about why the feature is needed, make users upvote your proposal to gain traction and so forth. Read feedbacks about the community. But in this first step **please don't write code yet**.
1. If it is a major feature or a semantical change, please don't start coding
straight away: if your feature is not a conceptual fit you'll lose a lot of
time writing the code without any reason. Start by posting in the mailing list
and creating an issue at Github with the description of, exactly, what you want
to accomplish and why. Use cases are important for features to be accepted.
Here you'll see if there is consensus about your idea.
2. If in step 1 you get an acknowledgment from the project leaders, use the
following procedure to submit a patch:
@ -35,6 +38,13 @@ each source file that you contribute.
d. Initiate a pull request on github ( https://help.github.com/articles/creating-a-pull-request/ )
e. Done :)
For minor fixes just open a pull request on Github.
3. Keep in mind that we are very overloaded, so issues and PRs sometimes wait
for a *very* long time. However this is not lack of interest, as the project
gets more and more users, we find ourselves in a constant need to prioritize
certain issues/PRs over others. If you think your issue/PR is very important
try to popularize it, have other users commenting and sharing their point of
view and so forth. This helps.
4. For minor fixes just open a pull request on Github.
Thanks!

53
MANIFESTO
View File

@ -34,7 +34,21 @@ Redis Manifesto
so that the complexity is obvious and more complex operations can be
performed as the sum of the basic operations.
4 - Code is like a poem; it's not just something we write to reach some
4 - We believe in code efficiency. Computers get faster and faster, yet we
believe that abusing computing capabilities is not wise: the amount of
operations you can do for a given amount of energy remains anyway a
significant parameter: it allows to do more with less computers and, at
the same time, having a smaller environmental impact. Similarly Redis is
able to "scale down" to smaller devices. It is perfectly usable in a
Raspberry Pi and other small ARM based computers. Faster code having
just the layers of abstractions that are really needed will also result,
often, in more predictable performances. We think likewise about memory
usage, one of the fundamental goals of the Redis project is to
incrementally build more and more memory efficient data structures, so that
problems that were not approachable in RAM in the past will be perfectly
fine to handle in the future.
5 - Code is like a poem; it's not just something we write to reach some
practical result. Sometimes people that are far from the Redis philosophy
suggest using other code written by other authors (frequently in other
languages) in order to implement something Redis currently lacks. But to us
@ -45,23 +59,48 @@ Redis Manifesto
when needed. At the same time, when writing the Redis story we're trying to
write smaller stories that will fit in to other code.
5 - We're against complexity. We believe designing systems is a fight against
6 - We're against complexity. We believe designing systems is a fight against
complexity. We'll accept to fight the complexity when it's worthwhile but
we'll try hard to recognize when a small feature is not worth 1000s of lines
of code. Most of the time the best way to fight complexity is by not
creating it at all.
creating it at all. Complexity is also a form of lock-in: code that is
very hard to understand cannot be modified by users in an independent way
regardless of the license. One of the main Redis goals is to remain
understandable, enough for a single programmer to have a clear idea of how
it works in detail just reading the source code for a couple of weeks.
6 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits
7 - Threading is not a silver bullet. Instead of making Redis threaded we
believe on the idea of an efficient (mostly) single threaded Redis core.
Multiple of such cores, that may run in the same computer or may run
in multiple computers, are abstracted away as a single big system by
higher order protocols and features: Redis Cluster and the upcoming
Redis Proxy are our main goals. A shared nothing approach is not just
much simpler (see the previous point in this document), is also optimal
in NUMA systems. In the specific case of Redis it allows for each instance
to have a more limited amount of data, making the Redis persist-by-fork
approach more sounding. In the future we may explore parallelism only for
I/O, which is the low hanging fruit: minimal complexity could provide an
improved single process experience.
8 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits
naturally into a distributed version of Redis and 2) a more complex API that
supports multi-key operations. Both are useful if used judiciously but
there's no way to make the more complex multi-keys API distributed in an
opaque way without violating our other principles. We don't want to provide
the illusion of something that will work magically when actually it can't in
all cases. Instead we'll provide commands to quickly migrate keys from one
instance to another to perform multi-key operations and expose the tradeoffs
to the user.
instance to another to perform multi-key operations and expose the
trade-offs to the user.
7 - We optimize for joy. We believe writing code is a lot of hard work, and the
9 - We optimize for joy. We believe writing code is a lot of hard work, and the
only way it can be worth is by enjoying it. When there is no longer joy in
writing code, the best thing to do is stop. To prevent this, we'll avoid
taking paths that will make Redis less of a joy to develop.
10 - All the above points are put together in what we call opportunistic
programming: trying to get the most for the user with minimal increases
in complexity (hanging fruits). Solve 95% of the problem with 5% of the
code when it is acceptable. Avoid a fixed schedule but follow the flow of
user requests, inspiration, Redis internal readiness for certain features
(sometimes many past changes reach a critical point making a previously
complex feature very easy to obtain).

2
README.md
View File

@ -406,7 +406,7 @@ replicas, or to continue the replication after a disconnection.
Other C files
---
* `t_hash.c`, `t_list.c`, `t_set.c`, `t_string.c` and `t_zset.c` contains the implementation of the Redis data types. They implement both an API to access a given data type, and the client commands implementations for these data types.
* `t_hash.c`, `t_list.c`, `t_set.c`, `t_string.c`, `t_zset.c` and `t_stream.c` contains the implementation of the Redis data types. They implement both an API to access a given data type, and the client commands implementations for these data types.
* `ae.c` implements the Redis event loop, it's a self contained library which is simple to read and understand.
* `sds.c` is the Redis string library, check http://github.com/antirez/sds for more information.
* `anet.c` is a library to use POSIX networking in a simpler way compared to the raw interface exposed by the kernel.

106
TLS.md Normal file
View File

@ -0,0 +1,106 @@
TLS Support -- Work In Progress
===============================
This is a brief note to capture current thoughts/ideas and track pending action
items.
Getting Started
---------------
### Building
To build with TLS support you'll need OpenSSL development libraries (e.g.
libssl-dev on Debian/Ubuntu).
Run `make BUILD_TLS=yes`.
### Tests
To run Redis test suite with TLS, you'll need TLS support for TCL (i.e.
`tcl-tls` package on Debian/Ubuntu).
1. Run `./utils/gen-test-certs.sh` to generate a root CA and a server
certificate.
2. Run `./runtest --tls` or `./runtest-cluster --tls` to run Redis and Redis
Cluster tests in TLS mode.
### Running manually
To manually run a Redis server with TLS mode (assuming `gen-test-certs.sh` was
invoked so sample certificates/keys are available):
./src/redis-server --tls-port 6379 --port 0 \
--tls-cert-file ./tests/tls/redis.crt \
--tls-key-file ./tests/tls/redis.key \
--tls-ca-cert-file ./tests/tls/ca.crt
To connect to this Redis server with `redis-cli`:
./src/redis-cli --tls \
--cert ./tests/tls/redis.crt \
--key ./tests/tls/redis.key \
--cacert ./tests/tls/ca.crt
This will disable TCP and enable TLS on port 6379. It's also possible to have
both TCP and TLS available, but you'll need to assign different ports.
To make a Replica connect to the master using TLS, use `--tls-replication yes`,
and to make Redis Cluster use TLS across nodes use `--tls-cluster yes`.
Connections
-----------
All socket operations now go through a connection abstraction layer that hides
I/O and read/write event handling from the caller.
**Multi-threading I/O is not currently supported for TLS**, as a TLS connection
needs to do its own manipulation of AE events which is not thread safe. The
solution is probably to manage independent AE loops for I/O threads and longer
term association of connections with threads. This may potentially improve
overall performance as well.
Sync IO for TLS is currently implemented in a hackish way, i.e. making the
socket blocking and configuring socket-level timeout. This means the timeout
value may not be so accurate, and there would be a lot of syscall overhead.
However I believe that getting rid of syncio completely in favor of pure async
work is probably a better move than trying to fix that. For replication it would
probably not be so hard. For cluster keys migration it might be more difficult,
but there are probably other good reasons to improve that part anyway.
To-Do List
==========
Additional TLS Features
-----------------------
1. Add metrics to INFO?
2. Add session caching support. Check if/how it's handled by clients to assess
how useful/important it is.
redis-benchmark
---------------
The current implementation is a mix of using hiredis for parsing and basic
networking (establishing connections), but directly manipulating sockets for
most actions.
This will need to be cleaned up for proper TLS support. The best approach is
probably to migrate to hiredis async mode.
redis-cli
---------
1. Add support for TLS in --slave and --rdb modes.
Others
------
Consider the implications of allowing TLS to be configured on a separate port,
making Redis listening on multiple ports.
This impacts many things, like
1. Startup banner port notification
2. Proctitle
3. How slaves announce themselves
4. Cluster bus port calculation

6
deps/Makefile vendored
View File

@ -41,9 +41,13 @@ distclean:
.PHONY: distclean
ifeq ($(BUILD_TLS),yes)
HIREDIS_MAKE_FLAGS = USE_SSL=1
endif
hiredis: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd hiredis && $(MAKE) static
cd hiredis && $(MAKE) static $(HIREDIS_MAKE_FLAGS)
.PHONY: hiredis

1
deps/hiredis/.gitignore vendored
View File

@ -5,3 +5,4 @@
/*.dylib
/*.a
/*.pc
*.dSYM

74
deps/hiredis/.travis.yml vendored
View File

@ -26,20 +26,72 @@ addons:
- libc6-dev-i386
- libc6-dbg:i386
- gcc-multilib
- g++-multilib
- valgrind
env:
- CFLAGS="-Werror"
- PRE="valgrind --track-origins=yes --leak-check=full"
- TARGET="32bit" TARGET_VARS="32bit-vars" CFLAGS="-Werror"
- TARGET="32bit" TARGET_VARS="32bit-vars" PRE="valgrind --track-origins=yes --leak-check=full"
- BITS="32"
- BITS="64"
script:
- EXTRA_CMAKE_OPTS="-DENABLE_EXAMPLES:BOOL=ON -DHIREDIS_SSL:BOOL=ON";
if [ "$TRAVIS_OS_NAME" == "osx" ]; then
if [ "$BITS" == "32" ]; then
CFLAGS="-m32 -Werror";
CXXFLAGS="-m32 -Werror";
LDFLAGS="-m32";
EXTRA_CMAKE_OPTS=;
else
CFLAGS="-Werror";
CXXFLAGS="-Werror";
fi;
else
TEST_PREFIX="valgrind --track-origins=yes --leak-check=full";
if [ "$BITS" == "32" ]; then
CFLAGS="-m32 -Werror";
CXXFLAGS="-m32 -Werror";
LDFLAGS="-m32";
EXTRA_CMAKE_OPTS=;
else
CFLAGS="-Werror";
CXXFLAGS="-Werror";
fi;
fi;
export CFLAGS CXXFLAGS LDFLAGS TEST_PREFIX EXTRA_CMAKE_OPTS
- mkdir build/ && cd build/
- cmake .. ${EXTRA_CMAKE_OPTS}
- make VERBOSE=1
- ctest -V
matrix:
exclude:
- os: osx
env: PRE="valgrind --track-origins=yes --leak-check=full"
include:
# Windows MinGW cross compile on Linux
- os: linux
dist: xenial
compiler: mingw
addons:
apt:
packages:
- ninja-build
- gcc-mingw-w64-x86-64
- g++-mingw-w64-x86-64
script:
- mkdir build && cd build
- CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_WITH_INSTALL_RPATH=on
- ninja -v
- os: osx
env: TARGET="32bit" TARGET_VARS="32bit-vars" PRE="valgrind --track-origins=yes --leak-check=full"
script: make $TARGET CFLAGS="$CFLAGS" && make check PRE="$PRE" && make $TARGET_VARS hiredis-example
# Windows MSVC 2017
- os: windows
compiler: msvc
env:
- MATRIX_EVAL="CC=cl.exe && CXX=cl.exe"
before_install:
- eval "${MATRIX_EVAL}"
install:
- choco install ninja
script:
- mkdir build && cd build
- cmd.exe /C '"C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" amd64 &&
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release &&
ninja -v'
- ctest -V

15
deps/hiredis/CHANGELOG.md vendored
View File

@ -12,6 +12,16 @@
compare to other values, casting might be necessary or can be removed, if
casting was applied before.
### 0.x.x (unreleased)
**BREAKING CHANGES**:
* Change `redisReply.len` to `size_t`, as it denotes the the size of a string
User code should compare this to `size_t` values as well.
If it was used to compare to other values, casting might be necessary or can be removed, if casting was applied before.
* `redisReplyObjectFunctions.createArray` now takes `size_t` for its length parameter.
### 0.14.0 (2018-09-25)
* Make string2ll static to fix conflict with Redis (Tom Lee [c3188b])
@ -50,8 +60,9 @@
* Import latest upstream sds. This breaks applications that are linked against the old hiredis v0.13
* Fix warnings, when compiled with -Wshadow
* Make hiredis compile in Cygwin on Windows, now CI-tested
**BREAKING CHANGES**:
* Bulk and multi-bulk lengths less than -1 or greater than `LLONG_MAX` are now
protocol errors. This is consistent with the RESP specification. On 32-bit
platforms, the upper bound is lowered to `SIZE_MAX`.
* Remove backwards compatibility macro's

90
deps/hiredis/CMakeLists.txt vendored Normal file
View File

@ -0,0 +1,90 @@
CMAKE_MINIMUM_REQUIRED(VERSION 3.4.0)
INCLUDE(GNUInstallDirs)
PROJECT(hiredis)
OPTION(ENABLE_SSL "Build hiredis_ssl for SSL support" OFF)
MACRO(getVersionBit name)
SET(VERSION_REGEX "^#define ${name} (.+)$")
FILE(STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/hiredis.h"
VERSION_BIT REGEX ${VERSION_REGEX})
STRING(REGEX REPLACE ${VERSION_REGEX} "\\1" ${name} "${VERSION_BIT}")
ENDMACRO(getVersionBit)
getVersionBit(HIREDIS_MAJOR)
getVersionBit(HIREDIS_MINOR)
getVersionBit(HIREDIS_PATCH)
getVersionBit(HIREDIS_SONAME)
SET(VERSION "${HIREDIS_MAJOR}.${HIREDIS_MINOR}.${HIREDIS_PATCH}")
MESSAGE("Detected version: ${VERSION}")
PROJECT(hiredis VERSION "${VERSION}")
SET(ENABLE_EXAMPLES OFF CACHE BOOL "Enable building hiredis examples")
ADD_LIBRARY(hiredis SHARED
async.c
dict.c
hiredis.c
net.c
read.c
sds.c
sockcompat.c)
SET_TARGET_PROPERTIES(hiredis
PROPERTIES
VERSION "${HIREDIS_SONAME}")
IF(WIN32 OR MINGW)
TARGET_LINK_LIBRARIES(hiredis PRIVATE ws2_32)
ENDIF()
TARGET_INCLUDE_DIRECTORIES(hiredis PUBLIC .)
CONFIGURE_FILE(hiredis.pc.in hiredis.pc @ONLY)
INSTALL(TARGETS hiredis
DESTINATION "${CMAKE_INSTALL_LIBDIR}")
INSTALL(FILES hiredis.h read.h sds.h async.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(DIRECTORY adapters
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
IF(ENABLE_SSL)
IF (NOT OPENSSL_ROOT_DIR)
IF (APPLE)
SET(OPENSSL_ROOT_DIR "/usr/local/opt/openssl")
ENDIF()
ENDIF()
FIND_PACKAGE(OpenSSL REQUIRED)
ADD_LIBRARY(hiredis_ssl SHARED
ssl.c)
TARGET_INCLUDE_DIRECTORIES(hiredis_ssl PRIVATE "${OPENSSL_INCLUDE_DIR}")
TARGET_LINK_LIBRARIES(hiredis_ssl PRIVATE ${OPENSSL_LIBRARIES})
CONFIGURE_FILE(hiredis_ssl.pc.in hiredis_ssl.pc @ONLY)
INSTALL(TARGETS hiredis_ssl
DESTINATION "${CMAKE_INSTALL_LIBDIR}")
INSTALL(FILES hiredis_ssl.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/hiredis)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/hiredis_ssl.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
ENDIF()
IF(NOT (WIN32 OR MINGW))
ENABLE_TESTING()
ADD_EXECUTABLE(hiredis-test test.c)
TARGET_LINK_LIBRARIES(hiredis-test hiredis)
ADD_TEST(NAME hiredis-test
COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/test.sh)
ENDIF()
# Add examples
IF(ENABLE_EXAMPLES)
ADD_SUBDIRECTORY(examples)
ENDIF(ENABLE_EXAMPLES)

106
deps/hiredis/Makefile vendored
View File

@ -3,11 +3,17 @@
# Copyright (C) 2010-2011 Pieter Noordhuis <pcnoordhuis at gmail dot com>
# This file is released under the BSD license, see the COPYING file
OBJ=net.o hiredis.o sds.o async.o read.o
OBJ=net.o hiredis.o sds.o async.o read.o sockcompat.o
SSL_OBJ=ssl.o
EXAMPLES=hiredis-example hiredis-example-libevent hiredis-example-libev hiredis-example-glib
ifeq ($(USE_SSL),1)
EXAMPLES+=hiredis-example-ssl hiredis-example-libevent-ssl
endif
TESTS=hiredis-test
LIBNAME=libhiredis
SSL_LIBNAME=libhiredis_ssl
PKGCONFNAME=hiredis.pc
SSL_PKGCONFNAME=hiredis_ssl.pc
HIREDIS_MAJOR=$(shell grep HIREDIS_MAJOR hiredis.h | awk '{print $$3}')
HIREDIS_MINOR=$(shell grep HIREDIS_MINOR hiredis.h | awk '{print $$3}')
@ -39,7 +45,7 @@ export REDIS_TEST_CONFIG
CC:=$(shell sh -c 'type $${CC%% *} >/dev/null 2>/dev/null && echo $(CC) || echo gcc')
CXX:=$(shell sh -c 'type $${CXX%% *} >/dev/null 2>/dev/null && echo $(CXX) || echo g++')
OPTIMIZATION?=-O3
WARNINGS=-Wall -W -Wstrict-prototypes -Wwrite-strings
WARNINGS=-Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers
DEBUG_FLAGS?= -g -ggdb
REAL_CFLAGS=$(OPTIMIZATION) -fPIC $(CPPFLAGS) $(CFLAGS) $(WARNINGS) $(DEBUG_FLAGS)
REAL_LDFLAGS=$(LDFLAGS)
@ -49,12 +55,30 @@ STLIBSUFFIX=a
DYLIB_MINOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_SONAME)
DYLIB_MAJOR_NAME=$(LIBNAME).$(DYLIBSUFFIX).$(HIREDIS_MAJOR)
DYLIBNAME=$(LIBNAME).$(DYLIBSUFFIX)
DYLIB_MAKE_CMD=$(CC) -shared -Wl,-soname,$(DYLIB_MINOR_NAME) -o $(DYLIBNAME) $(LDFLAGS)
SSL_DYLIBNAME=$(SSL_LIBNAME).$(DYLIBSUFFIX)
DYLIB_MAKE_CMD=$(CC) -shared -Wl,-soname,$(DYLIB_MINOR_NAME)
STLIBNAME=$(LIBNAME).$(STLIBSUFFIX)
STLIB_MAKE_CMD=ar rcs $(STLIBNAME)
SSL_STLIBNAME=$(SSL_LIBNAME).$(STLIBSUFFIX)
STLIB_MAKE_CMD=$(AR) rcs
# Platform-specific overrides
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
USE_SSL?=0
# This is required for test.c only
ifeq ($(USE_SSL),1)
CFLAGS+=-DHIREDIS_TEST_SSL
endif
ifeq ($(uname_S),Linux)
SSL_LDFLAGS=-lssl -lcrypto
else
OPENSSL_PREFIX?=/usr/local/opt/openssl
CFLAGS+=-I$(OPENSSL_PREFIX)/include
SSL_LDFLAGS+=-L$(OPENSSL_PREFIX)/lib -lssl -lcrypto
endif
ifeq ($(uname_S),SunOS)
REAL_LDFLAGS+= -ldl -lnsl -lsocket
DYLIB_MAKE_CMD=$(CC) -G -o $(DYLIBNAME) -h $(DYLIB_MINOR_NAME) $(LDFLAGS)
@ -66,40 +90,61 @@ ifeq ($(uname_S),Darwin)
endif
all: $(DYLIBNAME) $(STLIBNAME) hiredis-test $(PKGCONFNAME)
ifeq ($(USE_SSL),1)
all: $(SSL_DYLIBNAME) $(SSL_STLIBNAME) $(SSL_PKGCONFNAME)
endif
# Deps (use make dep to generate this)
async.o: async.c fmacros.h async.h hiredis.h read.h sds.h net.h dict.c dict.h
dict.o: dict.c fmacros.h dict.h
hiredis.o: hiredis.c fmacros.h hiredis.h read.h sds.h net.h
net.o: net.c fmacros.h net.h hiredis.h read.h sds.h
hiredis.o: hiredis.c fmacros.h hiredis.h read.h sds.h net.h win32.h
net.o: net.c fmacros.h net.h hiredis.h read.h sds.h sockcompat.h win32.h
read.o: read.c fmacros.h read.h sds.h
sds.o: sds.c sds.h
sockcompat.o: sockcompat.c sockcompat.h
ssl.o: ssl.c hiredis.h
test.o: test.c fmacros.h hiredis.h read.h sds.h
$(DYLIBNAME): $(OBJ)
$(DYLIB_MAKE_CMD) $(OBJ)
$(DYLIB_MAKE_CMD) -o $(DYLIBNAME) $(OBJ) $(REAL_LDFLAGS)
$(STLIBNAME): $(OBJ)
$(STLIB_MAKE_CMD) $(OBJ)
$(STLIB_MAKE_CMD) $(STLIBNAME) $(OBJ)
$(SSL_DYLIBNAME): $(SSL_OBJ)
$(DYLIB_MAKE_CMD) -o $(SSL_DYLIBNAME) $(SSL_OBJ) $(REAL_LDFLAGS) $(SSL_LDFLAGS)
$(SSL_STLIBNAME): $(SSL_OBJ)
$(STLIB_MAKE_CMD) $(SSL_STLIBNAME) $(SSL_OBJ)
dynamic: $(DYLIBNAME)
static: $(STLIBNAME)
ifeq ($(USE_SSL),1)
dynamic: $(SSL_DYLIBNAME)
static: $(SSL_STLIBNAME)
endif
# Binaries:
hiredis-example-libevent: examples/example-libevent.c adapters/libevent.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -levent $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -levent $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-libevent-ssl: examples/example-libevent-ssl.c adapters/libevent.h $(STLIBNAME) $(SSL_STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -levent $(STLIBNAME) $(SSL_STLIBNAME) $(REAL_LDFLAGS) $(SSL_LDFLAGS)
hiredis-example-libev: examples/example-libev.c adapters/libev.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -lev $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -lev $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-glib: examples/example-glib.c adapters/glib.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< $(shell pkg-config --cflags --libs glib-2.0) $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(shell pkg-config --cflags --libs glib-2.0) $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-ivykis: examples/example-ivykis.c adapters/ivykis.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -livykis $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -livykis $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-macosx: examples/example-macosx.c adapters/macosx.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< -framework CoreFoundation $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< -framework CoreFoundation $(STLIBNAME) $(REAL_LDFLAGS)
hiredis-example-ssl: examples/example-ssl.c $(STLIBNAME) $(SSL_STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(SSL_STLIBNAME) $(REAL_LDFLAGS) $(SSL_LDFLAGS)
ifndef AE_DIR
hiredis-example-ae:
@ -116,7 +161,7 @@ hiredis-example-libuv:
@false
else
hiredis-example-libuv: examples/example-libuv.c adapters/libuv.h $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. -I$(LIBUV_DIR)/include $< $(LIBUV_DIR)/.libs/libuv.a -lpthread -lrt $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. -I$(LIBUV_DIR)/include $< $(LIBUV_DIR)/.libs/libuv.a -lpthread -lrt $(STLIBNAME) $(REAL_LDFLAGS)
endif
ifeq ($(and $(QT_MOC),$(QT_INCLUDE_DIR),$(QT_LIBRARY_DIR)),)
@ -133,32 +178,33 @@ hiredis-example-qt: examples/example-qt.cpp adapters/qt.h $(STLIBNAME)
endif
hiredis-example: examples/example.c $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) $(REAL_LDFLAGS) -I. $< $(STLIBNAME)
$(CC) -o examples/$@ $(REAL_CFLAGS) -I. $< $(STLIBNAME) $(REAL_LDFLAGS)
examples: $(EXAMPLES)
hiredis-test: test.o $(STLIBNAME)
TEST_LIBS = $(STLIBNAME)
ifeq ($(USE_SSL),1)
TEST_LIBS += $(SSL_STLIBNAME) -lssl -lcrypto -lpthread
endif
hiredis-test: test.o $(TEST_LIBS)
hiredis-%: %.o $(STLIBNAME)
$(CC) $(REAL_CFLAGS) -o $@ $(REAL_LDFLAGS) $< $(STLIBNAME)
$(CC) $(REAL_CFLAGS) -o $@ $< $(TEST_LIBS) $(REAL_LDFLAGS)
test: hiredis-test
./hiredis-test
check: hiredis-test
@echo "$$REDIS_TEST_CONFIG" | $(REDIS_SERVER) -
$(PRE) ./hiredis-test -h 127.0.0.1 -p $(REDIS_PORT) -s /tmp/hiredis-test-redis.sock || \
( kill `cat /tmp/hiredis-test-redis.pid` && false )
kill `cat /tmp/hiredis-test-redis.pid`
TEST_SSL=$(USE_SSL) ./test.sh
.c.o:
$(CC) -std=c99 -pedantic -c $(REAL_CFLAGS) $<
clean:
rm -rf $(DYLIBNAME) $(STLIBNAME) $(TESTS) $(PKGCONFNAME) examples/hiredis-example* *.o *.gcda *.gcno *.gcov
rm -rf $(DYLIBNAME) $(STLIBNAME) $(SSL_DYLIBNAME) $(SSL_STLIBNAME) $(TESTS) $(PKGCONFNAME) examples/hiredis-example* *.o *.gcda *.gcno *.gcov
dep:
$(CC) -MM *.c
$(CC) $(CPPFLAGS) $(CFLAGS) -MM *.c
INSTALL?= cp -pPR
@ -175,6 +221,20 @@ $(PKGCONFNAME): hiredis.h
@echo Libs: -L\$${libdir} -lhiredis >> $@
@echo Cflags: -I\$${includedir} -D_FILE_OFFSET_BITS=64 >> $@
$(SSL_PKGCONFNAME): hiredis.h
@echo "Generating $@ for pkgconfig..."
@echo prefix=$(PREFIX) > $@
@echo exec_prefix=\$${prefix} >> $@
@echo libdir=$(PREFIX)/$(LIBRARY_PATH) >> $@
@echo includedir=$(PREFIX)/$(INCLUDE_PATH) >> $@
@echo >> $@
@echo Name: hiredis_ssl >> $@
@echo Description: SSL Support for hiredis. >> $@
@echo Version: $(HIREDIS_MAJOR).$(HIREDIS_MINOR).$(HIREDIS_PATCH) >> $@
@echo Requires: hiredis >> $@
@echo Libs: -L\$${libdir} -lhiredis_ssl >> $@
@echo Libs.private: -lssl -lcrypto >> $@
install: $(DYLIBNAME) $(STLIBNAME) $(PKGCONFNAME)
mkdir -p $(INSTALL_INCLUDE_PATH) $(INSTALL_INCLUDE_PATH)/adapters $(INSTALL_LIBRARY_PATH)
$(INSTALL) hiredis.h async.h read.h sds.h $(INSTALL_INCLUDE_PATH)

3
deps/hiredis/README.md vendored
View File

@ -286,6 +286,7 @@ return `REDIS_ERR`. The function to set the disconnect callback has the followin
```c
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
```
`ac->data` may be used to pass user data to this callback, the same can be done for redisConnectCallback.
### Sending commands and their callbacks
In an asynchronous context, commands are automatically pipelined due to the nature of an event loop.
@ -406,6 +407,6 @@ as soon as possible in order to prevent allocation of useless memory.
## AUTHORS
Hiredis was written by Salvatore Sanfilippo (antirez at gmail) and
Pieter Noordhuis (pcnoordhuis at gmail) and is released under the BSD license.
Pieter Noordhuis (pcnoordhuis at gmail) and is released under the BSD license.
Hiredis is currently maintained by Matt Stancliff (matt at genges dot com) and
Jan-Erik Rediger (janerik at fnordig dot com)

112
deps/hiredis/adapters/libevent.h vendored
View File

@ -34,48 +34,113 @@
#include "../hiredis.h"
#include "../async.h"
#define REDIS_LIBEVENT_DELETED 0x01
#define REDIS_LIBEVENT_ENTERED 0x02
typedef struct redisLibeventEvents {
redisAsyncContext *context;
struct event *rev, *wev;
struct event *ev;
struct event_base *base;
struct timeval tv;
short flags;
short state;
} redisLibeventEvents;
static void redisLibeventReadEvent(int fd, short event, void *arg) {
((void)fd); ((void)event);
redisLibeventEvents *e = (redisLibeventEvents*)arg;
redisAsyncHandleRead(e->context);
static void redisLibeventDestroy(redisLibeventEvents *e) {
free(e);
}
static void redisLibeventWriteEvent(int fd, short event, void *arg) {
((void)fd); ((void)event);
static void redisLibeventHandler(int fd, short event, void *arg) {
((void)fd);
redisLibeventEvents *e = (redisLibeventEvents*)arg;
redisAsyncHandleWrite(e->context);
e->state |= REDIS_LIBEVENT_ENTERED;
#define CHECK_DELETED() if (e->state & REDIS_LIBEVENT_DELETED) {\
redisLibeventDestroy(e);\
return; \
}
if ((event & EV_TIMEOUT) && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleTimeout(e->context);
CHECK_DELETED();
}
if ((event & EV_READ) && e->context && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleRead(e->context);
CHECK_DELETED();
}
if ((event & EV_WRITE) && e->context && (e->state & REDIS_LIBEVENT_DELETED) == 0) {
redisAsyncHandleWrite(e->context);
CHECK_DELETED();
}
e->state &= ~REDIS_LIBEVENT_ENTERED;
#undef CHECK_DELETED
}
static void redisLibeventUpdate(void *privdata, short flag, int isRemove) {
redisLibeventEvents *e = (redisLibeventEvents *)privdata;
const struct timeval *tv = e->tv.tv_sec || e->tv.tv_usec ? &e->tv : NULL;
if (isRemove) {
if ((e->flags & flag) == 0) {
return;
} else {
e->flags &= ~flag;
}
} else {
if (e->flags & flag) {
return;
} else {
e->flags |= flag;
}
}
event_del(e->ev);
event_assign(e->ev, e->base, e->context->c.fd, e->flags | EV_PERSIST,
redisLibeventHandler, privdata);
event_add(e->ev, tv);
}
static void redisLibeventAddRead(void *privdata) {
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_add(e->rev,NULL);
redisLibeventUpdate(privdata, EV_READ, 0);
}
static void redisLibeventDelRead(void *privdata) {
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_del(e->rev);
redisLibeventUpdate(privdata, EV_READ, 1);
}
static void redisLibeventAddWrite(void *privdata) {
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_add(e->wev,NULL);
redisLibeventUpdate(privdata, EV_WRITE, 0);
}
static void redisLibeventDelWrite(void *privdata) {
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_del(e->wev);
redisLibeventUpdate(privdata, EV_WRITE, 1);
}
static void redisLibeventCleanup(void *privdata) {
redisLibeventEvents *e = (redisLibeventEvents*)privdata;
event_free(e->rev);
event_free(e->wev);
free(e);
if (!e) {
return;
}
event_del(e->ev);
event_free(e->ev);
e->ev = NULL;
if (e->state & REDIS_LIBEVENT_ENTERED) {
e->state |= REDIS_LIBEVENT_DELETED;
} else {
redisLibeventDestroy(e);
}
}
static void redisLibeventSetTimeout(void *privdata, struct timeval tv) {
redisLibeventEvents *e = (redisLibeventEvents *)privdata;
short flags = e->flags;
e->flags = 0;
e->tv = tv;
redisLibeventUpdate(e, flags, 0);
}
static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
@ -87,7 +152,7 @@ static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisLibeventEvents*)malloc(sizeof(*e));
e = (redisLibeventEvents*)calloc(1, sizeof(*e));
e->context = ac;
/* Register functions to start/stop listening for events */
@ -96,13 +161,12 @@ static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
ac->ev.addWrite = redisLibeventAddWrite;
ac->ev.delWrite = redisLibeventDelWrite;
ac->ev.cleanup = redisLibeventCleanup;
ac->ev.scheduleTimer = redisLibeventSetTimeout;
ac->ev.data = e;
/* Initialize and install read/write events */
e->rev = event_new(base, c->fd, EV_READ, redisLibeventReadEvent, e);
e->wev = event_new(base, c->fd, EV_WRITE, redisLibeventWriteEvent, e);
event_add(e->rev, NULL);
event_add(e->wev, NULL);
e->ev = event_new(base, c->fd, EV_READ | EV_WRITE, redisLibeventHandler, e);
e->base = base;
return REDIS_OK;
}
#endif

7
deps/hiredis/appveyor.yml vendored
View File

@ -5,8 +5,9 @@ environment:
CC: gcc
- CYG_BASH: C:\cygwin\bin\bash
CC: gcc
TARGET: 32bit
TARGET_VARS: 32bit-vars
CFLAGS: -m32
CXXFLAGS: -m32
LDFLAGS: -m32
clone_depth: 1
@ -20,4 +21,4 @@ install:
build_script:
- 'echo building...'
- '%CYG_BASH% -lc "cd $APPVEYOR_BUILD_FOLDER; exec 0</dev/null; make LDFLAGS=$LDFLAGS CC=$CC $TARGET CFLAGS=$CFLAGS && make LDFLAGS=$LDFLAGS CC=$CC $TARGET_VARS hiredis-example"'
- '%CYG_BASH% -lc "cd $APPVEYOR_BUILD_FOLDER; exec 0</dev/null; mkdir build && cd build && cmake .. -G \"Unix Makefiles\" && make VERBOSE=1"'

176
deps/hiredis/async.c vendored
View File

@ -32,7 +32,9 @@
#include "fmacros.h"
#include <stdlib.h>
#include <string.h>
#ifndef _MSC_VER
#include <strings.h>
#endif
#include <assert.h>
#include <ctype.h>
#include <errno.h>
@ -40,22 +42,9 @@
#include "net.h"
#include "dict.c"
#include "sds.h"
#include "win32.h"
#define _EL_ADD_READ(ctx) do { \
if ((ctx)->ev.addRead) (ctx)->ev.addRead((ctx)->ev.data); \
} while(0)
#define _EL_DEL_READ(ctx) do { \
if ((ctx)->ev.delRead) (ctx)->ev.delRead((ctx)->ev.data); \
} while(0)
#define _EL_ADD_WRITE(ctx) do { \
if ((ctx)->ev.addWrite) (ctx)->ev.addWrite((ctx)->ev.data); \
} while(0)
#define _EL_DEL_WRITE(ctx) do { \
if ((ctx)->ev.delWrite) (ctx)->ev.delWrite((ctx)->ev.data); \
} while(0)
#define _EL_CLEANUP(ctx) do { \
if ((ctx)->ev.cleanup) (ctx)->ev.cleanup((ctx)->ev.data); \
} while(0);
#include "async_private.h"
/* Forward declaration of function in hiredis.c */
int __redisAppendCommand(redisContext *c, const char *cmd, size_t len);
@ -126,6 +115,7 @@ static redisAsyncContext *redisAsyncInitialize(redisContext *c) {
ac->ev.addWrite = NULL;
ac->ev.delWrite = NULL;
ac->ev.cleanup = NULL;
ac->ev.scheduleTimer = NULL;
ac->onConnect = NULL;
ac->onDisconnect = NULL;
@ -150,56 +140,52 @@ static void __redisAsyncCopyError(redisAsyncContext *ac) {
ac->errstr = c->errstr;
}
redisAsyncContext *redisAsyncConnect(const char *ip, int port) {
redisAsyncContext *redisAsyncConnectWithOptions(const redisOptions *options) {
redisOptions myOptions = *options;
redisContext *c;
redisAsyncContext *ac;
c = redisConnectNonBlock(ip,port);
if (c == NULL)
myOptions.options |= REDIS_OPT_NONBLOCK;
c = redisConnectWithOptions(&myOptions);
if (c == NULL) {
return NULL;
}
ac = redisAsyncInitialize(c);
if (ac == NULL) {
redisFree(c);
return NULL;
}
__redisAsyncCopyError(ac);
return ac;
}
redisAsyncContext *redisAsyncConnect(const char *ip, int port) {
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
return redisAsyncConnectWithOptions(&options);
}
redisAsyncContext *redisAsyncConnectBind(const char *ip, int port,
const char *source_addr) {
redisContext *c = redisConnectBindNonBlock(ip,port,source_addr);
redisAsyncContext *ac = redisAsyncInitialize(c);
__redisAsyncCopyError(ac);
return ac;
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
options.endpoint.tcp.source_addr = source_addr;
return redisAsyncConnectWithOptions(&options);
}
redisAsyncContext *redisAsyncConnectBindWithReuse(const char *ip, int port,
const char *source_addr) {
redisContext *c = redisConnectBindNonBlockWithReuse(ip,port,source_addr);
redisAsyncContext *ac = redisAsyncInitialize(c);
__redisAsyncCopyError(ac);
return ac;
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
options.options |= REDIS_OPT_REUSEADDR;
options.endpoint.tcp.source_addr = source_addr;
return redisAsyncConnectWithOptions(&options);
}
redisAsyncContext *redisAsyncConnectUnix(const char *path) {
redisContext *c;
redisAsyncContext *ac;
c = redisConnectUnixNonBlock(path);
if (c == NULL)
return NULL;
ac = redisAsyncInitialize(c);
if (ac == NULL) {
redisFree(c);
return NULL;
}
__redisAsyncCopyError(ac);
return ac;
redisOptions options = {0};
REDIS_OPTIONS_SET_UNIX(&options, path);
return redisAsyncConnectWithOptions(&options);
}
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn) {
@ -328,7 +314,7 @@ void redisAsyncFree(redisAsyncContext *ac) {
}
/* Helper function to make the disconnect happen and clean up. */
static void __redisAsyncDisconnect(redisAsyncContext *ac) {
void __redisAsyncDisconnect(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
/* Make sure error is accessible if there is any */
@ -344,9 +330,15 @@ static void __redisAsyncDisconnect(redisAsyncContext *ac) {
c->flags |= REDIS_DISCONNECTING;
}
/* cleanup event library on disconnect.
* this is safe to call multiple times */
_EL_CLEANUP(ac);
/* For non-clean disconnects, __redisAsyncFree() will execute pending
* callbacks with a NULL-reply. */
__redisAsyncFree(ac);
if (!(c->flags & REDIS_NO_AUTO_FREE)) {
__redisAsyncFree(ac);
}
}
/* Tries to do a clean disconnect from Redis, meaning it stops new commands
@ -358,6 +350,9 @@ static void __redisAsyncDisconnect(redisAsyncContext *ac) {
void redisAsyncDisconnect(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
c->flags |= REDIS_DISCONNECTING;
/** unset the auto-free flag here, because disconnect undoes this */
c->flags &= ~REDIS_NO_AUTO_FREE;
if (!(c->flags & REDIS_IN_CALLBACK) && ac->replies.head == NULL)
__redisAsyncDisconnect(ac);
}
@ -408,7 +403,7 @@ static int __redisGetSubscribeCallback(redisAsyncContext *ac, redisReply *reply,
assert(reply->element[2]->type == REDIS_REPLY_INTEGER);
/* Unset subscribed flag only when no pipelined pending subscribe. */
if (reply->element[2]->integer == 0
if (reply->element[2]->integer == 0
&& dictSize(ac->sub.channels) == 0
&& dictSize(ac->sub.patterns) == 0)
c->flags &= ~REDIS_SUBSCRIBED;
@ -524,6 +519,18 @@ static int __redisAsyncHandleConnect(redisAsyncContext *ac) {
}
}
void redisAsyncRead(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
if (redisBufferRead(c) == REDIS_ERR) {
__redisAsyncDisconnect(ac);
} else {
/* Always re-schedule reads */
_EL_ADD_READ(ac);
redisProcessCallbacks(ac);
}
}
/* This function should be called when the socket is readable.
* It processes all replies that can be read and executes their callbacks.
*/
@ -539,28 +546,13 @@ void redisAsyncHandleRead(redisAsyncContext *ac) {
return;
}
if (redisBufferRead(c) == REDIS_ERR) {
__redisAsyncDisconnect(ac);
} else {
/* Always re-schedule reads */
_EL_ADD_READ(ac);
redisProcessCallbacks(ac);
}
c->funcs->async_read(ac);
}
void redisAsyncHandleWrite(redisAsyncContext *ac) {
void redisAsyncWrite(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
int done = 0;
if (!(c->flags & REDIS_CONNECTED)) {
/* Abort connect was not successful. */
if (__redisAsyncHandleConnect(ac) != REDIS_OK)
return;
/* Try again later when the context is still not connected. */
if (!(c->flags & REDIS_CONNECTED))
return;
}
if (redisBufferWrite(c,&done) == REDIS_ERR) {
__redisAsyncDisconnect(ac);
} else {
@ -575,6 +567,51 @@ void redisAsyncHandleWrite(redisAsyncContext *ac) {
}
}
void redisAsyncHandleWrite(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
if (!(c->flags & REDIS_CONNECTED)) {
/* Abort connect was not successful. */
if (__redisAsyncHandleConnect(ac) != REDIS_OK)
return;
/* Try again later when the context is still not connected. */
if (!(c->flags & REDIS_CONNECTED))
return;
}
c->funcs->async_write(ac);
}
void __redisSetError(redisContext *c, int type, const char *str);
void redisAsyncHandleTimeout(redisAsyncContext *ac) {
redisContext *c = &(ac->c);
redisCallback cb;
if ((c->flags & REDIS_CONNECTED) && ac->replies.head == NULL) {
/* Nothing to do - just an idle timeout */
return;
}
if (!c->err) {
__redisSetError(c, REDIS_ERR_TIMEOUT, "Timeout");
}
if (!(c->flags & REDIS_CONNECTED) && ac->onConnect) {
ac->onConnect(ac, REDIS_ERR);
}
while (__redisShiftCallback(&ac->replies, &cb) == REDIS_OK) {
__redisRunCallback(ac, &cb, NULL);
}
/**
* TODO: Don't automatically sever the connection,
* rather, allow to ignore <x> responses before the queue is clear
*/
__redisAsyncDisconnect(ac);
}
/* Sets a pointer to the first argument and its length starting at p. Returns
* the number of bytes to skip to get to the following argument. */
static const char *nextArgument(const char *start, const char **str, size_t *len) {
@ -714,3 +751,16 @@ int redisAsyncFormattedCommand(redisAsyncContext *ac, redisCallbackFn *fn, void
int status = __redisAsyncCommand(ac,fn,privdata,cmd,len);
return status;
}
void redisAsyncSetTimeout(redisAsyncContext *ac, struct timeval tv) {
if (!ac->c.timeout) {
ac->c.timeout = calloc(1, sizeof(tv));
}
if (tv.tv_sec == ac->c.timeout->tv_sec &&
tv.tv_usec == ac->c.timeout->tv_usec) {
return;
}
*ac->c.timeout = tv;
}

8
deps/hiredis/async.h vendored
View File

@ -57,6 +57,7 @@ typedef struct redisCallbackList {
/* Connection callback prototypes */
typedef void (redisDisconnectCallback)(const struct redisAsyncContext*, int status);
typedef void (redisConnectCallback)(const struct redisAsyncContext*, int status);
typedef void(redisTimerCallback)(void *timer, void *privdata);
/* Context for an async connection to Redis */
typedef struct redisAsyncContext {
@ -81,6 +82,7 @@ typedef struct redisAsyncContext {
void (*addWrite)(void *privdata);
void (*delWrite)(void *privdata);
void (*cleanup)(void *privdata);
void (*scheduleTimer)(void *privdata, struct timeval tv);
} ev;
/* Called when either the connection is terminated due to an error or per
@ -106,6 +108,7 @@ typedef struct redisAsyncContext {
} redisAsyncContext;
/* Functions that proxy to hiredis */
redisAsyncContext *redisAsyncConnectWithOptions(const redisOptions *options);
redisAsyncContext *redisAsyncConnect(const char *ip, int port);
redisAsyncContext *redisAsyncConnectBind(const char *ip, int port, const char *source_addr);
redisAsyncContext *redisAsyncConnectBindWithReuse(const char *ip, int port,
@ -113,12 +116,17 @@ redisAsyncContext *redisAsyncConnectBindWithReuse(const char *ip, int port,
redisAsyncContext *redisAsyncConnectUnix(const char *path);
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
void redisAsyncSetTimeout(redisAsyncContext *ac, struct timeval tv);
void redisAsyncDisconnect(redisAsyncContext *ac);
void redisAsyncFree(redisAsyncContext *ac);
/* Handle read/write events */
void redisAsyncHandleRead(redisAsyncContext *ac);
void redisAsyncHandleWrite(redisAsyncContext *ac);
void redisAsyncHandleTimeout(redisAsyncContext *ac);
void redisAsyncRead(redisAsyncContext *ac);
void redisAsyncWrite(redisAsyncContext *ac);
/* Command functions for an async context. Write the command to the
* output buffer and register the provided callback. */

72
deps/hiredis/async_private.h vendored Normal file
View File

@ -0,0 +1,72 @@
/*
* Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __HIREDIS_ASYNC_PRIVATE_H
#define __HIREDIS_ASYNC_PRIVATE_H
#define _EL_ADD_READ(ctx) \
do { \
refreshTimeout(ctx); \
if ((ctx)->ev.addRead) (ctx)->ev.addRead((ctx)->ev.data); \
} while (0)
#define _EL_DEL_READ(ctx) do { \
if ((ctx)->ev.delRead) (ctx)->ev.delRead((ctx)->ev.data); \
} while(0)
#define _EL_ADD_WRITE(ctx) \
do { \
refreshTimeout(ctx); \
if ((ctx)->ev.addWrite) (ctx)->ev.addWrite((ctx)->ev.data); \
} while (0)
#define _EL_DEL_WRITE(ctx) do { \
if ((ctx)->ev.delWrite) (ctx)->ev.delWrite((ctx)->ev.data); \
} while(0)
#define _EL_CLEANUP(ctx) do { \
if ((ctx)->ev.cleanup) (ctx)->ev.cleanup((ctx)->ev.data); \
ctx->ev.cleanup = NULL; \
} while(0);
static inline void refreshTimeout(redisAsyncContext *ctx) {
if (ctx->c.timeout && ctx->ev.scheduleTimer &&
(ctx->c.timeout->tv_sec || ctx->c.timeout->tv_usec)) {
ctx->ev.scheduleTimer(ctx->ev.data, *ctx->c.timeout);
// } else {
// printf("Not scheduling timer.. (tmo=%p)\n", ctx->c.timeout);
// if (ctx->c.timeout){
// printf("tv_sec: %u. tv_usec: %u\n", ctx->c.timeout->tv_sec,
// ctx->c.timeout->tv_usec);
// }
}
}
void __redisAsyncDisconnect(redisAsyncContext *ac);
void redisProcessCallbacks(redisAsyncContext *ac);
#endif /* __HIREDIS_ASYNC_PRIVATE_H */

46
deps/hiredis/examples/CMakeLists.txt vendored Normal file
View File

@ -0,0 +1,46 @@
INCLUDE(FindPkgConfig)
# Check for GLib
PKG_CHECK_MODULES(GLIB2 glib-2.0)
if (GLIB2_FOUND)
INCLUDE_DIRECTORIES(${GLIB2_INCLUDE_DIRS})
LINK_DIRECTORIES(${GLIB2_LIBRARY_DIRS})
ADD_EXECUTABLE(example-glib example-glib.c)
TARGET_LINK_LIBRARIES(example-glib hiredis ${GLIB2_LIBRARIES})
ENDIF(GLIB2_FOUND)
FIND_PATH(LIBEV ev.h
HINTS /usr/local /usr/opt/local
ENV LIBEV_INCLUDE_DIR)
if (LIBEV)
# Just compile and link with libev
ADD_EXECUTABLE(example-libev example-libev.c)
TARGET_LINK_LIBRARIES(example-libev hiredis ev)
ENDIF()
FIND_PATH(LIBEVENT event.h)
if (LIBEVENT)
ADD_EXECUTABLE(example-libevent example-libevent)
TARGET_LINK_LIBRARIES(example-libevent hiredis event)
ENDIF()
FIND_PATH(LIBUV uv.h)
IF (LIBUV)
ADD_EXECUTABLE(example-libuv example-libuv.c)
TARGET_LINK_LIBRARIES(example-libuv hiredis uv)
ENDIF()
IF (APPLE)
FIND_LIBRARY(CF CoreFoundation)
ADD_EXECUTABLE(example-macosx example-macosx.c)
TARGET_LINK_LIBRARIES(example-macosx hiredis ${CF})
ENDIF()
IF (ENABLE_SSL)
ADD_EXECUTABLE(example-ssl example-ssl.c)
TARGET_LINK_LIBRARIES(example-ssl hiredis hiredis_ssl)
ENDIF()
ADD_EXECUTABLE(example example.c)
TARGET_LINK_LIBRARIES(example hiredis)

73
deps/hiredis/examples/example-libevent-ssl.c vendored Normal file
View File

@ -0,0 +1,73 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <hiredis.h>
#include <hiredis_ssl.h>
#include <async.h>
#include <adapters/libevent.h>
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) return;
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* Disconnect after receiving the reply to GET */
redisAsyncDisconnect(c);
}
void connectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Connected...\n");
}
void disconnectCallback(const redisAsyncContext *c, int status) {
if (status != REDIS_OK) {
printf("Error: %s\n", c->errstr);
return;
}
printf("Disconnected...\n");
}
int main (int argc, char **argv) {
signal(SIGPIPE, SIG_IGN);
struct event_base *base = event_base_new();
if (argc < 5) {
fprintf(stderr,
"Usage: %s <key> <host> <port> <cert> <certKey> [ca]\n", argv[0]);
exit(1);
}
const char *value = argv[1];
size_t nvalue = strlen(value);
const char *hostname = argv[2];
int port = atoi(argv[3]);
const char *cert = argv[4];
const char *certKey = argv[5];
const char *caCert = argc > 5 ? argv[6] : NULL;
redisAsyncContext *c = redisAsyncConnect(hostname, port);
if (c->err) {
/* Let *c leak for now... */
printf("Error: %s\n", c->errstr);
return 1;
}
if (redisSecureConnection(&c->c, caCert, cert, certKey, "sni") != REDIS_OK) {
printf("SSL Error!\n");
exit(1);
}
redisLibeventAttach(c,base);
redisAsyncSetConnectCallback(c,connectCallback);
redisAsyncSetDisconnectCallback(c,disconnectCallback);
redisAsyncCommand(c, NULL, NULL, "SET key %b", value, nvalue);
redisAsyncCommand(c, getCallback, (char*)"end-1", "GET key");
event_base_dispatch(base);
return 0;
}

15
deps/hiredis/examples/example-libevent.c vendored
View File

@ -9,7 +9,12 @@
void getCallback(redisAsyncContext *c, void *r, void *privdata) {
redisReply *reply = r;
if (reply == NULL) return;
if (reply == NULL) {
if (c->errstr) {
printf("errstr: %s\n", c->errstr);
}
return;
}
printf("argv[%s]: %s\n", (char*)privdata, reply->str);
/* Disconnect after receiving the reply to GET */
@ -35,8 +40,14 @@ void disconnectCallback(const redisAsyncContext *c, int status) {
int main (int argc, char **argv) {
signal(SIGPIPE, SIG_IGN);
struct event_base *base = event_base_new();
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
struct timeval tv = {0};
tv.tv_sec = 1;
options.timeout = &tv;
redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
redisAsyncContext *c = redisAsyncConnectWithOptions(&options);
if (c->err) {
/* Let *c leak for now... */
printf("Error: %s\n", c->errstr);

97
deps/hiredis/examples/example-ssl.c vendored Normal file
View File

@ -0,0 +1,97 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <hiredis.h>
#include <hiredis_ssl.h>
int main(int argc, char **argv) {
unsigned int j;
redisContext *c;
redisReply *reply;
if (argc < 4) {
printf("Usage: %s <host> <port> <cert> <key> [ca]\n", argv[0]);
exit(1);
}
const char *hostname = (argc > 1) ? argv[1] : "127.0.0.1";
int port = atoi(argv[2]);
const char *cert = argv[3];
const char *key = argv[4];
const char *ca = argc > 4 ? argv[5] : NULL;
struct timeval tv = { 1, 500000 }; // 1.5 seconds
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, hostname, port);
options.timeout = &tv;
c = redisConnectWithOptions(&options);
if (c == NULL || c->err) {
if (c) {
printf("Connection error: %s\n", c->errstr);
redisFree(c);
} else {
printf("Connection error: can't allocate redis context\n");
}
exit(1);
}
if (redisSecureConnection(c, ca, cert, key, "sni") != REDIS_OK) {
printf("Couldn't initialize SSL!\n");
printf("Error: %s\n", c->errstr);
redisFree(c);
exit(1);
}
/* PING server */
reply = redisCommand(c,"PING");
printf("PING: %s\n", reply->str);
freeReplyObject(reply);
/* Set a key */
reply = redisCommand(c,"SET %s %s", "foo", "hello world");
printf("SET: %s\n", reply->str);
freeReplyObject(reply);
/* Set a key using binary safe API */
reply = redisCommand(c,"SET %b %b", "bar", (size_t) 3, "hello", (size_t) 5);
printf("SET (binary API): %s\n", reply->str);
freeReplyObject(reply);
/* Try a GET and two INCR */
reply = redisCommand(c,"GET foo");
printf("GET foo: %s\n", reply->str);
freeReplyObject(reply);
reply = redisCommand(c,"INCR counter");
printf("INCR counter: %lld\n", reply->integer);
freeReplyObject(reply);
/* again ... */
reply = redisCommand(c,"INCR counter");
printf("INCR counter: %lld\n", reply->integer);
freeReplyObject(reply);
/* Create a list of numbers, from 0 to 9 */
reply = redisCommand(c,"DEL mylist");
freeReplyObject(reply);
for (j = 0; j < 10; j++) {
char buf[64];
snprintf(buf,64,"%u",j);
reply = redisCommand(c,"LPUSH mylist element-%s", buf);
freeReplyObject(reply);
}
/* Let's check what we have inside the list */
reply = redisCommand(c,"LRANGE mylist 0 -1");
if (reply->type == REDIS_REPLY_ARRAY) {
for (j = 0; j < reply->elements; j++) {
printf("%u) %s\n", j, reply->element[j]->str);
}
}
freeReplyObject(reply);
/* Disconnects and frees the context */
redisFree(c);
return 0;
}

17
deps/hiredis/examples/example.c vendored
View File

@ -5,14 +5,27 @@
#include <hiredis.h>
int main(int argc, char **argv) {
unsigned int j;
unsigned int j, isunix = 0;
redisContext *c;
redisReply *reply;
const char *hostname = (argc > 1) ? argv[1] : "127.0.0.1";
if (argc > 2) {
if (*argv[2] == 'u' || *argv[2] == 'U') {
isunix = 1;
/* in this case, host is the path to the unix socket */
printf("Will connect to unix socket @%s\n", hostname);
}
}
int port = (argc > 2) ? atoi(argv[2]) : 6379;
struct timeval timeout = { 1, 500000 }; // 1.5 seconds
c = redisConnectWithTimeout(hostname, port, timeout);
if (isunix) {
c = redisConnectUnixWithTimeout(hostname, timeout);
} else {
c = redisConnectWithTimeout(hostname, port, timeout);
}
if (c == NULL || c->err) {
if (c) {
printf("Connection error: %s\n", c->errstr);

256
deps/hiredis/hiredis.c vendored
View File

@ -34,7 +34,6 @@
#include "fmacros.h"
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <assert.h>
#include <errno.h>
#include <ctype.h>
@ -42,10 +41,20 @@
#include "hiredis.h"
#include "net.h"
#include "sds.h"
#include "async.h"
#include "win32.h"
static redisContextFuncs redisContextDefaultFuncs = {
.free_privdata = NULL,
.async_read = redisAsyncRead,
.async_write = redisAsyncWrite,
.read = redisNetRead,
.write = redisNetWrite
};
static redisReply *createReplyObject(int type);
static void *createStringObject(const redisReadTask *task, char *str, size_t len);
static void *createArrayObject(const redisReadTask *task, int elements);
static void *createArrayObject(const redisReadTask *task, size_t elements);
static void *createIntegerObject(const redisReadTask *task, long long value);
static void *createDoubleObject(const redisReadTask *task, double value, char *str, size_t len);
static void *createNilObject(const redisReadTask *task);
@ -112,21 +121,34 @@ static void *createStringObject(const redisReadTask *task, char *str, size_t len
if (r == NULL)
return NULL;
buf = malloc(len+1);
if (buf == NULL) {
freeReplyObject(r);
return NULL;
}
assert(task->type == REDIS_REPLY_ERROR ||
task->type == REDIS_REPLY_STATUS ||
task->type == REDIS_REPLY_STRING);
task->type == REDIS_REPLY_STRING ||
task->type == REDIS_REPLY_VERB);
/* Copy string value */
memcpy(buf,str,len);
buf[len] = '\0';
if (task->type == REDIS_REPLY_VERB) {
buf = malloc(len-4+1); /* Skip 4 bytes of verbatim type header. */
if (buf == NULL) {
freeReplyObject(r);
return NULL;
}
memcpy(r->vtype,str,3);
r->vtype[3] = '\0';
memcpy(buf,str+4,len-4);
buf[len-4] = '\0';
r->len = len-4;
} else {
buf = malloc(len+1);
if (buf == NULL) {
freeReplyObject(r);
return NULL;
}
memcpy(buf,str,len);
buf[len] = '\0';
r->len = len;
}
r->str = buf;
r->len = len;
if (task->parent) {
parent = task->parent->obj;
@ -138,7 +160,7 @@ static void *createStringObject(const redisReadTask *task, char *str, size_t len
return r;
}
static void *createArrayObject(const redisReadTask *task, int elements) {
static void *createArrayObject(const redisReadTask *task, size_t elements) {
redisReply *r, *parent;
r = createReplyObject(task->type);
@ -649,29 +671,30 @@ redisReader *redisReaderCreate(void) {
return redisReaderCreateWithFunctions(&defaultFunctions);
}
static redisContext *redisContextInit(void) {
static redisContext *redisContextInit(const redisOptions *options) {
redisContext *c;
c = calloc(1,sizeof(redisContext));
c = calloc(1, sizeof(*c));
if (c == NULL)
return NULL;
c->funcs = &redisContextDefaultFuncs;
c->obuf = sdsempty();
c->reader = redisReaderCreate();
c->fd = REDIS_INVALID_FD;
if (c->obuf == NULL || c->reader == NULL) {
redisFree(c);
return NULL;
}
(void)options; /* options are used in other functions */
return c;
}
void redisFree(redisContext *c) {
if (c == NULL)
return;
if (c->fd > 0)
close(c->fd);
redisNetClose(c);
sdsfree(c->obuf);
redisReaderFree(c->reader);
@ -680,12 +703,16 @@ void redisFree(redisContext *c) {
free(c->unix_sock.path);
free(c->timeout);
free(c->saddr);
if (c->funcs->free_privdata) {
c->funcs->free_privdata(c->privdata);
}
memset(c, 0xff, sizeof(*c));
free(c);
}
int redisFreeKeepFd(redisContext *c) {
int fd = c->fd;
c->fd = -1;
redisFD redisFreeKeepFd(redisContext *c) {
redisFD fd = c->fd;
c->fd = REDIS_INVALID_FD;
redisFree(c);
return fd;
}
@ -694,10 +721,13 @@ int redisReconnect(redisContext *c) {
c->err = 0;
memset(c->errstr, '\0', strlen(c->errstr));
if (c->fd > 0) {
close(c->fd);
if (c->privdata && c->funcs->free_privdata) {
c->funcs->free_privdata(c->privdata);
c->privdata = NULL;
}
redisNetClose(c);
sdsfree(c->obuf);
redisReaderFree(c->reader);
@ -718,112 +748,107 @@ int redisReconnect(redisContext *c) {
return REDIS_ERR;
}
redisContext *redisConnectWithOptions(const redisOptions *options) {
redisContext *c = redisContextInit(options);
if (c == NULL) {
return NULL;
}
if (!(options->options & REDIS_OPT_NONBLOCK)) {
c->flags |= REDIS_BLOCK;
}
if (options->options & REDIS_OPT_REUSEADDR) {
c->flags |= REDIS_REUSEADDR;
}
if (options->options & REDIS_OPT_NOAUTOFREE) {
c->flags |= REDIS_NO_AUTO_FREE;
}
if (options->type == REDIS_CONN_TCP) {
redisContextConnectBindTcp(c, options->endpoint.tcp.ip,
options->endpoint.tcp.port, options->timeout,
options->endpoint.tcp.source_addr);
} else if (options->type == REDIS_CONN_UNIX) {
redisContextConnectUnix(c, options->endpoint.unix_socket,
options->timeout);
} else if (options->type == REDIS_CONN_USERFD) {
c->fd = options->endpoint.fd;
c->flags |= REDIS_CONNECTED;
} else {
// Unknown type - FIXME - FREE
return NULL;
}
if (options->timeout != NULL && (c->flags & REDIS_BLOCK) && c->fd != REDIS_INVALID_FD) {
redisContextSetTimeout(c, *options->timeout);
}
return c;
}
/* Connect to a Redis instance. On error the field error in the returned
* context will be set to the return value of the error function.
* When no set of reply functions is given, the default set will be used. */
redisContext *redisConnect(const char *ip, int port) {
redisContext *c;
c = redisContextInit();
if (c == NULL)
return NULL;
c->flags |= REDIS_BLOCK;
redisContextConnectTcp(c,ip,port,NULL);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
return redisConnectWithOptions(&options);
}
redisContext *redisConnectWithTimeout(const char *ip, int port, const struct timeval tv) {
redisContext *c;
c = redisContextInit();
if (c == NULL)
return NULL;
c->flags |= REDIS_BLOCK;
redisContextConnectTcp(c,ip,port,&tv);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
options.timeout = &tv;
return redisConnectWithOptions(&options);
}
redisContext *redisConnectNonBlock(const char *ip, int port) {
redisContext *c;
c = redisContextInit();
if (c == NULL)
return NULL;
c->flags &= ~REDIS_BLOCK;
redisContextConnectTcp(c,ip,port,NULL);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
options.options |= REDIS_OPT_NONBLOCK;
return redisConnectWithOptions(&options);
}
redisContext *redisConnectBindNonBlock(const char *ip, int port,
const char *source_addr) {
redisContext *c = redisContextInit();
if (c == NULL)
return NULL;
c->flags &= ~REDIS_BLOCK;
redisContextConnectBindTcp(c,ip,port,NULL,source_addr);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
options.endpoint.tcp.source_addr = source_addr;
options.options |= REDIS_OPT_NONBLOCK;
return redisConnectWithOptions(&options);
}
redisContext *redisConnectBindNonBlockWithReuse(const char *ip, int port,
const char *source_addr) {
redisContext *c = redisContextInit();
if (c == NULL)
return NULL;
c->flags &= ~REDIS_BLOCK;
c->flags |= REDIS_REUSEADDR;
redisContextConnectBindTcp(c,ip,port,NULL,source_addr);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_TCP(&options, ip, port);
options.endpoint.tcp.source_addr = source_addr;
options.options |= REDIS_OPT_NONBLOCK|REDIS_OPT_REUSEADDR;
return redisConnectWithOptions(&options);
}
redisContext *redisConnectUnix(const char *path) {
redisContext *c;
c = redisContextInit();
if (c == NULL)
return NULL;
c->flags |= REDIS_BLOCK;
redisContextConnectUnix(c,path,NULL);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_UNIX(&options, path);
return redisConnectWithOptions(&options);
}
redisContext *redisConnectUnixWithTimeout(const char *path, const struct timeval tv) {
redisContext *c;
c = redisContextInit();
if (c == NULL)
return NULL;
c->flags |= REDIS_BLOCK;
redisContextConnectUnix(c,path,&tv);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_UNIX(&options, path);
options.timeout = &tv;
return redisConnectWithOptions(&options);
}
redisContext *redisConnectUnixNonBlock(const char *path) {
redisContext *c;
c = redisContextInit();
if (c == NULL)
return NULL;
c->flags &= ~REDIS_BLOCK;
redisContextConnectUnix(c,path,NULL);
return c;
redisOptions options = {0};
REDIS_OPTIONS_SET_UNIX(&options, path);
options.options |= REDIS_OPT_NONBLOCK;
return redisConnectWithOptions(&options);
}
redisContext *redisConnectFd(int fd) {
redisContext *c;
c = redisContextInit();
if (c == NULL)
return NULL;
c->fd = fd;
c->flags |= REDIS_BLOCK | REDIS_CONNECTED;
return c;
redisContext *redisConnectFd(redisFD fd) {
redisOptions options = {0};
options.type = REDIS_CONN_USERFD;
options.endpoint.fd = fd;
return redisConnectWithOptions(&options);
}
/* Set read/write timeout on a blocking socket. */
@ -853,22 +878,15 @@ int redisBufferRead(redisContext *c) {
if (c->err)
return REDIS_ERR;
nread = read(c->fd,buf,sizeof(buf));
if (nread == -1) {
if ((errno == EAGAIN && !(c->flags & REDIS_BLOCK)) || (errno == EINTR)) {
/* Try again later */
nread = c->funcs->read(c, buf, sizeof(buf));
if (nread > 0) {
if (redisReaderFeed(c->reader, buf, nread) != REDIS_OK) {
__redisSetError(c, c->reader->err, c->reader->errstr);
return REDIS_ERR;
} else {
__redisSetError(c,REDIS_ERR_IO,NULL);
return REDIS_ERR;
}
} else if (nread == 0) {
__redisSetError(c,REDIS_ERR_EOF,"Server closed the connection");
} else if (nread < 0) {
return REDIS_ERR;
} else {
if (redisReaderFeed(c->reader,buf,nread) != REDIS_OK) {
__redisSetError(c,c->reader->err,c->reader->errstr);
return REDIS_ERR;
}
}
return REDIS_OK;
}
@ -883,21 +901,15 @@ int redisBufferRead(redisContext *c) {
* c->errstr to hold the appropriate error string.
*/
int redisBufferWrite(redisContext *c, int *done) {
int nwritten;
/* Return early when the context has seen an error. */
if (c->err)
return REDIS_ERR;
if (sdslen(c->obuf) > 0) {
nwritten = write(c->fd,c->obuf,sdslen(c->obuf));
if (nwritten == -1) {
if ((errno == EAGAIN && !(c->flags & REDIS_BLOCK)) || (errno == EINTR)) {
/* Try again later */
} else {
__redisSetError(c,REDIS_ERR_IO,NULL);
return REDIS_ERR;
}
int nwritten = c->funcs->write(c);
if (nwritten < 0) {
return REDIS_ERR;
} else if (nwritten > 0) {
if (nwritten == (signed)sdslen(c->obuf)) {
sdsfree(c->obuf);

103
deps/hiredis/hiredis.h vendored
View File

@ -35,7 +35,11 @@
#define __HIREDIS_H
#include "read.h"
#include <stdarg.h> /* for va_list */
#ifndef _MSC_VER
#include <sys/time.h> /* for struct timeval */
#else
struct timeval; /* forward declaration */
#endif
#include <stdint.h> /* uintXX_t, etc */
#include "sds.h" /* for sds */
@ -74,6 +78,12 @@
/* Flag that is set when we should set SO_REUSEADDR before calling bind() */
#define REDIS_REUSEADDR 0x80
/**
* Flag that indicates the user does not want the context to
* be automatically freed upon error
*/
#define REDIS_NO_AUTO_FREE 0x200
#define REDIS_KEEPALIVE_INTERVAL 15 /* seconds */
/* number of times we retry to connect in the case of EADDRNOTAVAIL and
@ -92,6 +102,8 @@ typedef struct redisReply {
size_t len; /* Length of string */
char *str; /* Used for REDIS_REPLY_ERROR, REDIS_REPLY_STRING
and REDIS_REPLY_DOUBLE (in additionl to dval). */
char vtype[4]; /* Used for REDIS_REPLY_VERB, contains the null
terminated 3 character content type, such as "txt". */
size_t elements; /* number of elements, for REDIS_REPLY_ARRAY */
struct redisReply **element; /* elements vector for REDIS_REPLY_ARRAY */
} redisReply;
@ -111,14 +123,93 @@ void redisFreeSdsCommand(sds cmd);
enum redisConnectionType {
REDIS_CONN_TCP,
REDIS_CONN_UNIX
REDIS_CONN_UNIX,
REDIS_CONN_USERFD
};
struct redisSsl;
#define REDIS_OPT_NONBLOCK 0x01
#define REDIS_OPT_REUSEADDR 0x02
/**
* Don't automatically free the async object on a connection failure,
* or other implicit conditions. Only free on an explicit call to disconnect() or free()
*/
#define REDIS_OPT_NOAUTOFREE 0x04
/* In Unix systems a file descriptor is a regular signed int, with -1
* representing an invalid descriptor. In Windows it is a SOCKET
* (32- or 64-bit unsigned integer depending on the architecture), where
* all bits set (~0) is INVALID_SOCKET. */
#ifndef _WIN32
typedef int redisFD;
#define REDIS_INVALID_FD -1
#else
#ifdef _WIN64
typedef unsigned long long redisFD; /* SOCKET = 64-bit UINT_PTR */
#else
typedef unsigned long redisFD; /* SOCKET = 32-bit UINT_PTR */
#endif
#define REDIS_INVALID_FD ((redisFD)(~0)) /* INVALID_SOCKET */
#endif
typedef struct {
/*
* the type of connection to use. This also indicates which
* `endpoint` member field to use
*/
int type;
/* bit field of REDIS_OPT_xxx */
int options;
/* timeout value. if NULL, no timeout is used */
const struct timeval *timeout;
union {
/** use this field for tcp/ip connections */
struct {
const char *source_addr;
const char *ip;
int port;
} tcp;
/** use this field for unix domain sockets */
const char *unix_socket;
/**
* use this field to have hiredis operate an already-open
* file descriptor */
redisFD fd;
} endpoint;
} redisOptions;
/**
* Helper macros to initialize options to their specified fields.
*/
#define REDIS_OPTIONS_SET_TCP(opts, ip_, port_) \
(opts)->type = REDIS_CONN_TCP; \
(opts)->endpoint.tcp.ip = ip_; \
(opts)->endpoint.tcp.port = port_;
#define REDIS_OPTIONS_SET_UNIX(opts, path) \
(opts)->type = REDIS_CONN_UNIX; \
(opts)->endpoint.unix_socket = path;
struct redisAsyncContext;
struct redisContext;
typedef struct redisContextFuncs {
void (*free_privdata)(void *);
void (*async_read)(struct redisAsyncContext *);
void (*async_write)(struct redisAsyncContext *);
int (*read)(struct redisContext *, char *, size_t);
int (*write)(struct redisContext *);
} redisContextFuncs;
/* Context for a connection to Redis */
typedef struct redisContext {
const redisContextFuncs *funcs; /* Function table */
int err; /* Error flags, 0 when there is no error */
char errstr[128]; /* String representation of error when applicable */
int fd;
redisFD fd;
int flags;
char *obuf; /* Write buffer */
redisReader *reader; /* Protocol reader */
@ -139,8 +230,12 @@ typedef struct redisContext {
/* For non-blocking connect */
struct sockadr *saddr;
size_t addrlen;
/* Additional private data for hiredis addons such as SSL */
void *privdata;
} redisContext;
redisContext *redisConnectWithOptions(const redisOptions *options);
redisContext *redisConnect(const char *ip, int port);
redisContext *redisConnectWithTimeout(const char *ip, int port, const struct timeval tv);
redisContext *redisConnectNonBlock(const char *ip, int port);
@ -151,7 +246,7 @@ redisContext *redisConnectBindNonBlockWithReuse(const char *ip, int port,
redisContext *redisConnectUnix(const char *path);
redisContext *redisConnectUnixWithTimeout(const char *path, const struct timeval tv);
redisContext *redisConnectUnixNonBlock(const char *path);
redisContext *redisConnectFd(int fd);
redisContext *redisConnectFd(redisFD fd);
/**
* Reconnect the given context using the saved information.
@ -167,7 +262,7 @@ int redisReconnect(redisContext *c);
int redisSetTimeout(redisContext *c, const struct timeval tv);
int redisEnableKeepAlive(redisContext *c);
void redisFree(redisContext *c);
int redisFreeKeepFd(redisContext *c);
redisFD redisFreeKeepFd(redisContext *c);
int redisBufferRead(redisContext *c);
int redisBufferWrite(redisContext *c, int *done);

11
deps/hiredis/hiredis.pc.in vendored Normal file
View File

@ -0,0 +1,11 @@
prefix=@CMAKE_INSTALL_PREFIX@
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
pkgincludedir=${includedir}/hiredis
Name: hiredis
Description: Minimalistic C client library for Redis.
Version: @PROJECT_VERSION@
Libs: -L${libdir} -lhiredis
Cflags: -I${pkgincludedir} -D_FILE_OFFSET_BITS=64

53
deps/hiredis/hiredis_ssl.h vendored Normal file
View File

@ -0,0 +1,53 @@
/*
* Copyright (c) 2019, Redis Labs
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __HIREDIS_SSL_H
#define __HIREDIS_SSL_H
/* This is the underlying struct for SSL in ssl.h, which is not included to
* keep build dependencies short here.
*/
struct ssl_st;
/**
* Secure the connection using SSL. This should be done before any command is
* executed on the connection.
*/
int redisSecureConnection(redisContext *c, const char *capath, const char *certpath,
const char *keypath, const char *servername);
/**
* Initiate SSL/TLS negotiation on a provided context.
*/
int redisInitiateSSL(redisContext *c, struct ssl_st *ssl);
#endif /* __HIREDIS_SSL_H */

12
deps/hiredis/hiredis_ssl.pc.in vendored Normal file
View File

@ -0,0 +1,12 @@
prefix=@CMAKE_INSTALL_PREFIX@
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
pkgincludedir=${includedir}/hiredis
Name: hiredis_ssl
Description: SSL Support for hiredis.
Version: @PROJECT_VERSION@
Requires: hiredis
Libs: -L${libdir} -lhiredis_ssl
Libs.private: -lssl -lcrypto

122
deps/hiredis/net.c vendored
View File

@ -34,36 +34,64 @@
#include "fmacros.h"
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/select.h>
#include <sys/un.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <netdb.h>
#include <errno.h>
#include <stdarg.h>
#include <stdio.h>
#include <poll.h>
#include <limits.h>
#include <stdlib.h>
#include "net.h"
#include "sds.h"
#include "sockcompat.h"
#include "win32.h"
/* Defined in hiredis.c */
void __redisSetError(redisContext *c, int type, const char *str);
static void redisContextCloseFd(redisContext *c) {
if (c && c->fd >= 0) {
void redisNetClose(redisContext *c) {
if (c && c->fd != REDIS_INVALID_FD) {
close(c->fd);
c->fd = -1;
c->fd = REDIS_INVALID_FD;
}
}
int redisNetRead(redisContext *c, char *buf, size_t bufcap) {
int nread = recv(c->fd, buf, bufcap, 0);
if (nread == -1) {
if ((errno == EWOULDBLOCK && !(c->flags & REDIS_BLOCK)) || (errno == EINTR)) {
/* Try again later */
return 0;
} else if(errno == ETIMEDOUT && (c->flags & REDIS_BLOCK)) {
/* especially in windows */
__redisSetError(c, REDIS_ERR_TIMEOUT, "recv timeout");
return -1;
} else {
__redisSetError(c, REDIS_ERR_IO, NULL);
return -1;
}
} else if (nread == 0) {
__redisSetError(c, REDIS_ERR_EOF, "Server closed the connection");
return -1;
} else {
return nread;
}
}
int redisNetWrite(redisContext *c) {
int nwritten = send(c->fd, c->obuf, sdslen(c->obuf), 0);
if (nwritten < 0) {
if ((errno == EWOULDBLOCK && !(c->flags & REDIS_BLOCK)) || (errno == EINTR)) {
/* Try again later */
} else {
__redisSetError(c, REDIS_ERR_IO, NULL);
return -1;
}
}
return nwritten;
}
static void __redisSetErrorFromErrno(redisContext *c, int type, const char *prefix) {
int errorno = errno; /* snprintf() may change errno */
char buf[128] = { 0 };
@ -79,15 +107,15 @@ static int redisSetReuseAddr(redisContext *c) {
int on = 1;
if (setsockopt(c->fd, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) == -1) {
__redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
redisContextCloseFd(c);
redisNetClose(c);
return REDIS_ERR;
}
return REDIS_OK;
}
static int redisCreateSocket(redisContext *c, int type) {
int s;
if ((s = socket(type, SOCK_STREAM, 0)) == -1) {
redisFD s;
if ((s = socket(type, SOCK_STREAM, 0)) == REDIS_INVALID_FD) {
__redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
return REDIS_ERR;
}
@ -101,6 +129,7 @@ static int redisCreateSocket(redisContext *c, int type) {
}
static int redisSetBlocking(redisContext *c, int blocking) {
#ifndef _WIN32
int flags;
/* Set the socket nonblocking.
@ -108,7 +137,7 @@ static int redisSetBlocking(redisContext *c, int blocking) {
* interrupted by a signal. */
if ((flags = fcntl(c->fd, F_GETFL)) == -1) {
__redisSetErrorFromErrno(c,REDIS_ERR_IO,"fcntl(F_GETFL)");
redisContextCloseFd(c);
redisNetClose(c);
return REDIS_ERR;
}
@ -119,15 +148,23 @@ static int redisSetBlocking(redisContext *c, int blocking) {
if (fcntl(c->fd, F_SETFL, flags) == -1) {
__redisSetErrorFromErrno(c,REDIS_ERR_IO,"fcntl(F_SETFL)");
redisContextCloseFd(c);
redisNetClose(c);
return REDIS_ERR;
}
#else
u_long mode = blocking ? 0 : 1;
if (ioctl(c->fd, FIONBIO, &mode) == -1) {
__redisSetErrorFromErrno(c, REDIS_ERR_IO, "ioctl(FIONBIO)");
redisNetClose(c);
return REDIS_ERR;
}
#endif /* _WIN32 */
return REDIS_OK;
}
int redisKeepAlive(redisContext *c, int interval) {
int val = 1;
int fd = c->fd;
redisFD fd = c->fd;
if (setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof(val)) == -1){
__redisSetError(c,REDIS_ERR_OTHER,strerror(errno));
@ -170,7 +207,7 @@ static int redisSetTcpNoDelay(redisContext *c) {
int yes = 1;
if (setsockopt(c->fd, IPPROTO_TCP, TCP_NODELAY, &yes, sizeof(yes)) == -1) {
__redisSetErrorFromErrno(c,REDIS_ERR_IO,"setsockopt(TCP_NODELAY)");
redisContextCloseFd(c);
redisNetClose(c);
return REDIS_ERR;
}
return REDIS_OK;
@ -212,12 +249,12 @@ static int redisContextWaitReady(redisContext *c, long msec) {
if ((res = poll(wfd, 1, msec)) == -1) {
__redisSetErrorFromErrno(c, REDIS_ERR_IO, "poll(2)");
redisContextCloseFd(c);
redisNetClose(c);
return REDIS_ERR;
} else if (res == 0) {
errno = ETIMEDOUT;
__redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
redisContextCloseFd(c);
redisNetClose(c);
return REDIS_ERR;
}
@ -230,7 +267,7 @@ static int redisContextWaitReady(redisContext *c, long msec) {
}
__redisSetErrorFromErrno(c,REDIS_ERR_IO,NULL);
redisContextCloseFd(c);
redisNetClose(c);
return REDIS_ERR;
}
@ -277,11 +314,18 @@ int redisCheckSocketError(redisContext *c) {
}
int redisContextSetTimeout(redisContext *c, const struct timeval tv) {
if (setsockopt(c->fd,SOL_SOCKET,SO_RCVTIMEO,&tv,sizeof(tv)) == -1) {
const void *to_ptr = &tv;
size_t to_sz = sizeof(tv);
#ifdef _WIN32
DWORD timeout_msec = tv.tv_sec * 1000 + tv.tv_usec / 1000;
to_ptr = &timeout_msec;
to_sz = sizeof(timeout_msec);
#endif
if (setsockopt(c->fd,SOL_SOCKET,SO_RCVTIMEO,to_ptr,to_sz) == -1) {
__redisSetErrorFromErrno(c,REDIS_ERR_IO,"setsockopt(SO_RCVTIMEO)");
return REDIS_ERR;
}
if (setsockopt(c->fd,SOL_SOCKET,SO_SNDTIMEO,&tv,sizeof(tv)) == -1) {
if (setsockopt(c->fd,SOL_SOCKET,SO_SNDTIMEO,to_ptr,to_sz) == -1) {
__redisSetErrorFromErrno(c,REDIS_ERR_IO,"setsockopt(SO_SNDTIMEO)");
return REDIS_ERR;
}
@ -291,7 +335,8 @@ int redisContextSetTimeout(redisContext *c, const struct timeval tv) {
static int _redisContextConnectTcp(redisContext *c, const char *addr, int port,
const struct timeval *timeout,
const char *source_addr) {
int s, rv, n;
redisFD s;
int rv, n;
char _port[6]; /* strlen("65535"); */
struct addrinfo hints, *servinfo, *bservinfo, *p, *b;
int blocking = (c->flags & REDIS_BLOCK);
@ -360,7 +405,7 @@ static int _redisContextConnectTcp(redisContext *c, const char *addr, int port,
}
for (p = servinfo; p != NULL; p = p->ai_next) {
addrretry:
if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == -1)
if ((s = socket(p->ai_family,p->ai_socktype,p->ai_protocol)) == REDIS_INVALID_FD)
continue;
c->fd = s;
@ -401,16 +446,14 @@ addrretry:
}
/* For repeat connection */
if (c->saddr) {
free(c->saddr);
}
free(c->saddr);
c->saddr = malloc(p->ai_addrlen);
memcpy(c->saddr, p->ai_addr, p->ai_addrlen);
c->addrlen = p->ai_addrlen;
if (connect(s,p->ai_addr,p->ai_addrlen) == -1) {
if (errno == EHOSTUNREACH) {
redisContextCloseFd(c);
redisNetClose(c);
continue;
} else if (errno == EINPROGRESS) {
if (blocking) {
@ -424,7 +467,7 @@ addrretry:
if (++reuses >= REDIS_CONNECT_RETRIES) {
goto error;
} else {
redisContextCloseFd(c);
redisNetClose(c);
goto addrretry;
}
} else {
@ -471,8 +514,9 @@ int redisContextConnectBindTcp(redisContext *c, const char *addr, int port,
}
int redisContextConnectUnix(redisContext *c, const char *path, const struct timeval *timeout) {
#ifndef _WIN32
int blocking = (c->flags & REDIS_BLOCK);
struct sockaddr_un sa;
struct sockaddr_un *sa;
long timeout_msec = -1;
if (redisCreateSocket(c,AF_UNIX) < 0)
@ -499,9 +543,11 @@ int redisContextConnectUnix(redisContext *c, const char *path, const struct time
if (redisContextTimeoutMsec(c,&timeout_msec) != REDIS_OK)
return REDIS_ERR;
sa.sun_family = AF_UNIX;
strncpy(sa.sun_path,path,sizeof(sa.sun_path)-1);
if (connect(c->fd, (struct sockaddr*)&sa, sizeof(sa)) == -1) {
sa = (struct sockaddr_un*)(c->saddr = malloc(sizeof(struct sockaddr_un)));
c->addrlen = sizeof(struct sockaddr_un);
sa->sun_family = AF_UNIX;
strncpy(sa->sun_path, path, sizeof(sa->sun_path) - 1);
if (connect(c->fd, (struct sockaddr*)sa, sizeof(*sa)) == -1) {
if (errno == EINPROGRESS && !blocking) {
/* This is ok. */
} else {
@ -516,4 +562,10 @@ int redisContextConnectUnix(redisContext *c, const char *path, const struct time
c->flags |= REDIS_CONNECTED;
return REDIS_OK;
#else
/* We currently do not support Unix sockets for Windows. */
/* TODO(m): https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ */
errno = EPROTONOSUPPORT;
return REDIS_ERR;
#endif /* _WIN32 */
}

4
deps/hiredis/net.h vendored
View File

@ -37,6 +37,10 @@
#include "hiredis.h"
void redisNetClose(redisContext *c);
int redisNetRead(redisContext *c, char *buf, size_t bufcap);
int redisNetWrite(redisContext *c);
int redisCheckSocketError(redisContext *c);
int redisContextSetTimeout(redisContext *c, const struct timeval tv);
int redisContextConnectTcp(redisContext *c, const char *addr, int port, const struct timeval *timeout);

27
deps/hiredis/read.c vendored
View File

@ -34,6 +34,7 @@
#include <stdlib.h>
#ifndef _MSC_VER
#include <unistd.h>
#include <strings.h>
#endif
#include <assert.h>
#include <errno.h>
@ -43,6 +44,7 @@
#include "read.h"
#include "sds.h"
#include "win32.h"
static void __redisReaderSetError(redisReader *r, int type, const char *str) {
size_t len;
@ -293,9 +295,9 @@ static int processLineItem(redisReader *r) {
buf[len] = '\0';
if (strcasecmp(buf,",inf") == 0) {
d = 1.0/0.0; /* Positive infinite. */
d = INFINITY; /* Positive infinite. */
} else if (strcasecmp(buf,",-inf") == 0) {
d = -1.0/0.0; /* Nevative infinite. */
d = -INFINITY; /* Nevative infinite. */
} else {
d = strtod((char*)buf,&eptr);
if (buf[0] == '\0' || eptr[0] != '\0' || isnan(d)) {
@ -378,10 +380,18 @@ static int processBulkItem(redisReader *r) {
/* Only continue when the buffer contains the entire bulk item. */
bytelen += len+2; /* include \r\n */
if (r->pos+bytelen <= r->len) {
if ((cur->type == REDIS_REPLY_VERB && len < 4) ||
(cur->type == REDIS_REPLY_VERB && s[5] != ':'))
{
__redisReaderSetError(r,REDIS_ERR_PROTOCOL,
"Verbatim string 4 bytes of content type are "
"missing or incorrectly encoded.");
return REDIS_ERR;
}
if (r->fn && r->fn->createString)
obj = r->fn->createString(cur,s+2,len);
else
obj = (void*)REDIS_REPLY_STRING;
obj = (void*)(long)cur->type;
success = 1;
}
}
@ -429,7 +439,7 @@ static int processAggregateItem(redisReader *r) {
root = (r->ridx == 0);
if (elements < -1 || elements > INT_MAX) {
if (elements < -1 || (LLONG_MAX > SIZE_MAX && elements > SIZE_MAX)) {
__redisReaderSetError(r,REDIS_ERR_PROTOCOL,
"Multi-bulk length out of range");
return REDIS_ERR;
@ -522,6 +532,9 @@ static int processItem(redisReader *r) {
case '#':
cur->type = REDIS_REPLY_BOOL;
break;
case '=':
cur->type = REDIS_REPLY_VERB;
break;
default:
__redisReaderSetErrorProtocolByte(r,*p);
return REDIS_ERR;
@ -542,6 +555,7 @@ static int processItem(redisReader *r) {
case REDIS_REPLY_BOOL:
return processLineItem(r);
case REDIS_REPLY_STRING:
case REDIS_REPLY_VERB:
return processBulkItem(r);
case REDIS_REPLY_ARRAY:
case REDIS_REPLY_MAP:
@ -656,8 +670,11 @@ int redisReaderGetReply(redisReader *r, void **reply) {
/* Emit a reply when there is one. */
if (r->ridx == -1) {
if (reply != NULL)
if (reply != NULL) {
*reply = r->reply;
} else if (r->reply != NULL && r->fn && r->fn->freeObject) {
r->fn->freeObject(r->reply);
}
r->reply = NULL;
}
return REDIS_OK;

5
deps/hiredis/read.h vendored
View File

@ -45,6 +45,7 @@
#define REDIS_ERR_EOF 3 /* End of file */
#define REDIS_ERR_PROTOCOL 4 /* Protocol error */
#define REDIS_ERR_OOM 5 /* Out of memory */
#define REDIS_ERR_TIMEOUT 6 /* Timed out */
#define REDIS_ERR_OTHER 2 /* Everything else... */
#define REDIS_REPLY_STRING 1
@ -55,12 +56,12 @@
#define REDIS_REPLY_ERROR 6
#define REDIS_REPLY_DOUBLE 7
#define REDIS_REPLY_BOOL 8
#define REDIS_REPLY_VERB 9
#define REDIS_REPLY_MAP 9
#define REDIS_REPLY_SET 10
#define REDIS_REPLY_ATTR 11
#define REDIS_REPLY_PUSH 12
#define REDIS_REPLY_BIGNUM 13
#define REDIS_REPLY_VERB 14
#define REDIS_READER_MAX_BUF (1024*16) /* Default max unused reader buffer. */
@ -79,7 +80,7 @@ typedef struct redisReadTask {
typedef struct redisReplyObjectFunctions {
void *(*createString)(const redisReadTask*, char*, size_t);
void *(*createArray)(const redisReadTask*, int);
void *(*createArray)(const redisReadTask*, size_t);
void *(*createInteger)(const redisReadTask*, long long);
void *(*createDouble)(const redisReadTask*, double, char*, size_t);
void *(*createNil)(const redisReadTask*);

2
deps/hiredis/sds.c vendored
View File

@ -1035,7 +1035,7 @@ sds *sdssplitargs(const char *line, int *argc) {
s_free(vector);
return NULL;
}
vector = new_vector;
vector[*argc] = current;
(*argc)++;

31
deps/hiredis/sds.h vendored
View File

@ -34,6 +34,9 @@
#define __SDS_H
#define SDS_MAX_PREALLOC (1024*1024)
#ifdef _MSC_VER
#define __attribute__(x)
#endif
#include <sys/types.h>
#include <stdarg.h>
@ -132,20 +135,20 @@ static inline void sdssetlen(sds s, size_t newlen) {
case SDS_TYPE_5:
{
unsigned char *fp = ((unsigned char*)s)-1;
*fp = SDS_TYPE_5 | (newlen << SDS_TYPE_BITS);
*fp = (unsigned char)(SDS_TYPE_5 | (newlen << SDS_TYPE_BITS));
}
break;
case SDS_TYPE_8:
SDS_HDR(8,s)->len = newlen;
SDS_HDR(8,s)->len = (uint8_t)newlen;
break;
case SDS_TYPE_16:
SDS_HDR(16,s)->len = newlen;
SDS_HDR(16,s)->len = (uint16_t)newlen;
break;
case SDS_TYPE_32:
SDS_HDR(32,s)->len = newlen;
SDS_HDR(32,s)->len = (uint32_t)newlen;
break;
case SDS_TYPE_64:
SDS_HDR(64,s)->len = newlen;
SDS_HDR(64,s)->len = (uint64_t)newlen;
break;
}
}
@ -156,21 +159,21 @@ static inline void sdsinclen(sds s, size_t inc) {
case SDS_TYPE_5:
{
unsigned char *fp = ((unsigned char*)s)-1;
unsigned char newlen = SDS_TYPE_5_LEN(flags)+inc;
unsigned char newlen = SDS_TYPE_5_LEN(flags)+(unsigned char)inc;
*fp = SDS_TYPE_5 | (newlen << SDS_TYPE_BITS);
}
break;
case SDS_TYPE_8:
SDS_HDR(8,s)->len += inc;
SDS_HDR(8,s)->len += (uint8_t)inc;
break;
case SDS_TYPE_16:
SDS_HDR(16,s)->len += inc;
SDS_HDR(16,s)->len += (uint16_t)inc;
break;
case SDS_TYPE_32:
SDS_HDR(32,s)->len += inc;
SDS_HDR(32,s)->len += (uint32_t)inc;
break;
case SDS_TYPE_64:
SDS_HDR(64,s)->len += inc;
SDS_HDR(64,s)->len += (uint64_t)inc;
break;
}
}
@ -200,16 +203,16 @@ static inline void sdssetalloc(sds s, size_t newlen) {
/* Nothing to do, this type has no total allocation info. */
break;
case SDS_TYPE_8:
SDS_HDR(8,s)->alloc = newlen;
SDS_HDR(8,s)->alloc = (uint8_t)newlen;
break;
case SDS_TYPE_16:
SDS_HDR(16,s)->alloc = newlen;
SDS_HDR(16,s)->alloc = (uint16_t)newlen;
break;
case SDS_TYPE_32:
SDS_HDR(32,s)->alloc = newlen;
SDS_HDR(32,s)->alloc = (uint32_t)newlen;
break;
case SDS_TYPE_64:
SDS_HDR(64,s)->alloc = newlen;
SDS_HDR(64,s)->alloc = (uint64_t)newlen;
break;
}
}

248
deps/hiredis/sockcompat.c vendored Normal file
View File

@ -0,0 +1,248 @@
/*
* Copyright (c) 2019, Marcus Geelnard <m at bitsnbites dot eu>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#define REDIS_SOCKCOMPAT_IMPLEMENTATION
#include "sockcompat.h"
#ifdef _WIN32
static int _wsaErrorToErrno(int err) {
switch (err) {
case WSAEWOULDBLOCK:
return EWOULDBLOCK;
case WSAEINPROGRESS:
return EINPROGRESS;
case WSAEALREADY:
return EALREADY;
case WSAENOTSOCK:
return ENOTSOCK;
case WSAEDESTADDRREQ:
return EDESTADDRREQ;
case WSAEMSGSIZE:
return EMSGSIZE;
case WSAEPROTOTYPE:
return EPROTOTYPE;
case WSAENOPROTOOPT:
return ENOPROTOOPT;
case WSAEPROTONOSUPPORT:
return EPROTONOSUPPORT;
case WSAEOPNOTSUPP:
return EOPNOTSUPP;
case WSAEAFNOSUPPORT:
return EAFNOSUPPORT;
case WSAEADDRINUSE:
return EADDRINUSE;
case WSAEADDRNOTAVAIL:
return EADDRNOTAVAIL;
case WSAENETDOWN:
return ENETDOWN;
case WSAENETUNREACH:
return ENETUNREACH;
case WSAENETRESET:
return ENETRESET;
case WSAECONNABORTED:
return ECONNABORTED;
case WSAECONNRESET:
return ECONNRESET;
case WSAENOBUFS:
return ENOBUFS;
case WSAEISCONN:
return EISCONN;
case WSAENOTCONN:
return ENOTCONN;
case WSAETIMEDOUT:
return ETIMEDOUT;
case WSAECONNREFUSED:
return ECONNREFUSED;
case WSAELOOP:
return ELOOP;
case WSAENAMETOOLONG:
return ENAMETOOLONG;
case WSAEHOSTUNREACH:
return EHOSTUNREACH;
case WSAENOTEMPTY:
return ENOTEMPTY;
default:
/* We just return a generic I/O error if we could not find a relevant error. */
return EIO;
}
}
static void _updateErrno(int success) {
errno = success ? 0 : _wsaErrorToErrno(WSAGetLastError());
}
static int _initWinsock() {
static int s_initialized = 0;
if (!s_initialized) {
static WSADATA wsadata;
int err = WSAStartup(MAKEWORD(2,2), &wsadata);
if (err != 0) {
errno = _wsaErrorToErrno(err);
return 0;
}
s_initialized = 1;
}
return 1;
}
int win32_getaddrinfo(const char *node, const char *service, const struct addrinfo *hints, struct addrinfo **res) {
/* Note: This function is likely to be called before other functions, so run init here. */
if (!_initWinsock()) {
return EAI_FAIL;
}
switch (getaddrinfo(node, service, hints, res)) {
case 0: return 0;
case WSATRY_AGAIN: return EAI_AGAIN;
case WSAEINVAL: return EAI_BADFLAGS;
case WSAEAFNOSUPPORT: return EAI_FAMILY;
case WSA_NOT_ENOUGH_MEMORY: return EAI_MEMORY;
case WSAHOST_NOT_FOUND: return EAI_NONAME;
case WSATYPE_NOT_FOUND: return EAI_SERVICE;
case WSAESOCKTNOSUPPORT: return EAI_SOCKTYPE;
default: return EAI_FAIL; /* Including WSANO_RECOVERY */
}
}
const char *win32_gai_strerror(int errcode) {
switch (errcode) {
case 0: errcode = 0; break;
case EAI_AGAIN: errcode = WSATRY_AGAIN; break;
case EAI_BADFLAGS: errcode = WSAEINVAL; break;
case EAI_FAMILY: errcode = WSAEAFNOSUPPORT; break;
case EAI_MEMORY: errcode = WSA_NOT_ENOUGH_MEMORY; break;
case EAI_NONAME: errcode = WSAHOST_NOT_FOUND; break;
case EAI_SERVICE: errcode = WSATYPE_NOT_FOUND; break;
case EAI_SOCKTYPE: errcode = WSAESOCKTNOSUPPORT; break;
default: errcode = WSANO_RECOVERY; break; /* Including EAI_FAIL */
}
return gai_strerror(errcode);
}
void win32_freeaddrinfo(struct addrinfo *res) {
freeaddrinfo(res);
}
SOCKET win32_socket(int domain, int type, int protocol) {
SOCKET s;
/* Note: This function is likely to be called before other functions, so run init here. */
if (!_initWinsock()) {
return INVALID_SOCKET;
}
_updateErrno((s = socket(domain, type, protocol)) != INVALID_SOCKET);
return s;
}
int win32_ioctl(SOCKET fd, unsigned long request, unsigned long *argp) {
int ret = ioctlsocket(fd, (long)request, argp);
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
int win32_bind(SOCKET sockfd, const struct sockaddr *addr, socklen_t addrlen) {
int ret = bind(sockfd, addr, addrlen);
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
int win32_connect(SOCKET sockfd, const struct sockaddr *addr, socklen_t addrlen) {
int ret = connect(sockfd, addr, addrlen);
_updateErrno(ret != SOCKET_ERROR);
/* For Winsock connect(), the WSAEWOULDBLOCK error means the same thing as
* EINPROGRESS for POSIX connect(), so we do that translation to keep POSIX
* logic consistent. */
if (errno == EWOULDBLOCK) {
errno = EINPROGRESS;
}
return ret != SOCKET_ERROR ? ret : -1;
}
int win32_getsockopt(SOCKET sockfd, int level, int optname, void *optval, socklen_t *optlen) {
int ret = 0;
if ((level == SOL_SOCKET) && ((optname == SO_RCVTIMEO) || (optname == SO_SNDTIMEO))) {
if (*optlen >= sizeof (struct timeval)) {
struct timeval *tv = optval;
DWORD timeout = 0;
socklen_t dwlen = 0;
ret = getsockopt(sockfd, level, optname, (char *)&timeout, &dwlen);
tv->tv_sec = timeout / 1000;
tv->tv_usec = (timeout * 1000) % 1000000;
} else {
ret = WSAEFAULT;
}
*optlen = sizeof (struct timeval);
} else {
ret = getsockopt(sockfd, level, optname, (char*)optval, optlen);
}
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
int win32_setsockopt(SOCKET sockfd, int level, int optname, const void *optval, socklen_t optlen) {
int ret = 0;
if ((level == SOL_SOCKET) && ((optname == SO_RCVTIMEO) || (optname == SO_SNDTIMEO))) {
struct timeval *tv = optval;
DWORD timeout = tv->tv_sec * 1000 + tv->tv_usec / 1000;
ret = setsockopt(sockfd, level, optname, (const char*)&timeout, sizeof(DWORD));
} else {
ret = setsockopt(sockfd, level, optname, (const char*)optval, optlen);
}
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
int win32_close(SOCKET fd) {
int ret = closesocket(fd);
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
ssize_t win32_recv(SOCKET sockfd, void *buf, size_t len, int flags) {
int ret = recv(sockfd, (char*)buf, (int)len, flags);
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
ssize_t win32_send(SOCKET sockfd, const void *buf, size_t len, int flags) {
int ret = send(sockfd, (const char*)buf, (int)len, flags);
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
int win32_poll(struct pollfd *fds, nfds_t nfds, int timeout) {
int ret = WSAPoll(fds, nfds, timeout);
_updateErrno(ret != SOCKET_ERROR);
return ret != SOCKET_ERROR ? ret : -1;
}
#endif /* _WIN32 */

91
deps/hiredis/sockcompat.h vendored Normal file
View File

@ -0,0 +1,91 @@
/*
* Copyright (c) 2019, Marcus Geelnard <m at bitsnbites dot eu>
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __SOCKCOMPAT_H
#define __SOCKCOMPAT_H
#ifndef _WIN32
/* For POSIX systems we use the standard BSD socket API. */
#include <unistd.h>
#include <sys/socket.h>
#include <sys/select.h>
#include <sys/un.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <poll.h>
#else
/* For Windows we use winsock. */
#undef _WIN32_WINNT
#define _WIN32_WINNT 0x0600 /* To get WSAPoll etc. */
#include <winsock2.h>
#include <ws2tcpip.h>
#include <stddef.h>
#ifdef _MSC_VER
typedef signed long ssize_t;
#endif
/* Emulate the parts of the BSD socket API that we need (override the winsock signatures). */
int win32_getaddrinfo(const char *node, const char *service, const struct addrinfo *hints, struct addrinfo **res);
const char *win32_gai_strerror(int errcode);
void win32_freeaddrinfo(struct addrinfo *res);
SOCKET win32_socket(int domain, int type, int protocol);
int win32_ioctl(SOCKET fd, unsigned long request, unsigned long *argp);
int win32_bind(SOCKET sockfd, const struct sockaddr *addr, socklen_t addrlen);
int win32_connect(SOCKET sockfd, const struct sockaddr *addr, socklen_t addrlen);
int win32_getsockopt(SOCKET sockfd, int level, int optname, void *optval, socklen_t *optlen);
int win32_setsockopt(SOCKET sockfd, int level, int optname, const void *optval, socklen_t optlen);
int win32_close(SOCKET fd);
ssize_t win32_recv(SOCKET sockfd, void *buf, size_t len, int flags);
ssize_t win32_send(SOCKET sockfd, const void *buf, size_t len, int flags);
typedef ULONG nfds_t;
int win32_poll(struct pollfd *fds, nfds_t nfds, int timeout);
#ifndef REDIS_SOCKCOMPAT_IMPLEMENTATION
#define getaddrinfo(node, service, hints, res) win32_getaddrinfo(node, service, hints, res)
#undef gai_strerror
#define gai_strerror(errcode) win32_gai_strerror(errcode)
#define freeaddrinfo(res) win32_freeaddrinfo(res)
#define socket(domain, type, protocol) win32_socket(domain, type, protocol)
#define ioctl(fd, request, argp) win32_ioctl(fd, request, argp)
#define bind(sockfd, addr, addrlen) win32_bind(sockfd, addr, addrlen)
#define connect(sockfd, addr, addrlen) win32_connect(sockfd, addr, addrlen)
#define getsockopt(sockfd, level, optname, optval, optlen) win32_getsockopt(sockfd, level, optname, optval, optlen)
#define setsockopt(sockfd, level, optname, optval, optlen) win32_setsockopt(sockfd, level, optname, optval, optlen)
#define close(fd) win32_close(fd)
#define recv(sockfd, buf, len, flags) win32_recv(sockfd, buf, len, flags)
#define send(sockfd, buf, len, flags) win32_send(sockfd, buf, len, flags)
#define poll(fds, nfds, timeout) win32_poll(fds, nfds, timeout)
#endif /* REDIS_SOCKCOMPAT_IMPLEMENTATION */
#endif /* _WIN32 */
#endif /* __SOCKCOMPAT_H */

448
deps/hiredis/ssl.c vendored Normal file
View File

@ -0,0 +1,448 @@
/*
* Copyright (c) 2009-2011, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2010-2011, Pieter Noordhuis <pcnoordhuis at gmail dot com>
* Copyright (c) 2019, Redis Labs
*
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "hiredis.h"
#include "async.h"
#include <assert.h>
#include <pthread.h>
#include <errno.h>
#include <string.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include "async_private.h"
void __redisSetError(redisContext *c, int type, const char *str);
/* The SSL context is attached to SSL/TLS connections as a privdata. */
typedef struct redisSSLContext {
/**
* OpenSSL SSL_CTX; It is optional and will not be set when using
* user-supplied SSL.
*/
SSL_CTX *ssl_ctx;
/**
* OpenSSL SSL object.
*/
SSL *ssl;
/**
* SSL_write() requires to be called again with the same arguments it was
* previously called with in the event of an SSL_read/SSL_write situation
*/
size_t lastLen;
/** Whether the SSL layer requires read (possibly before a write) */
int wantRead;
/**
* Whether a write was requested prior to a read. If set, the write()
* should resume whenever a read takes place, if possible
*/
int pendingWrite;
} redisSSLContext;
/* Forward declaration */
redisContextFuncs redisContextSSLFuncs;
#ifdef HIREDIS_SSL_TRACE
/**
* Callback used for debugging
*/
static void sslLogCallback(const SSL *ssl, int where, int ret) {
const char *retstr = "";
int should_log = 1;
/* Ignore low-level SSL stuff */
if (where & SSL_CB_ALERT) {
should_log = 1;
}
if (where == SSL_CB_HANDSHAKE_START || where == SSL_CB_HANDSHAKE_DONE) {
should_log = 1;
}
if ((where & SSL_CB_EXIT) && ret == 0) {
should_log = 1;
}
if (!should_log) {
return;
}
retstr = SSL_alert_type_string(ret);
printf("ST(0x%x). %s. R(0x%x)%s\n", where, SSL_state_string_long(ssl), ret, retstr);
if (where == SSL_CB_HANDSHAKE_DONE) {
printf("Using SSL version %s. Cipher=%s\n", SSL_get_version(ssl), SSL_get_cipher_name(ssl));
}
}
#endif
/**
* OpenSSL global initialization and locking handling callbacks.
* Note that this is only required for OpenSSL < 1.1.0.
*/
#if OPENSSL_VERSION_NUMBER < 0x10100000L
#define HIREDIS_USE_CRYPTO_LOCKS
#endif
#ifdef HIREDIS_USE_CRYPTO_LOCKS
typedef pthread_mutex_t sslLockType;
static void sslLockInit(sslLockType *l) {
pthread_mutex_init(l, NULL);
}
static void sslLockAcquire(sslLockType *l) {
pthread_mutex_lock(l);
}
static void sslLockRelease(sslLockType *l) {
pthread_mutex_unlock(l);
}
static pthread_mutex_t *ossl_locks;
static void opensslDoLock(int mode, int lkid, const char *f, int line) {
sslLockType *l = ossl_locks + lkid;
if (mode & CRYPTO_LOCK) {
sslLockAcquire(l);
} else {
sslLockRelease(l);
}
(void)f;
(void)line;
}
static void initOpensslLocks(void) {
unsigned ii, nlocks;
if (CRYPTO_get_locking_callback() != NULL) {
/* Someone already set the callback before us. Don't destroy it! */
return;
}
nlocks = CRYPTO_num_locks();
ossl_locks = malloc(sizeof(*ossl_locks) * nlocks);
for (ii = 0; ii < nlocks; ii++) {
sslLockInit(ossl_locks + ii);
}
CRYPTO_set_locking_callback(opensslDoLock);
}
#endif /* HIREDIS_USE_CRYPTO_LOCKS */
/**
* SSL Connection initialization.
*/
static int redisSSLConnect(redisContext *c, SSL_CTX *ssl_ctx, SSL *ssl) {
if (c->privdata) {
__redisSetError(c, REDIS_ERR_OTHER, "redisContext was already associated");
return REDIS_ERR;
}
c->privdata = calloc(1, sizeof(redisSSLContext));
c->funcs = &redisContextSSLFuncs;
redisSSLContext *rssl = c->privdata;
rssl->ssl_ctx = ssl_ctx;
rssl->ssl = ssl;
SSL_set_mode(rssl->ssl, SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER);
SSL_set_fd(rssl->ssl, c->fd);
SSL_set_connect_state(rssl->ssl);
ERR_clear_error();
int rv = SSL_connect(rssl->ssl);
if (rv == 1) {
return REDIS_OK;
}
rv = SSL_get_error(rssl->ssl, rv);
if (((c->flags & REDIS_BLOCK) == 0) &&
(rv == SSL_ERROR_WANT_READ || rv == SSL_ERROR_WANT_WRITE)) {
return REDIS_OK;
}
if (c->err == 0) {
char err[512];
if (rv == SSL_ERROR_SYSCALL)
snprintf(err,sizeof(err)-1,"SSL_connect failed: %s",strerror(errno));
else {
unsigned long e = ERR_peek_last_error();
snprintf(err,sizeof(err)-1,"SSL_connect failed: %s",
ERR_reason_error_string(e));
}
__redisSetError(c, REDIS_ERR_IO, err);
}
return REDIS_ERR;
}
int redisInitiateSSL(redisContext *c, SSL *ssl) {
return redisSSLConnect(c, NULL, ssl);
}
int redisSecureConnection(redisContext *c, const char *capath,
const char *certpath, const char *keypath, const char *servername) {
SSL_CTX *ssl_ctx = NULL;
SSL *ssl = NULL;
/* Initialize global OpenSSL stuff */
static int isInit = 0;
if (!isInit) {
isInit = 1;
SSL_library_init();
#ifdef HIREDIS_USE_CRYPTO_LOCKS
initOpensslLocks();
#endif
}
ssl_ctx = SSL_CTX_new(SSLv23_client_method());
if (!ssl_ctx) {
__redisSetError(c, REDIS_ERR_OTHER, "Failed to create SSL_CTX");
goto error;
}
#ifdef HIREDIS_SSL_TRACE
SSL_CTX_set_info_callback(ssl_ctx, sslLogCallback);
#endif
SSL_CTX_set_options(ssl_ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3);
SSL_CTX_set_verify(ssl_ctx, SSL_VERIFY_PEER, NULL);
if ((certpath != NULL && keypath == NULL) || (keypath != NULL && certpath == NULL)) {
__redisSetError(c, REDIS_ERR_OTHER, "certpath and keypath must be specified together");
goto error;
}
if (capath) {
if (!SSL_CTX_load_verify_locations(ssl_ctx, capath, NULL)) {
__redisSetError(c, REDIS_ERR_OTHER, "Invalid CA certificate");
goto error;
}
}
if (certpath) {
if (!SSL_CTX_use_certificate_chain_file(ssl_ctx, certpath)) {
__redisSetError(c, REDIS_ERR_OTHER, "Invalid client certificate");
goto error;
}
if (!SSL_CTX_use_PrivateKey_file(ssl_ctx, keypath, SSL_FILETYPE_PEM)) {
__redisSetError(c, REDIS_ERR_OTHER, "Invalid client key");
goto error;
}
}
ssl = SSL_new(ssl_ctx);
if (!ssl) {
__redisSetError(c, REDIS_ERR_OTHER, "Couldn't create new SSL instance");
goto error;
}
if (servername) {
if (!SSL_set_tlsext_host_name(ssl, servername)) {
__redisSetError(c, REDIS_ERR_OTHER, "Couldn't set server name indication");
goto error;
}
}
return redisSSLConnect(c, ssl_ctx, ssl);
error:
if (ssl) SSL_free(ssl);
if (ssl_ctx) SSL_CTX_free(ssl_ctx);
return REDIS_ERR;
}
static int maybeCheckWant(redisSSLContext *rssl, int rv) {
/**
* If the error is WANT_READ or WANT_WRITE, the appropriate flags are set
* and true is returned. False is returned otherwise
*/
if (rv == SSL_ERROR_WANT_READ) {
rssl->wantRead = 1;
return 1;
} else if (rv == SSL_ERROR_WANT_WRITE) {
rssl->pendingWrite = 1;
return 1;
} else {
return 0;
}
}
/**
* Implementation of redisContextFuncs for SSL connections.
*/
static void redisSSLFreeContext(void *privdata){
redisSSLContext *rsc = privdata;
if (!rsc) return;
if (rsc->ssl) {
SSL_free(rsc->ssl);
rsc->ssl = NULL;
}
if (rsc->ssl_ctx) {
SSL_CTX_free(rsc->ssl_ctx);
rsc->ssl_ctx = NULL;
}
free(rsc);
}
static int redisSSLRead(redisContext *c, char *buf, size_t bufcap) {
redisSSLContext *rssl = c->privdata;
int nread = SSL_read(rssl->ssl, buf, bufcap);
if (nread > 0) {
return nread;
} else if (nread == 0) {
__redisSetError(c, REDIS_ERR_EOF, "Server closed the connection");
return -1;
} else {
int err = SSL_get_error(rssl->ssl, nread);
if (c->flags & REDIS_BLOCK) {
/**
* In blocking mode, we should never end up in a situation where
* we get an error without it being an actual error, except
* in the case of EINTR, which can be spuriously received from
* debuggers or whatever.
*/
if (errno == EINTR) {
return 0;
} else {
const char *msg = NULL;
if (errno == EAGAIN) {
msg = "Resource temporarily unavailable";
}
__redisSetError(c, REDIS_ERR_IO, msg);
return -1;
}
}
/**
* We can very well get an EWOULDBLOCK/EAGAIN, however
*/
if (maybeCheckWant(rssl, err)) {
return 0;
} else {
__redisSetError(c, REDIS_ERR_IO, NULL);
return -1;
}
}
}
static int redisSSLWrite(redisContext *c) {
redisSSLContext *rssl = c->privdata;
size_t len = rssl->lastLen ? rssl->lastLen : sdslen(c->obuf);
int rv = SSL_write(rssl->ssl, c->obuf, len);
if (rv > 0) {
rssl->lastLen = 0;
} else if (rv < 0) {
rssl->lastLen = len;
int err = SSL_get_error(rssl->ssl, rv);
if ((c->flags & REDIS_BLOCK) == 0 && maybeCheckWant(rssl, err)) {
return 0;
} else {
__redisSetError(c, REDIS_ERR_IO, NULL);
return -1;
}
}
return rv;
}
static void redisSSLAsyncRead(redisAsyncContext *ac) {
int rv;
redisSSLContext *rssl = ac->c.privdata;
redisContext *c = &ac->c;
rssl->wantRead = 0;
if (rssl->pendingWrite) {
int done;
/* This is probably just a write event */
rssl->pendingWrite = 0;
rv = redisBufferWrite(c, &done);
if (rv == REDIS_ERR) {
__redisAsyncDisconnect(ac);
return;
} else if (!done) {
_EL_ADD_WRITE(ac);
}
}
rv = redisBufferRead(c);
if (rv == REDIS_ERR) {
__redisAsyncDisconnect(ac);
} else {
_EL_ADD_READ(ac);
redisProcessCallbacks(ac);
}
}
static void redisSSLAsyncWrite(redisAsyncContext *ac) {
int rv, done = 0;
redisSSLContext *rssl = ac->c.privdata;
redisContext *c = &ac->c;
rssl->pendingWrite = 0;
rv = redisBufferWrite(c, &done);
if (rv == REDIS_ERR) {
__redisAsyncDisconnect(ac);
return;
}
if (!done) {
if (rssl->wantRead) {
/* Need to read-before-write */
rssl->pendingWrite = 1;
_EL_DEL_WRITE(ac);
} else {
/* No extra reads needed, just need to write more */
_EL_ADD_WRITE(ac);
}
} else {
/* Already done! */
_EL_DEL_WRITE(ac);
}
/* Always reschedule a read */
_EL_ADD_READ(ac);
}
redisContextFuncs redisContextSSLFuncs = {
.free_privdata = redisSSLFreeContext,
.async_read = redisSSLAsyncRead,
.async_write = redisSSLAsyncWrite,
.read = redisSSLRead,
.write = redisSSLWrite
};

93
deps/hiredis/test.c vendored
View File

@ -13,12 +13,16 @@
#include <limits.h>
#include "hiredis.h"
#ifdef HIREDIS_TEST_SSL
#include "hiredis_ssl.h"
#endif
#include "net.h"
enum connection_type {
CONN_TCP,
CONN_UNIX,
CONN_FD
CONN_FD,
CONN_SSL
};
struct config {
@ -33,6 +37,14 @@ struct config {
struct {
const char *path;
} unix_sock;
struct {
const char *host;
int port;
const char *ca_cert;
const char *cert;
const char *key;
} ssl;
};
/* The following lines make up our testing "framework" :) */
@ -93,11 +105,27 @@ static int disconnect(redisContext *c, int keep_fd) {
return -1;
}
static void do_ssl_handshake(redisContext *c, struct config config) {
#ifdef HIREDIS_TEST_SSL
redisSecureConnection(c, config.ssl.ca_cert, config.ssl.cert, config.ssl.key, NULL);
if (c->err) {
printf("SSL error: %s\n", c->errstr);
redisFree(c);
exit(1);
}
#else
(void) c;
(void) config;
#endif
}
static redisContext *do_connect(struct config config) {
redisContext *c = NULL;
if (config.type == CONN_TCP) {
c = redisConnect(config.tcp.host, config.tcp.port);
} else if (config.type == CONN_SSL) {
c = redisConnect(config.ssl.host, config.ssl.port);
} else if (config.type == CONN_UNIX) {
c = redisConnectUnix(config.unix_sock.path);
} else if (config.type == CONN_FD) {
@ -121,9 +149,21 @@ static redisContext *do_connect(struct config config) {
exit(1);
}
if (config.type == CONN_SSL) {
do_ssl_handshake(c, config);
}
return select_database(c);
}
static void do_reconnect(redisContext *c, struct config config) {
redisReconnect(c);
if (config.type == CONN_SSL) {
do_ssl_handshake(c, config);
}
}
static void test_format_commands(void) {
char *cmd;
int len;
@ -360,7 +400,8 @@ static void test_reply_reader(void) {
freeReplyObject(reply);
redisReaderFree(reader);
test("Set error when array > INT_MAX: ");
#if LLONG_MAX > SIZE_MAX
test("Set error when array > SIZE_MAX: ");
reader = redisReaderCreate();
redisReaderFeed(reader, "*9223372036854775807\r\n+asdf\r\n",29);
ret = redisReaderGetReply(reader,&reply);
@ -369,7 +410,6 @@ static void test_reply_reader(void) {
freeReplyObject(reply);
redisReaderFree(reader);
#if LLONG_MAX > SIZE_MAX
test("Set error when bulk > SIZE_MAX: ");
reader = redisReaderCreate();
redisReaderFeed(reader, "$9223372036854775807\r\nasdf\r\n",28);
@ -434,22 +474,23 @@ static void test_free_null(void) {
test_cond(reply == NULL);
}
#define HIREDIS_BAD_DOMAIN "idontexist-noreally.com"
static void test_blocking_connection_errors(void) {
redisContext *c;
struct addrinfo hints = {.ai_family = AF_INET};
struct addrinfo *ai_tmp = NULL;
const char *bad_domain = "idontexist.com";
int rv = getaddrinfo(bad_domain, "6379", &hints, &ai_tmp);
int rv = getaddrinfo(HIREDIS_BAD_DOMAIN, "6379", &hints, &ai_tmp);
if (rv != 0) {
// Address does *not* exist
test("Returns error when host cannot be resolved: ");
// First see if this domain name *actually* resolves to NXDOMAIN
c = redisConnect("dontexist.com", 6379);
c = redisConnect(HIREDIS_BAD_DOMAIN, 6379);
test_cond(
c->err == REDIS_ERR_OTHER &&
(strcmp(c->errstr, "Name or service not known") == 0 ||
strcmp(c->errstr, "Can't resolve: sadkfjaskfjsa.com") == 0 ||
strcmp(c->errstr, "Can't resolve: " HIREDIS_BAD_DOMAIN) == 0 ||
strcmp(c->errstr, "Name does not resolve") == 0 ||
strcmp(c->errstr,
"nodename nor servname provided, or not known") == 0 ||
strcmp(c->errstr, "No address associated with hostname") == 0 ||
@ -574,7 +615,8 @@ static void test_blocking_connection_timeouts(struct config config) {
c = do_connect(config);
test("Does not return a reply when the command times out: ");
s = write(c->fd, cmd, strlen(cmd));
redisAppendFormattedCommand(c, cmd, strlen(cmd));
s = c->funcs->write(c);
tv.tv_sec = 0;
tv.tv_usec = 10000;
redisSetTimeout(c, tv);
@ -583,7 +625,7 @@ static void test_blocking_connection_timeouts(struct config config) {
freeReplyObject(reply);
test("Reconnect properly reconnects after a timeout: ");
redisReconnect(c);
do_reconnect(c, config);
reply = redisCommand(c, "PING");
test_cond(reply != NULL && reply->type == REDIS_REPLY_STATUS && strcmp(reply->str, "PONG") == 0);
freeReplyObject(reply);
@ -591,7 +633,7 @@ static void test_blocking_connection_timeouts(struct config config) {
test("Reconnect properly uses owned parameters: ");
config.tcp.host = "foo";
config.unix_sock.path = "foo";
redisReconnect(c);
do_reconnect(c, config);
reply = redisCommand(c, "PING");
test_cond(reply != NULL && reply->type == REDIS_REPLY_STATUS && strcmp(reply->str, "PONG") == 0);
freeReplyObject(reply);
@ -894,6 +936,23 @@ int main(int argc, char **argv) {
throughput = 0;
} else if (argc >= 1 && !strcmp(argv[0],"--skip-inherit-fd")) {
test_inherit_fd = 0;
#ifdef HIREDIS_TEST_SSL
} else if (argc >= 2 && !strcmp(argv[0],"--ssl-port")) {
argv++; argc--;
cfg.ssl.port = atoi(argv[0]);
} else if (argc >= 2 && !strcmp(argv[0],"--ssl-host")) {
argv++; argc--;
cfg.ssl.host = argv[0];
} else if (argc >= 2 && !strcmp(argv[0],"--ssl-ca-cert")) {
argv++; argc--;
cfg.ssl.ca_cert = argv[0];
} else if (argc >= 2 && !strcmp(argv[0],"--ssl-cert")) {
argv++; argc--;
cfg.ssl.cert = argv[0];
} else if (argc >= 2 && !strcmp(argv[0],"--ssl-key")) {
argv++; argc--;
cfg.ssl.key = argv[0];
#endif
} else {
fprintf(stderr, "Invalid argument: %s\n", argv[0]);
exit(1);
@ -922,6 +981,20 @@ int main(int argc, char **argv) {
test_blocking_io_errors(cfg);
if (throughput) test_throughput(cfg);
#ifdef HIREDIS_TEST_SSL
if (cfg.ssl.port && cfg.ssl.host) {
printf("\nTesting against SSL connection (%s:%d):\n", cfg.ssl.host, cfg.ssl.port);
cfg.type = CONN_SSL;
test_blocking_connection(cfg);
test_blocking_connection_timeouts(cfg);
test_blocking_io_errors(cfg);
test_invalid_timeout_errors(cfg);
test_append_formatted_commands(cfg);
if (throughput) test_throughput(cfg);
}
#endif
if (test_inherit_fd) {
printf("\nTesting against inherited fd (%s):\n", cfg.unix_sock.path);
cfg.type = CONN_FD;

70
deps/hiredis/test.sh vendored Executable file
View File

@ -0,0 +1,70 @@
#!/bin/sh -ue
REDIS_SERVER=${REDIS_SERVER:-redis-server}
REDIS_PORT=${REDIS_PORT:-56379}
REDIS_SSL_PORT=${REDIS_SSL_PORT:-56443}
TEST_SSL=${TEST_SSL:-0}
SSL_TEST_ARGS=
tmpdir=$(mktemp -d)
PID_FILE=${tmpdir}/hiredis-test-redis.pid
SOCK_FILE=${tmpdir}/hiredis-test-redis.sock
if [ "$TEST_SSL" = "1" ]; then
SSL_CA_CERT=${tmpdir}/ca.crt
SSL_CA_KEY=${tmpdir}/ca.key
SSL_CERT=${tmpdir}/redis.crt
SSL_KEY=${tmpdir}/redis.key
openssl genrsa -out ${tmpdir}/ca.key 4096
openssl req \
-x509 -new -nodes -sha256 \
-key ${SSL_CA_KEY} \
-days 3650 \
-subj '/CN=Hiredis Test CA' \
-out ${SSL_CA_CERT}
openssl genrsa -out ${SSL_KEY} 2048
openssl req \
-new -sha256 \
-key ${SSL_KEY} \
-subj '/CN=Hiredis Test Cert' | \
openssl x509 \
-req -sha256 \
-CA ${SSL_CA_CERT} \
-CAkey ${SSL_CA_KEY} \
-CAserial ${tmpdir}/ca.txt \
-CAcreateserial \
-days 365 \
-out ${SSL_CERT}
SSL_TEST_ARGS="--ssl-host 127.0.0.1 --ssl-port ${REDIS_SSL_PORT} --ssl-ca-cert ${SSL_CA_CERT} --ssl-cert ${SSL_CERT} --ssl-key ${SSL_KEY}"
fi
cleanup() {
set +e
kill $(cat ${PID_FILE})
rm -rf ${tmpdir}
}
trap cleanup INT TERM EXIT
cat > ${tmpdir}/redis.conf <<EOF
daemonize yes
pidfile ${PID_FILE}
port ${REDIS_PORT}
bind 127.0.0.1
unixsocket ${SOCK_FILE}
EOF
if [ "$TEST_SSL" = "1" ]; then
cat >> ${tmpdir}/redis.conf <<EOF
tls-port ${REDIS_SSL_PORT}
tls-ca-cert-file ${SSL_CA_CERT}
tls-cert-file ${SSL_CERT}
tls-key-file ${SSL_KEY}
EOF
fi
cat ${tmpdir}/redis.conf
${REDIS_SERVER} ${tmpdir}/redis.conf
${TEST_PREFIX:-} ./hiredis-test -h 127.0.0.1 -p ${REDIS_PORT} -s ${SOCK_FILE} ${SSL_TEST_ARGS}

18
deps/hiredis/win32.h vendored
View File

@ -2,10 +2,20 @@
#define _WIN32_HELPER_INCLUDE
#ifdef _MSC_VER
#include <winsock2.h> /* for struct timeval */
#ifndef inline
#define inline __inline
#endif
#ifndef strcasecmp
#define strcasecmp stricmp
#endif
#ifndef strncasecmp
#define strncasecmp strnicmp
#endif
#ifndef va_copy
#define va_copy(d,s) ((d) = (s))
#endif
@ -37,6 +47,10 @@ __inline int c99_snprintf(char* str, size_t size, const char* format, ...)
return count;
}
#endif
#endif /* _MSC_VER */
#endif
#endif
#ifdef _WIN32
#define strerror_r(errno,buf,len) strerror_s(buf,len,errno)
#endif /* _WIN32 */
#endif /* _WIN32_HELPER_INCLUDE */

8
deps/jemalloc/src/background_thread.c vendored
View File

@ -787,7 +787,13 @@ background_thread_stats_read(tsdn_t *tsdn, background_thread_stats_t *stats) {
nstime_init(&stats->run_interval, 0);
for (unsigned i = 0; i < max_background_threads; i++) {
background_thread_info_t *info = &background_thread_info[i];
malloc_mutex_lock(tsdn, &info->mtx);
if (malloc_mutex_trylock(tsdn, &info->mtx)) {
/*
* Each background thread run may take a long time;
* avoid waiting on the stats if the thread is active.
*/
continue;
}
if (info->state != background_thread_stopped) {
num_runs += info->tot_n_runs;
nstime_add(&stats->run_interval, &info->tot_sleep_time);

203
redis.conf
View File

@ -129,6 +129,76 @@ timeout 0
# Redis default starting with Redis 3.2.1.
tcp-keepalive 300
################################# TLS/SSL #####################################
# By default, TLS/SSL is disabled. To enable it, the "tls-port" configuration
# directive can be used to define TLS-listening ports. To enable TLS on the
# default port, use:
#
# port 0
# tls-port 6379
# Configure a X.509 certificate and private key to use for authenticating the
# server to connected clients, masters or cluster peers. These files should be
# PEM formatted.
#
# tls-cert-file redis.crt tls-key-file redis.key
# Configure a DH parameters file to enable Diffie-Hellman (DH) key exchange:
#
# tls-dh-params-file redis.dh
# Configure a CA certificate(s) bundle or directory to authenticate TLS/SSL
# clients and peers. Redis requires an explicit configuration of at least one
# of these, and will not implicitly use the system wide configuration.
#
# tls-ca-cert-file ca.crt
# tls-ca-cert-dir /etc/ssl/certs
# If TLS/SSL clients are required to authenticate using a client side
# certificate, use this directive.
#
# Note: this applies to all incoming clients, including replicas.
#
# tls-auth-clients yes
# If TLS/SSL should be used when connecting as a replica to a master, enable
# this configuration directive:
#
# tls-replication yes
# If TLS/SSL should be used for the Redis Cluster bus, enable this configuration
# directive.
#
# NOTE: If TLS/SSL is enabled for Cluster Bus, mutual authentication is always
# enforced.
#
# tls-cluster yes
# Explicitly specify TLS versions to support. Allowed values are case insensitive
# and include "TLSv1", "TLSv1.1", "TLSv1.2", "TLSv1.3" (OpenSSL >= 1.1.1) or
# "default" which is currently >= TLSv1.1.
#
# tls-protocols TLSv1.2
# Configure allowed ciphers. See the ciphers(1ssl) manpage for more information
# about the syntax of this string.
#
# Note: this configuration applies only to <= TLSv1.2.
#
# tls-ciphers DEFAULT:!MEDIUM
# Configure allowed TLSv1.3 ciphersuites. See the ciphers(1ssl) manpage for more
# information about the syntax of this string, and specifically for TLSv1.3
# ciphersuites.
#
# tls-ciphersuites TLS_CHACHA20_POLY1305_SHA256
# When choosing a cipher, use the server's preference instead of the client
# preference. By default, the server follows the client's preference.
#
# tls-prefer-server-cipher yes
################################# GENERAL #####################################
# By default Redis does not run as a daemon. Use 'yes' if you need it.
@ -336,13 +406,11 @@ replica-read-only yes
# Replication SYNC strategy: disk or socket.
#
# -------------------------------------------------------
# WARNING: DISKLESS REPLICATION IS EXPERIMENTAL CURRENTLY
# -------------------------------------------------------
# New replicas and reconnecting replicas that are not able to continue the
# replication process just receiving differences, need to do what is called a
# "full synchronization". An RDB file is transmitted from the master to the
# replicas.
#
# New replicas and reconnecting replicas that are not able to continue the replication
# process just receiving differences, need to do what is called a "full
# synchronization". An RDB file is transmitted from the master to the replicas.
# The transmission can happen in two different ways:
#
# 1) Disk-backed: The Redis master creates a new process that writes the RDB
@ -352,14 +420,14 @@ replica-read-only yes
# RDB file to replica sockets, without touching the disk at all.
#
# With disk-backed replication, while the RDB file is generated, more replicas
# can be queued and served with the RDB file as soon as the current child producing
# the RDB file finishes its work. With diskless replication instead once
# the transfer starts, new replicas arriving will be queued and a new transfer
# will start when the current one terminates.
# can be queued and served with the RDB file as soon as the current child
# producing the RDB file finishes its work. With diskless replication instead
# once the transfer starts, new replicas arriving will be queued and a new
# transfer will start when the current one terminates.
#
# When diskless replication is used, the master waits a configurable amount of
# time (in seconds) before starting the transfer in the hope that multiple replicas
# will arrive and the transfer can be parallelized.
# time (in seconds) before starting the transfer in the hope that multiple
# replicas will arrive and the transfer can be parallelized.
#
# With slow disks and fast (large bandwidth) networks, diskless replication
# works better.
@ -370,16 +438,42 @@ repl-diskless-sync no
# to the replicas.
#
# This is important since once the transfer starts, it is not possible to serve
# new replicas arriving, that will be queued for the next RDB transfer, so the server
# waits a delay in order to let more replicas arrive.
# new replicas arriving, that will be queued for the next RDB transfer, so the
# server waits a delay in order to let more replicas arrive.
#
# The delay is specified in seconds, and by default is 5 seconds. To disable
# it entirely just set it to 0 seconds and the transfer will start ASAP.
repl-diskless-sync-delay 5
# Replicas send PINGs to server in a predefined interval. It's possible to change
# this interval with the repl_ping_replica_period option. The default value is 10
# seconds.
# -----------------------------------------------------------------------------
# WARNING: RDB diskless load is experimental. Since in this setup the replica
# does not immediately store an RDB on disk, it may cause data loss during
# failovers. RDB diskless load + Redis modules not handling I/O reads may also
# cause Redis to abort in case of I/O errors during the initial synchronization
# stage with the master. Use only if your do what you are doing.
# -----------------------------------------------------------------------------
#
# Replica can load the RDB it reads from the replication link directly from the
# socket, or store the RDB to a file and read that file after it was completely
# recived from the master.
#
# In many cases the disk is slower than the network, and storing and loading
# the RDB file may increase replication time (and even increase the master's
# Copy on Write memory and salve buffers).
# However, parsing the RDB file directly from the socket may mean that we have
# to flush the contents of the current database before the full rdb was
# received. For this reason we have the following options:
#
# "disabled" - Don't use diskless load (store the rdb file to the disk first)
# "on-empty-db" - Use diskless load only when it is completely safe.
# "swapdb" - Keep a copy of the current db contents in RAM while parsing
# the data directly from the socket. note that this requires
# sufficient memory, if you don't have it, you risk an OOM kill.
repl-diskless-load disabled
# Replicas send PINGs to server in a predefined interval. It's possible to
# change this interval with the repl_ping_replica_period option. The default
# value is 10 seconds.
#
# repl-ping-replica-period 10
@ -411,10 +505,10 @@ repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
# Set the replication backlog size. The backlog is a buffer that accumulates
# replica data when replicas are disconnected for some time, so that when a replica
# wants to reconnect again, often a full resync is not needed, but a partial
# resync is enough, just passing the portion of data the replica missed while
# disconnected.
# replica data when replicas are disconnected for some time, so that when a
# replica wants to reconnect again, often a full resync is not needed, but a
# partial resync is enough, just passing the portion of data the replica
# missed while disconnected.
#
# The bigger the replication backlog, the longer the time the replica can be
# disconnected and later be able to perform a partial resynchronization.
@ -436,13 +530,13 @@ repl-disable-tcp-nodelay no
#
# repl-backlog-ttl 3600
# The replica priority is an integer number published by Redis in the INFO output.
# It is used by Redis Sentinel in order to select a replica to promote into a
# master if the master is no longer working correctly.
# The replica priority is an integer number published by Redis in the INFO
# output. It is used by Redis Sentinel in order to select a replica to promote
# into a master if the master is no longer working correctly.
#
# A replica with a low priority number is considered better for promotion, so
# for instance if there are three replicas with priority 10, 100, 25 Sentinel will
# pick the one with priority 10, that is the lowest.
# for instance if there are three replicas with priority 10, 100, 25 Sentinel
# will pick the one with priority 10, that is the lowest.
#
# However a special priority of 0 marks the replica as not able to perform the
# role of master, so a replica with priority of 0 will never be selected by
@ -502,6 +596,39 @@ replica-priority 100
# replica-announce-ip 5.5.5.5
# replica-announce-port 1234
############################### KEYS TRACKING #################################
# Redis implements server assisted support for client side caching of values.
# This is implemented using an invalidation table that remembers, using
# 16 millions of slots, what clients may have certain subsets of keys. In turn
# this is used in order to send invalidation messages to clients. Please
# to understand more about the feature check this page:
#
# https://redis.io/topics/client-side-caching
#
# When tracking is enabled for a client, all the read only queries are assumed
# to be cached: this will force Redis to store information in the invalidation
# table. When keys are modified, such information is flushed away, and
# invalidation messages are sent to the clients. However if the workload is
# heavily dominated by reads, Redis could use more and more memory in order
# to track the keys fetched by many clients.
#
# For this reason it is possible to configure a maximum fill value for the
# invalidation table. By default it is set to 10%, and once this limit is
# reached, Redis will start to evict caching slots in the invalidation table
# even if keys are not modified, just to reclaim memory: this will in turn
# force the clients to invalidate the cached values. Basically the table
# maximum fill rate is a trade off between the memory you want to spend server
# side to track information about who cached what, and the ability of clients
# to retain cached objects in memory.
#
# If you set the value to 0, it means there are no limits, and all the 16
# millions of caching slots can be used at the same time. In the "stats"
# INFO section, you can find information about the amount of caching slots
# used at every given moment.
#
# tracking-table-max-fill 10
################################## SECURITY ###################################
# Warning: since Redis is pretty fast an outside user can try up to
@ -731,17 +858,17 @@ replica-priority 100
# DEL commands to the replica as keys evict in the master side.
#
# This behavior ensures that masters and replicas stay consistent, and is usually
# what you want, however if your replica is writable, or you want the replica to have
# a different memory setting, and you are sure all the writes performed to the
# replica are idempotent, then you may change this default (but be sure to understand
# what you are doing).
# what you want, however if your replica is writable, or you want the replica
# to have a different memory setting, and you are sure all the writes performed
# to the replica are idempotent, then you may change this default (but be sure
# to understand what you are doing).
#
# Note that since the replica by default does not evict, it may end using more
# memory than the one set via maxmemory (there are certain buffers that may
# be larger on the replica, or data structures may sometimes take more memory and so
# forth). So make sure you monitor your replicas and make sure they have enough
# memory to never hit a real out-of-memory condition before the master hits
# the configured maxmemory setting.
# be larger on the replica, or data structures may sometimes take more memory
# and so forth). So make sure you monitor your replicas and make sure they
# have enough memory to never hit a real out-of-memory condition before the
# master hits the configured maxmemory setting.
#
# replica-ignore-maxmemory yes
@ -942,13 +1069,7 @@ aof-use-rdb-preamble yes
lua-time-limit 5000
################################ REDIS CLUSTER ###############################
#
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however
# in order to mark it as "mature" we need to wait for a non trivial percentage
# of users to deploy it in production.
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
# started as cluster nodes can. In order to start a Redis instance as a
# cluster node enable the cluster support uncommenting the following:

16
runtest-moduleapi Executable file
View File

@ -0,0 +1,16 @@
#!/bin/sh
TCL_VERSIONS="8.5 8.6"
TCLSH=""
for VERSION in $TCL_VERSIONS; do
TCL=`which tclsh$VERSION 2>/dev/null` && TCLSH=$TCL
done
if [ -z $TCLSH ]
then
echo "You need tcl 8.5 or newer in order to run the Redis test"
exit 1
fi
make -C tests/modules && \
$TCLSH tests/test_helper.tcl --single unit/moduleapi/commandfilter --single unit/moduleapi/fork --single unit/moduleapi/testrdb --single unit/moduleapi/infotest --single unit/moduleapi/propagate "${@}"

12
src/Makefile
View File

@ -20,7 +20,7 @@ DEPENDENCY_TARGETS=hiredis linenoise lua
NODEPS:=clean distclean
# Default settings
STD=-std=c99 -pedantic -DREDIS_STATIC=''
STD=-std=c11 -pedantic -DREDIS_STATIC=''
ifneq (,$(findstring clang,$(CC)))
ifneq (,$(findstring FreeBSD,$(uname_S)))
STD+=-Wno-c11-extensions
@ -93,6 +93,8 @@ else
ifeq ($(uname_S),Darwin)
# Darwin
FINAL_LIBS+= -ldl
OPENSSL_CFLAGS=-I/usr/local/opt/openssl/include
OPENSSL_LDFLAGS=-L/usr/local/opt/openssl/lib
else
ifeq ($(uname_S),AIX)
# AIX
@ -145,6 +147,12 @@ ifeq ($(MALLOC),jemalloc)
FINAL_LIBS := ../deps/jemalloc/lib/libjemalloc.a $(FINAL_LIBS)
endif
ifeq ($(BUILD_TLS),yes)
FINAL_CFLAGS+=-DUSE_OPENSSL $(OPENSSL_CFLAGS)
FINAL_LDFLAGS+=$(OPENSSL_LDFLAGS)
FINAL_LIBS += ../deps/hiredis/libhiredis_ssl.a -lssl -lcrypto
endif
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)
REDIS_LD=$(QUIET_LINK)$(CC) $(FINAL_LDFLAGS)
REDIS_INSTALL=$(QUIET_INSTALL)$(INSTALL)
@ -164,7 +172,7 @@ endif
REDIS_SERVER_NAME=redis-server
REDIS_SENTINEL_NAME=redis-sentinel
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o acl.o gopher.o
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o
REDIS_CLI_NAME=redis-cli
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o siphash.o crc16.o
REDIS_BENCHMARK_NAME=redis-benchmark

115
src/acl.c
View File

@ -28,6 +28,7 @@
*/
#include "server.h"
#include "sha256.h"
#include <fcntl.h>
/* =============================================================================
@ -93,6 +94,9 @@ void ACLResetSubcommandsForCommand(user *u, unsigned long id);
void ACLResetSubcommands(user *u);
void ACLAddAllowedSubcommand(user *u, unsigned long id, const char *sub);
/* The length of the string representation of a hashed password. */
#define HASH_PASSWORD_LEN SHA256_BLOCK_SIZE*2
/* =============================================================================
* Helper functions for the rest of the ACL implementation
* ==========================================================================*/
@ -139,6 +143,25 @@ int time_independent_strcmp(char *a, char *b) {
return diff; /* If zero strings are the same. */
}
/* Given an SDS string, returns the SHA256 hex representation as a
* new SDS string. */
sds ACLHashPassword(unsigned char *cleartext, size_t len) {
SHA256_CTX ctx;
unsigned char hash[SHA256_BLOCK_SIZE];
char hex[HASH_PASSWORD_LEN];
char *cset = "0123456789abcdef";
sha256_init(&ctx);
sha256_update(&ctx,(unsigned char*)cleartext,len);
sha256_final(&ctx,hash);
for (int j = 0; j < SHA256_BLOCK_SIZE; j++) {
hex[j*2] = cset[((hash[j]&0xF0)>>4)];
hex[j*2+1] = cset[(hash[j]&0xF)];
}
return sdsnewlen(hex,HASH_PASSWORD_LEN);
}
/* =============================================================================
* Low level ACL API
* ==========================================================================*/
@ -295,7 +318,7 @@ int ACLGetCommandBitCoordinates(uint64_t id, uint64_t *word, uint64_t *bit) {
* Note that this function does not check the ALLCOMMANDS flag of the user
* but just the lowlevel bitmask.
*
* If the bit overflows the user internal represetation, zero is returned
* If the bit overflows the user internal representation, zero is returned
* in order to disallow the execution of the command in such edge case. */
int ACLGetUserCommandBit(user *u, unsigned long id) {
uint64_t word, bit;
@ -311,7 +334,7 @@ int ACLUserCanExecuteFutureCommands(user *u) {
}
/* Set the specified command bit for the specified user to 'value' (0 or 1).
* If the bit overflows the user internal represetation, no operation
* If the bit overflows the user internal representation, no operation
* is performed. As a side effect of calling this function with a value of
* zero, the user flag ALLCOMMANDS is cleared since it is no longer possible
* to skip the command bit explicit test. */
@ -350,7 +373,7 @@ int ACLSetUserCommandBitsForCategory(user *u, const char *category, int value) {
/* Return the number of commands allowed (on) and denied (off) for the user 'u'
* in the subset of commands flagged with the specified category name.
* If the categoty name is not valid, C_ERR is returend, otherwise C_OK is
* If the category name is not valid, C_ERR is returned, otherwise C_OK is
* returned and on and off are populated by reference. */
int ACLCountCategoryBitsForUser(user *u, unsigned long *on, unsigned long *off,
const char *category)
@ -502,7 +525,7 @@ sds ACLDescribeUser(user *u) {
listRewind(u->passwords,&li);
while((ln = listNext(&li))) {
sds thispass = listNodeValue(ln);
res = sdscatlen(res,">",1);
res = sdscatlen(res,"#",1);
res = sdscatsds(res,thispass);
res = sdscatlen(res," ",1);
}
@ -542,6 +565,8 @@ struct redisCommand *ACLLookupCommand(const char *name) {
* and command ID. */
void ACLResetSubcommandsForCommand(user *u, unsigned long id) {
if (u->allowed_subcommands && u->allowed_subcommands[id]) {
for (int i = 0; u->allowed_subcommands[id][i]; i++)
sdsfree(u->allowed_subcommands[id][i]);
zfree(u->allowed_subcommands[id]);
u->allowed_subcommands[id] = NULL;
}
@ -624,10 +649,17 @@ void ACLAddAllowedSubcommand(user *u, unsigned long id, const char *sub) {
* It is possible to specify multiple patterns.
* allkeys Alias for ~*
* resetkeys Flush the list of allowed keys patterns.
* ><password> Add this passowrd to the list of valid password for the user.
* ><password> Add this password to the list of valid password for the user.
* For example >mypass will add "mypass" to the list.
* This directive clears the "nopass" flag (see later).
* #<hash> Add this password hash to the list of valid hashes for
* the user. This is useful if you have previously computed
* the hash, and don't want to store it in plaintext.
* This directive clears the "nopass" flag (see later).
* <<password> Remove this password from the list of valid passwords.
* !<hash> Remove this hashed password from the list of valid passwords.
* This is useful when you want to remove a password just by
* hash without knowing its plaintext version at all.
* nopass All the set passwords of the user are removed, and the user
* is flagged as requiring no password: it means that every
* password will work against this user. If this directive is
@ -663,6 +695,7 @@ void ACLAddAllowedSubcommand(user *u, unsigned long id, const char *sub) {
* EEXIST: You are adding a key pattern after "*" was already added. This is
* almost surely an error on the user side.
* ENODEV: The password you are trying to remove from the user does not exist.
* EBADMSG: The hash you are trying to add is not a valid hash.
*/
int ACLSetUser(user *u, const char *op, ssize_t oplen) {
if (oplen == -1) oplen = strlen(op);
@ -698,14 +731,48 @@ int ACLSetUser(user *u, const char *op, ssize_t oplen) {
} else if (!strcasecmp(op,"resetpass")) {
u->flags &= ~USER_FLAG_NOPASS;
listEmpty(u->passwords);
} else if (op[0] == '>') {
sds newpass = sdsnewlen(op+1,oplen-1);
} else if (op[0] == '>' || op[0] == '#') {
sds newpass;
if (op[0] == '>') {
newpass = ACLHashPassword((unsigned char*)op+1,oplen-1);
} else {
if (oplen != HASH_PASSWORD_LEN + 1) {
errno = EBADMSG;
return C_ERR;
}
/* Password hashes can only be characters that represent
* hexadecimal values, which are numbers and lowercase
* characters 'a' through 'f'.
*/
for(int i = 1; i < HASH_PASSWORD_LEN + 1; i++) {
char c = op[i];
if ((c < 'a' || c > 'f') && (c < '0' || c > '9')) {
errno = EBADMSG;
return C_ERR;
}
}
newpass = sdsnewlen(op+1,oplen-1);
}
listNode *ln = listSearchKey(u->passwords,newpass);
/* Avoid re-adding the same password multiple times. */
if (ln == NULL) listAddNodeTail(u->passwords,newpass);
if (ln == NULL)
listAddNodeTail(u->passwords,newpass);
else
sdsfree(newpass);
u->flags &= ~USER_FLAG_NOPASS;
} else if (op[0] == '<') {
sds delpass = sdsnewlen(op+1,oplen-1);
} else if (op[0] == '<' || op[0] == '!') {
sds delpass;
if (op[0] == '<') {
delpass = ACLHashPassword((unsigned char*)op+1,oplen-1);
} else {
if (oplen != HASH_PASSWORD_LEN + 1) {
errno = EBADMSG;
return C_ERR;
}
delpass = sdsnewlen(op+1,oplen-1);
}
listNode *ln = listSearchKey(u->passwords,delpass);
sdsfree(delpass);
if (ln) {
@ -722,7 +789,10 @@ int ACLSetUser(user *u, const char *op, ssize_t oplen) {
sds newpat = sdsnewlen(op+1,oplen-1);
listNode *ln = listSearchKey(u->patterns,newpat);
/* Avoid re-adding the same pattern multiple times. */
if (ln == NULL) listAddNodeTail(u->patterns,newpat);
if (ln == NULL)
listAddNodeTail(u->patterns,newpat);
else
sdsfree(newpat);
u->flags &= ~USER_FLAG_ALLKEYS;
} else if (op[0] == '+' && op[1] != '@') {
if (strchr(op,'|') == NULL) {
@ -820,6 +890,9 @@ char *ACLSetUserStringError(void) {
else if (errno == ENODEV)
errmsg = "The password you are trying to remove from the user does "
"not exist";
else if (errno == EBADMSG)
errmsg = "The password hash must be exactly 64 characters and contain "
"only lowercase hexadecimal characters";
return errmsg;
}
@ -877,11 +950,15 @@ int ACLCheckUserCredentials(robj *username, robj *password) {
listIter li;
listNode *ln;
listRewind(u->passwords,&li);
sds hashed = ACLHashPassword(password->ptr,sdslen(password->ptr));
while((ln = listNext(&li))) {
sds thispass = listNodeValue(ln);
if (!time_independent_strcmp(password->ptr, thispass))
if (!time_independent_strcmp(hashed, thispass)) {
sdsfree(hashed);
return C_OK;
}
}
sdsfree(hashed);
/* If we reached this point, no password matched. */
errno = EINVAL;
@ -947,9 +1024,9 @@ user *ACLGetUserByName(const char *name, size_t namelen) {
return myuser;
}
/* Check if the command ready to be excuted in the client 'c', and already
* referenced by c->cmd, can be executed by this client according to the
* ACls associated to the client user c->user.
/* Check if the command is ready to be executed in the client 'c', already
* referenced by c->cmd, and can be executed by this client according to the
* ACLs associated to the client user c->user.
*
* If the user can execute the command ACL_OK is returned, otherwise
* ACL_DENIED_CMD or ACL_DENIED_KEY is returned: the first in case the
@ -1120,7 +1197,7 @@ int ACLLoadConfiguredUsers(void) {
}
/* This function loads the ACL from the specified filename: every line
* is validated and shold be either empty or in the format used to specify
* is validated and should be either empty or in the format used to specify
* users in the redis.conf configuration or in the ACL file, that is:
*
* user <username> ... rules ...
@ -1170,7 +1247,7 @@ sds ACLLoadFromFile(const char *filename) {
* to the real user mentioned in the ACL line. */
user *fakeuser = ACLCreateUnlinkedUser();
/* We do all the loading in a fresh insteance of the Users radix tree,
/* We do all the loading in a fresh instance of the Users radix tree,
* so if there are errors loading the ACL file we can rollback to the
* old version. */
rax *old_users = Users;
@ -1246,7 +1323,7 @@ sds ACLLoadFromFile(const char *filename) {
}
/* Note that the same rules already applied to the fake user, so
* we just assert that everything goess well: it should. */
* we just assert that everything goes well: it should. */
for (j = 2; j < argc; j++)
serverAssert(ACLSetUser(u,argv[j],sdslen(argv[j])) == C_OK);
@ -1609,7 +1686,7 @@ void addReplyCommandCategories(client *c, struct redisCommand *cmd) {
setDeferredSetLen(c, flaglen, flagcount);
}
/* AUTH <passowrd>
/* AUTH <password>
* AUTH <username> <password> (Redis >= 6.0 form)
*
* When the user is omitted it means that we are trying to authenticate

14
src/ae.c
View File

@ -76,6 +76,7 @@ aeEventLoop *aeCreateEventLoop(int setsize) {
eventLoop->maxfd = -1;
eventLoop->beforesleep = NULL;
eventLoop->aftersleep = NULL;
eventLoop->flags = 0;
if (aeApiCreate(eventLoop) == -1) goto err;
/* Events with mask == AE_NONE are not set. So let's initialize the
* vector with it. */
@ -97,6 +98,14 @@ int aeGetSetSize(aeEventLoop *eventLoop) {
return eventLoop->setsize;
}
/* Tells the next iteration/s of the event processing to set timeout of 0. */
void aeSetDontWait(aeEventLoop *eventLoop, int noWait) {
if (noWait)
eventLoop->flags |= AE_DONT_WAIT;
else
eventLoop->flags &= ~AE_DONT_WAIT;
}
/* Resize the maximum set size of the event loop.
* If the requested set size is smaller than the current set size, but
* there is already a file descriptor in use that is >= the requested
@ -406,6 +415,11 @@ int aeProcessEvents(aeEventLoop *eventLoop, int flags)
}
}
if (eventLoop->flags & AE_DONT_WAIT) {
tv.tv_sec = tv.tv_usec = 0;
tvp = &tv;
}
/* Call the multiplexing API, will return only on timeout or when
* some event fires. */
numevents = aeApiPoll(eventLoop, tvp);

2
src/ae.h
View File

@ -106,6 +106,7 @@ typedef struct aeEventLoop {
void *apidata; /* This is used for polling API specific data */
aeBeforeSleepProc *beforesleep;
aeBeforeSleepProc *aftersleep;
int flags;
} aeEventLoop;
/* Prototypes */
@ -128,5 +129,6 @@ void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *beforesleep
void aeSetAfterSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *aftersleep);
int aeGetSetSize(aeEventLoop *eventLoop);
int aeResizeSetSize(aeEventLoop *eventLoop, int setsize);
void aeSetDontWait(aeEventLoop *eventLoop, int noWait);
#endif

4
src/ae_epoll.c
View File

@ -121,8 +121,8 @@ static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
if (e->events & EPOLLIN) mask |= AE_READABLE;
if (e->events & EPOLLOUT) mask |= AE_WRITABLE;
if (e->events & EPOLLERR) mask |= AE_WRITABLE;
if (e->events & EPOLLHUP) mask |= AE_WRITABLE;
if (e->events & EPOLLERR) mask |= AE_WRITABLE|AE_READABLE;
if (e->events & EPOLLHUP) mask |= AE_WRITABLE|AE_READABLE;
eventLoop->fired[j].fd = e->data.fd;
eventLoop->fired[j].mask = mask;
}

36
src/anet.c
View File

@ -193,6 +193,20 @@ int anetSendTimeout(char *err, int fd, long long ms) {
return ANET_OK;
}
/* Set the socket receive timeout (SO_RCVTIMEO socket option) to the specified
* number of milliseconds, or disable it if the 'ms' argument is zero. */
int anetRecvTimeout(char *err, int fd, long long ms) {
struct timeval tv;
tv.tv_sec = ms/1000;
tv.tv_usec = (ms%1000)*1000;
if (setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)) == -1) {
anetSetError(err, "setsockopt SO_RCVTIMEO: %s", strerror(errno));
return ANET_ERR;
}
return ANET_OK;
}
/* anetGenericResolve() is called by anetResolve() and anetResolveIP() to
* do the actual work. It resolves the hostname "host" and set the string
* representation of the IP address into the buffer pointed by "ipbuf".
@ -265,8 +279,8 @@ static int anetCreateSocket(char *err, int domain) {
#define ANET_CONNECT_NONE 0
#define ANET_CONNECT_NONBLOCK 1
#define ANET_CONNECT_BE_BINDING 2 /* Best effort binding. */
static int anetTcpGenericConnect(char *err, char *addr, int port,
char *source_addr, int flags)
static int anetTcpGenericConnect(char *err, const char *addr, int port,
const char *source_addr, int flags)
{
int s = ANET_ERR, rv;
char portstr[6]; /* strlen("65535") + 1; */
@ -345,31 +359,31 @@ end:
}
}
int anetTcpConnect(char *err, char *addr, int port)
int anetTcpConnect(char *err, const char *addr, int port)
{
return anetTcpGenericConnect(err,addr,port,NULL,ANET_CONNECT_NONE);
}
int anetTcpNonBlockConnect(char *err, char *addr, int port)
int anetTcpNonBlockConnect(char *err, const char *addr, int port)
{
return anetTcpGenericConnect(err,addr,port,NULL,ANET_CONNECT_NONBLOCK);
}
int anetTcpNonBlockBindConnect(char *err, char *addr, int port,
char *source_addr)
int anetTcpNonBlockBindConnect(char *err, const char *addr, int port,
const char *source_addr)
{
return anetTcpGenericConnect(err,addr,port,source_addr,
ANET_CONNECT_NONBLOCK);
}
int anetTcpNonBlockBestEffortBindConnect(char *err, char *addr, int port,
char *source_addr)
int anetTcpNonBlockBestEffortBindConnect(char *err, const char *addr, int port,
const char *source_addr)
{
return anetTcpGenericConnect(err,addr,port,source_addr,
ANET_CONNECT_NONBLOCK|ANET_CONNECT_BE_BINDING);
}
int anetUnixGenericConnect(char *err, char *path, int flags)
int anetUnixGenericConnect(char *err, const char *path, int flags)
{
int s;
struct sockaddr_un sa;
@ -397,12 +411,12 @@ int anetUnixGenericConnect(char *err, char *path, int flags)
return s;
}
int anetUnixConnect(char *err, char *path)
int anetUnixConnect(char *err, const char *path)
{
return anetUnixGenericConnect(err,path,ANET_CONNECT_NONE);
}
int anetUnixNonBlockConnect(char *err, char *path)
int anetUnixNonBlockConnect(char *err, const char *path)
{
return anetUnixGenericConnect(err,path,ANET_CONNECT_NONBLOCK);
}

13
src/anet.h
View File

@ -49,12 +49,12 @@
#undef ip_len
#endif
int anetTcpConnect(char *err, char *addr, int port);
int anetTcpNonBlockConnect(char *err, char *addr, int port);
int anetTcpNonBlockBindConnect(char *err, char *addr, int port, char *source_addr);
int anetTcpNonBlockBestEffortBindConnect(char *err, char *addr, int port, char *source_addr);
int anetUnixConnect(char *err, char *path);
int anetUnixNonBlockConnect(char *err, char *path);
int anetTcpConnect(char *err, const char *addr, int port);
int anetTcpNonBlockConnect(char *err, const char *addr, int port);
int anetTcpNonBlockBindConnect(char *err, const char *addr, int port, const char *source_addr);
int anetTcpNonBlockBestEffortBindConnect(char *err, const char *addr, int port, const char *source_addr);
int anetUnixConnect(char *err, const char *path);
int anetUnixNonBlockConnect(char *err, const char *path);
int anetRead(int fd, char *buf, int count);
int anetResolve(char *err, char *host, char *ipbuf, size_t ipbuf_len);
int anetResolveIP(char *err, char *host, char *ipbuf, size_t ipbuf_len);
@ -70,6 +70,7 @@ int anetEnableTcpNoDelay(char *err, int fd);
int anetDisableTcpNoDelay(char *err, int fd);
int anetTcpKeepAlive(char *err, int fd);
int anetSendTimeout(char *err, int fd, long long ms);
int anetRecvTimeout(char *err, int fd, long long ms);
int anetPeerToString(int fd, char *ip, size_t ip_len, int *port);
int anetKeepAlive(char *err, int fd, int interval);
int anetSockName(int fd, char *ip, size_t ip_len, int *port);

96
src/aof.c
View File

@ -197,6 +197,12 @@ ssize_t aofRewriteBufferWrite(int fd) {
* AOF file implementation
* ------------------------------------------------------------------------- */
/* Return true if an AOf fsync is currently already in progress in a
* BIO thread. */
int aofFsyncInProgress(void) {
return bioPendingJobsOfType(BIO_AOF_FSYNC) != 0;
}
/* Starts a background task that performs fsync() against the specified
* file descriptor (the one of the AOF file) in another thread. */
void aof_background_fsync(int fd) {
@ -258,9 +264,9 @@ int startAppendOnly(void) {
strerror(errno));
return C_ERR;
}
if (server.rdb_child_pid != -1) {
if (hasActiveChildProcess() && server.aof_child_pid == -1) {
server.aof_rewrite_scheduled = 1;
serverLog(LL_WARNING,"AOF was enabled but there is already a child process saving an RDB file on disk. An AOF background was scheduled to start when possible.");
serverLog(LL_WARNING,"AOF was enabled but there is already another background operation. An AOF background was scheduled to start when possible.");
} else {
/* If there is a pending AOF rewrite, we need to switch it off and
* start a new one: the old one cannot be reused because it is not
@ -297,9 +303,7 @@ ssize_t aofWrite(int fd, const char *buf, size_t len) {
nwritten = write(fd, buf, len);
if (nwritten < 0) {
if (errno == EINTR) {
continue;
}
if (errno == EINTR) continue;
return totwritten ? totwritten : -1;
}
@ -335,10 +339,24 @@ void flushAppendOnlyFile(int force) {
int sync_in_progress = 0;
mstime_t latency;
if (sdslen(server.aof_buf) == 0) return;
if (sdslen(server.aof_buf) == 0) {
/* Check if we need to do fsync even the aof buffer is empty,
* because previously in AOF_FSYNC_EVERYSEC mode, fsync is
* called only when aof buffer is not empty, so if users
* stop write commands before fsync called in one second,
* the data in page cache cannot be flushed in time. */
if (server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.aof_fsync_offset != server.aof_current_size &&
server.unixtime > server.aof_last_fsync &&
!(sync_in_progress = aofFsyncInProgress())) {
goto try_fsync;
} else {
return;
}
}
if (server.aof_fsync == AOF_FSYNC_EVERYSEC)
sync_in_progress = bioPendingJobsOfType(BIO_AOF_FSYNC) != 0;
sync_in_progress = aofFsyncInProgress();
if (server.aof_fsync == AOF_FSYNC_EVERYSEC && !force) {
/* With this append fsync policy we do background fsyncing.
@ -367,6 +385,10 @@ void flushAppendOnlyFile(int force) {
* there is much to do about the whole server stopping for power problems
* or alike */
if (server.aof_flush_sleep && sdslen(server.aof_buf)) {
usleep(server.aof_flush_sleep);
}
latencyStartMonitor(latency);
nwritten = aofWrite(server.aof_fd,server.aof_buf,sdslen(server.aof_buf));
latencyEndMonitor(latency);
@ -377,7 +399,7 @@ void flushAppendOnlyFile(int force) {
* useful for graphing / monitoring purposes. */
if (sync_in_progress) {
latencyAddSampleIfNeeded("aof-write-pending-fsync",latency);
} else if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) {
} else if (hasActiveChildProcess()) {
latencyAddSampleIfNeeded("aof-write-active-child",latency);
} else {
latencyAddSampleIfNeeded("aof-write-alone",latency);
@ -470,11 +492,11 @@ void flushAppendOnlyFile(int force) {
server.aof_buf = sdsempty();
}
try_fsync:
/* Don't fsync if no-appendfsync-on-rewrite is set to yes and there are
* children doing I/O in the background. */
if (server.aof_no_fsync_on_rewrite &&
(server.aof_child_pid != -1 || server.rdb_child_pid != -1))
return;
if (server.aof_no_fsync_on_rewrite && hasActiveChildProcess())
return;
/* Perform the fsync if needed. */
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
@ -484,10 +506,14 @@ void flushAppendOnlyFile(int force) {
redis_fsync(server.aof_fd); /* Let's try to get this data on the disk */
latencyEndMonitor(latency);
latencyAddSampleIfNeeded("aof-fsync-always",latency);
server.aof_fsync_offset = server.aof_current_size;
server.aof_last_fsync = server.unixtime;
} else if ((server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.unixtime > server.aof_last_fsync)) {
if (!sync_in_progress) aof_background_fsync(server.aof_fd);
if (!sync_in_progress) {
aof_background_fsync(server.aof_fd);
server.aof_fsync_offset = server.aof_current_size;
}
server.aof_last_fsync = server.unixtime;
}
}
@ -626,11 +652,12 @@ void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int a
/* In Redis commands are always executed in the context of a client, so in
* order to load the append only file we need to create a fake client. */
struct client *createFakeClient(void) {
struct client *createAOFClient(void) {
struct client *c = zmalloc(sizeof(*c));
selectDb(c,0);
c->fd = -1;
c->id = CLIENT_ID_AOF; /* So modules can identify it's the AOF client. */
c->conn = NULL;
c->name = NULL;
c->querybuf = sdsempty();
c->querybuf_peak = 0;
@ -694,6 +721,7 @@ int loadAppendOnlyFile(char *filename) {
* operation is received. */
if (fp && redis_fstat(fileno(fp),&sb) != -1 && sb.st_size == 0) {
server.aof_current_size = 0;
server.aof_fsync_offset = server.aof_current_size;
fclose(fp);
return C_ERR;
}
@ -702,8 +730,8 @@ int loadAppendOnlyFile(char *filename) {
* to the same file we're about to read. */
server.aof_state = AOF_OFF;
fakeClient = createFakeClient();
startLoading(fp);
fakeClient = createAOFClient();
startLoadingFile(fp, filename);
/* Check if this AOF file has an RDB preamble. In that case we need to
* load the RDB file and later continue loading the AOF tail. */
@ -812,6 +840,8 @@ int loadAppendOnlyFile(char *filename) {
freeFakeClientArgv(fakeClient);
fakeClient->cmd = NULL;
if (server.aof_load_truncated) valid_up_to = ftello(fp);
if (server.key_load_delay)
usleep(server.key_load_delay);
}
/* This point can only be reached when EOF is reached without errors.
@ -832,11 +862,13 @@ loaded_ok: /* DB loaded, cleanup and return C_OK to the caller. */
stopLoading();
aofUpdateCurrentSize();
server.aof_rewrite_base_size = server.aof_current_size;
server.aof_fsync_offset = server.aof_current_size;
return C_OK;
readerr: /* Read error. If feof(fp) is true, fall through to unexpected EOF. */
if (!feof(fp)) {
if (fakeClient) freeFakeClient(fakeClient); /* avoid valgrind warning */
fclose(fp);
serverLog(LL_WARNING,"Unrecoverable error reading the append only file: %s", strerror(errno));
exit(1);
}
@ -867,11 +899,13 @@ uxeof: /* Unexpected AOF end of file. */
}
}
if (fakeClient) freeFakeClient(fakeClient); /* avoid valgrind warning */
fclose(fp);
serverLog(LL_WARNING,"Unexpected end of file reading the append only file. You can: 1) Make a backup of your AOF file, then use ./redis-check-aof --fix <filename>. 2) Alternatively you can set the 'aof-load-truncated' configuration option to yes and restart the server.");
exit(1);
fmterr: /* Format error. */
if (fakeClient) freeFakeClient(fakeClient); /* avoid valgrind warning */
fclose(fp);
serverLog(LL_WARNING,"Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>");
exit(1);
}
@ -1239,7 +1273,7 @@ int rewriteModuleObject(rio *r, robj *key, robj *o) {
RedisModuleIO io;
moduleValue *mv = o->ptr;
moduleType *mt = mv->type;
moduleInitIOContext(io,mt,r);
moduleInitIOContext(io,mt,r,key);
mt->aof_rewrite(&io,key,mv->value);
if (io.ctx) {
moduleFreeContext(io.ctx);
@ -1535,39 +1569,24 @@ void aofClosePipes(void) {
*/
int rewriteAppendOnlyFileBackground(void) {
pid_t childpid;
long long start;
if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) return C_ERR;
if (hasActiveChildProcess()) return C_ERR;
if (aofCreatePipes() != C_OK) return C_ERR;
openChildInfoPipe();
start = ustime();
if ((childpid = fork()) == 0) {
if ((childpid = redisFork()) == 0) {
char tmpfile[256];
/* Child */
closeListeningSockets(0);
redisSetProcTitle("redis-aof-rewrite");
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
if (rewriteAppendOnlyFile(tmpfile) == C_OK) {
size_t private_dirty = zmalloc_get_private_dirty(-1);
if (private_dirty) {
serverLog(LL_NOTICE,
"AOF rewrite: %zu MB of memory used by copy-on-write",
private_dirty/(1024*1024));
}
server.child_info_data.cow_size = private_dirty;
sendChildInfo(CHILD_INFO_TYPE_AOF);
sendChildCOWInfo(CHILD_INFO_TYPE_AOF, "AOF rewrite");
exitFromChild(0);
} else {
exitFromChild(1);
}
} else {
/* Parent */
server.stat_fork_time = ustime()-start;
server.stat_fork_rate = (double) zmalloc_used_memory() * 1000000 / server.stat_fork_time / (1024*1024*1024); /* GB per second. */
latencyAddSampleIfNeeded("fork",server.stat_fork_time/1000);
if (childpid == -1) {
closeChildInfoPipe();
serverLog(LL_WARNING,
@ -1581,7 +1600,6 @@ int rewriteAppendOnlyFileBackground(void) {
server.aof_rewrite_scheduled = 0;
server.aof_rewrite_time_start = time(NULL);
server.aof_child_pid = childpid;
updateDictResizePolicy();
/* We set appendseldb to -1 in order to force the next call to the
* feedAppendOnlyFile() to issue a SELECT command, so the differences
* accumulated by the parent into server.aof_rewrite_buf will start
@ -1596,13 +1614,14 @@ int rewriteAppendOnlyFileBackground(void) {
void bgrewriteaofCommand(client *c) {
if (server.aof_child_pid != -1) {
addReplyError(c,"Background append only file rewriting already in progress");
} else if (server.rdb_child_pid != -1) {
} else if (hasActiveChildProcess()) {
server.aof_rewrite_scheduled = 1;
addReplyStatus(c,"Background append only file rewriting scheduled");
} else if (rewriteAppendOnlyFileBackground() == C_OK) {
addReplyStatus(c,"Background append only file rewriting started");
} else {
addReply(c,shared.err);
addReplyError(c,"Can't execute an AOF background rewriting. "
"Please check the server logs for more information.");
}
}
@ -1741,6 +1760,7 @@ void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
server.aof_selected_db = -1; /* Make sure SELECT is re-issued */
aofUpdateCurrentSize();
server.aof_rewrite_base_size = server.aof_current_size;
server.aof_current_size = server.aof_current_size;
/* Clear regular AOF buffer since its contents was just written to
* the new AOF from the background rewrite buffer. */

10
src/bitops.c
View File

@ -994,12 +994,18 @@ void bitfieldCommand(client *c) {
/* Lookup for read is ok if key doesn't exit, but errors
* if it's not a string. */
o = lookupKeyRead(c->db,c->argv[1]);
if (o != NULL && checkType(c,o,OBJ_STRING)) return;
if (o != NULL && checkType(c,o,OBJ_STRING)) {
zfree(ops);
return;
}
} else {
/* Lookup by making room up to the farest bit reached by
* this operation. */
if ((o = lookupStringForBitCommand(c,
highest_write_offset)) == NULL) return;
highest_write_offset)) == NULL) {
zfree(ops);
return;
}
}
addReplyArrayLen(c,numops);

405
src/blocked.c
View File

@ -229,6 +229,207 @@ void disconnectAllBlockedClients(void) {
}
}
/* Helper function for handleClientsBlockedOnKeys(). This function is called
* when there may be clients blocked on a list key, and there may be new
* data to fetch (the key is ready). */
void serveClientsBlockedOnListKey(robj *o, readyList *rl) {
/* We serve clients in the same order they blocked for
* this key, from the first blocked to the last. */
dictEntry *de = dictFind(rl->db->blocking_keys,rl->key);
if (de) {
list *clients = dictGetVal(de);
int numclients = listLength(clients);
while(numclients--) {
listNode *clientnode = listFirst(clients);
client *receiver = clientnode->value;
if (receiver->btype != BLOCKED_LIST) {
/* Put at the tail, so that at the next call
* we'll not run into it again. */
listDelNode(clients,clientnode);
listAddNodeTail(clients,receiver);
continue;
}
robj *dstkey = receiver->bpop.target;
int where = (receiver->lastcmd &&
receiver->lastcmd->proc == blpopCommand) ?
LIST_HEAD : LIST_TAIL;
robj *value = listTypePop(o,where);
if (value) {
/* Protect receiver->bpop.target, that will be
* freed by the next unblockClient()
* call. */
if (dstkey) incrRefCount(dstkey);
unblockClient(receiver);
if (serveClientBlockedOnList(receiver,
rl->key,dstkey,rl->db,value,
where) == C_ERR)
{
/* If we failed serving the client we need
* to also undo the POP operation. */
listTypePush(o,value,where);
}
if (dstkey) decrRefCount(dstkey);
decrRefCount(value);
} else {
break;
}
}
}
if (listTypeLength(o) == 0) {
dbDelete(rl->db,rl->key);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",rl->key,rl->db->id);
}
/* We don't call signalModifiedKey() as it was already called
* when an element was pushed on the list. */
}
/* Helper function for handleClientsBlockedOnKeys(). This function is called
* when there may be clients blocked on a sorted set key, and there may be new
* data to fetch (the key is ready). */
void serveClientsBlockedOnSortedSetKey(robj *o, readyList *rl) {
/* We serve clients in the same order they blocked for
* this key, from the first blocked to the last. */
dictEntry *de = dictFind(rl->db->blocking_keys,rl->key);
if (de) {
list *clients = dictGetVal(de);
int numclients = listLength(clients);
unsigned long zcard = zsetLength(o);
while(numclients-- && zcard) {
listNode *clientnode = listFirst(clients);
client *receiver = clientnode->value;
if (receiver->btype != BLOCKED_ZSET) {
/* Put at the tail, so that at the next call
* we'll not run into it again. */
listDelNode(clients,clientnode);
listAddNodeTail(clients,receiver);
continue;
}
int where = (receiver->lastcmd &&
receiver->lastcmd->proc == bzpopminCommand)
? ZSET_MIN : ZSET_MAX;
unblockClient(receiver);
genericZpopCommand(receiver,&rl->key,1,where,1,NULL);
zcard--;
/* Replicate the command. */
robj *argv[2];
struct redisCommand *cmd = where == ZSET_MIN ?
server.zpopminCommand :
server.zpopmaxCommand;
argv[0] = createStringObject(cmd->name,strlen(cmd->name));
argv[1] = rl->key;
incrRefCount(rl->key);
propagate(cmd,receiver->db->id,
argv,2,PROPAGATE_AOF|PROPAGATE_REPL);
decrRefCount(argv[0]);
decrRefCount(argv[1]);
}
}
}
/* Helper function for handleClientsBlockedOnKeys(). This function is called
* when there may be clients blocked on a stream key, and there may be new
* data to fetch (the key is ready). */
void serveClientsBlockedOnStreamKey(robj *o, readyList *rl) {
dictEntry *de = dictFind(rl->db->blocking_keys,rl->key);
stream *s = o->ptr;
/* We need to provide the new data arrived on the stream
* to all the clients that are waiting for an offset smaller
* than the current top item. */
if (de) {
list *clients = dictGetVal(de);
listNode *ln;
listIter li;
listRewind(clients,&li);
while((ln = listNext(&li))) {
client *receiver = listNodeValue(ln);
if (receiver->btype != BLOCKED_STREAM) continue;
streamID *gt = dictFetchValue(receiver->bpop.keys,
rl->key);
/* If we blocked in the context of a consumer
* group, we need to resolve the group and update the
* last ID the client is blocked for: this is needed
* because serving other clients in the same consumer
* group will alter the "last ID" of the consumer
* group, and clients blocked in a consumer group are
* always blocked for the ">" ID: we need to deliver
* only new messages and avoid unblocking the client
* otherwise. */
streamCG *group = NULL;
if (receiver->bpop.xread_group) {
group = streamLookupCG(s,
receiver->bpop.xread_group->ptr);
/* If the group was not found, send an error
* to the consumer. */
if (!group) {
addReplyError(receiver,
"-NOGROUP the consumer group this client "
"was blocked on no longer exists");
unblockClient(receiver);
continue;
} else {
*gt = group->last_id;
}
}
if (streamCompareID(&s->last_id, gt) > 0) {
streamID start = *gt;
start.seq++; /* Can't overflow, it's an uint64_t */
/* Lookup the consumer for the group, if any. */
streamConsumer *consumer = NULL;
int noack = 0;
if (group) {
consumer = streamLookupConsumer(group,
receiver->bpop.xread_consumer->ptr,
1);
noack = receiver->bpop.xread_group_noack;
}
/* Emit the two elements sub-array consisting of
* the name of the stream and the data we
* extracted from it. Wrapped in a single-item
* array, since we have just one key. */
if (receiver->resp == 2) {
addReplyArrayLen(receiver,1);
addReplyArrayLen(receiver,2);
} else {
addReplyMapLen(receiver,1);
}
addReplyBulk(receiver,rl->key);
streamPropInfo pi = {
rl->key,
receiver->bpop.xread_group
};
streamReplyWithRange(receiver,s,&start,NULL,
receiver->bpop.xread_count,
0, group, consumer, noack, &pi);
/* Note that after we unblock the client, 'gt'
* and other receiver->bpop stuff are no longer
* valid, so we must do the setup above before
* this call. */
unblockClient(receiver);
}
}
}
}
/* This function should be called by Redis every time a single command,
* a MULTI/EXEC block, or a Lua script, terminated its execution after
* being called by a client. It handles serving clients blocked in
@ -271,202 +472,14 @@ void handleClientsBlockedOnKeys(void) {
/* Serve clients blocked on list key. */
robj *o = lookupKeyWrite(rl->db,rl->key);
if (o != NULL && o->type == OBJ_LIST) {
dictEntry *de;
/* We serve clients in the same order they blocked for
* this key, from the first blocked to the last. */
de = dictFind(rl->db->blocking_keys,rl->key);
if (de) {
list *clients = dictGetVal(de);
int numclients = listLength(clients);
while(numclients--) {
listNode *clientnode = listFirst(clients);
client *receiver = clientnode->value;
if (receiver->btype != BLOCKED_LIST) {
/* Put at the tail, so that at the next call
* we'll not run into it again. */
listDelNode(clients,clientnode);
listAddNodeTail(clients,receiver);
continue;
}
robj *dstkey = receiver->bpop.target;
int where = (receiver->lastcmd &&
receiver->lastcmd->proc == blpopCommand) ?
LIST_HEAD : LIST_TAIL;
robj *value = listTypePop(o,where);
if (value) {
/* Protect receiver->bpop.target, that will be
* freed by the next unblockClient()
* call. */
if (dstkey) incrRefCount(dstkey);
unblockClient(receiver);
if (serveClientBlockedOnList(receiver,
rl->key,dstkey,rl->db,value,
where) == C_ERR)
{
/* If we failed serving the client we need
* to also undo the POP operation. */
listTypePush(o,value,where);
}
if (dstkey) decrRefCount(dstkey);
decrRefCount(value);
} else {
break;
}
}
}
if (listTypeLength(o) == 0) {
dbDelete(rl->db,rl->key);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",rl->key,rl->db->id);
}
/* We don't call signalModifiedKey() as it was already called
* when an element was pushed on the list. */
}
/* Serve clients blocked on sorted set key. */
else if (o != NULL && o->type == OBJ_ZSET) {
dictEntry *de;
/* We serve clients in the same order they blocked for
* this key, from the first blocked to the last. */
de = dictFind(rl->db->blocking_keys,rl->key);
if (de) {
list *clients = dictGetVal(de);
int numclients = listLength(clients);
unsigned long zcard = zsetLength(o);
while(numclients-- && zcard) {
listNode *clientnode = listFirst(clients);
client *receiver = clientnode->value;
if (receiver->btype != BLOCKED_ZSET) {
/* Put at the tail, so that at the next call
* we'll not run into it again. */
listDelNode(clients,clientnode);
listAddNodeTail(clients,receiver);
continue;
}
int where = (receiver->lastcmd &&
receiver->lastcmd->proc == bzpopminCommand)
? ZSET_MIN : ZSET_MAX;
unblockClient(receiver);
genericZpopCommand(receiver,&rl->key,1,where,1,NULL);
zcard--;
/* Replicate the command. */
robj *argv[2];
struct redisCommand *cmd = where == ZSET_MIN ?
server.zpopminCommand :
server.zpopmaxCommand;
argv[0] = createStringObject(cmd->name,strlen(cmd->name));
argv[1] = rl->key;
incrRefCount(rl->key);
propagate(cmd,receiver->db->id,
argv,2,PROPAGATE_AOF|PROPAGATE_REPL);
decrRefCount(argv[0]);
decrRefCount(argv[1]);
}
}
}
/* Serve clients blocked on stream key. */
else if (o != NULL && o->type == OBJ_STREAM) {
dictEntry *de = dictFind(rl->db->blocking_keys,rl->key);
stream *s = o->ptr;
/* We need to provide the new data arrived on the stream
* to all the clients that are waiting for an offset smaller
* than the current top item. */
if (de) {
list *clients = dictGetVal(de);
listNode *ln;
listIter li;
listRewind(clients,&li);
while((ln = listNext(&li))) {
client *receiver = listNodeValue(ln);
if (receiver->btype != BLOCKED_STREAM) continue;
streamID *gt = dictFetchValue(receiver->bpop.keys,
rl->key);
/* If we blocked in the context of a consumer
* group, we need to resolve the group and update the
* last ID the client is blocked for: this is needed
* because serving other clients in the same consumer
* group will alter the "last ID" of the consumer
* group, and clients blocked in a consumer group are
* always blocked for the ">" ID: we need to deliver
* only new messages and avoid unblocking the client
* otherwise. */
streamCG *group = NULL;
if (receiver->bpop.xread_group) {
group = streamLookupCG(s,
receiver->bpop.xread_group->ptr);
/* If the group was not found, send an error
* to the consumer. */
if (!group) {
addReplyError(receiver,
"-NOGROUP the consumer group this client "
"was blocked on no longer exists");
unblockClient(receiver);
continue;
} else {
*gt = group->last_id;
}
}
if (streamCompareID(&s->last_id, gt) > 0) {
streamID start = *gt;
start.seq++; /* Can't overflow, it's an uint64_t */
/* Lookup the consumer for the group, if any. */
streamConsumer *consumer = NULL;
int noack = 0;
if (group) {
consumer = streamLookupConsumer(group,
receiver->bpop.xread_consumer->ptr,
1);
noack = receiver->bpop.xread_group_noack;
}
/* Emit the two elements sub-array consisting of
* the name of the stream and the data we
* extracted from it. Wrapped in a single-item
* array, since we have just one key. */
if (receiver->resp == 2) {
addReplyArrayLen(receiver,1);
addReplyArrayLen(receiver,2);
} else {
addReplyMapLen(receiver,1);
}
addReplyBulk(receiver,rl->key);
streamPropInfo pi = {
rl->key,
receiver->bpop.xread_group
};
streamReplyWithRange(receiver,s,&start,NULL,
receiver->bpop.xread_count,
0, group, consumer, noack, &pi);
/* Note that after we unblock the client, 'gt'
* and other receiver->bpop stuff are no longer
* valid, so we must do the setup above before
* this call. */
unblockClient(receiver);
}
}
}
if (o != NULL) {
if (o->type == OBJ_LIST)
serveClientsBlockedOnListKey(o,rl);
else if (o->type == OBJ_ZSET)
serveClientsBlockedOnSortedSetKey(o,rl);
else if (o->type == OBJ_STREAM)
serveClientsBlockedOnStreamKey(o,rl);
}
/* Free this item. */
@ -592,7 +605,7 @@ void unblockClientWaitingData(client *c) {
* the same key again and again in the list in case of multiple pushes
* made by a script or in the context of MULTI/EXEC.
*
* The list will be finally processed by handleClientsBlockedOnLists() */
* The list will be finally processed by handleClientsBlockedOnKeys() */
void signalKeyAsReady(redisDb *db, robj *key) {
readyList *rl;

2
src/childinfo.c
View File

@ -80,6 +80,8 @@ void receiveChildInfo(void) {
server.stat_rdb_cow_bytes = server.child_info_data.cow_size;
} else if (server.child_info_data.process_type == CHILD_INFO_TYPE_AOF) {
server.stat_aof_cow_bytes = server.child_info_data.cow_size;
} else if (server.child_info_data.process_type == CHILD_INFO_TYPE_MODULE) {
server.stat_module_cow_bytes = server.child_info_data.cow_size;
}
}
}

299
src/cluster.c
View File

@ -49,7 +49,7 @@ clusterNode *myself = NULL;
clusterNode *createClusterNode(char *nodename, int flags);
int clusterAddNode(clusterNode *node);
void clusterAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void clusterReadHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void clusterReadHandler(connection *conn);
void clusterSendPing(clusterLink *link, int type);
void clusterSendFail(char *nodename);
void clusterSendFailoverAuthIfNeeded(clusterNode *node, clusterMsg *request);
@ -138,6 +138,7 @@ int clusterLoadConfig(char *filename) {
/* Handle the special "vars" line. Don't pretend it is the last
* line even if it actually is when generated by Redis. */
if (strcasecmp(argv[0],"vars") == 0) {
if (!(argc % 2)) goto fmterr;
for (j = 1; j < argc; j += 2) {
if (strcasecmp(argv[j],"currentEpoch") == 0) {
server.cluster->currentEpoch =
@ -476,7 +477,8 @@ void clusterInit(void) {
/* Port sanity check II
* The other handshake port check is triggered too late to stop
* us from trying to use a too-high cluster port number. */
if (server.port > (65535-CLUSTER_PORT_INCR)) {
int port = server.tls_cluster ? server.tls_port : server.port;
if (port > (65535-CLUSTER_PORT_INCR)) {
serverLog(LL_WARNING, "Redis port number too high. "
"Cluster communication port is 10,000 port "
"numbers higher than your Redis port. "
@ -484,8 +486,7 @@ void clusterInit(void) {
"lower than 55535.");
exit(1);
}
if (listenToPort(server.port+CLUSTER_PORT_INCR,
if (listenToPort(port+CLUSTER_PORT_INCR,
server.cfd,&server.cfd_count) == C_ERR)
{
exit(1);
@ -507,8 +508,8 @@ void clusterInit(void) {
/* Set myself->port / cport to my listening ports, we'll just need to
* discover the IP address via MEET messages. */
myself->port = server.port;
myself->cport = server.port+CLUSTER_PORT_INCR;
myself->port = port;
myself->cport = port+CLUSTER_PORT_INCR;
if (server.cluster_announce_port)
myself->port = server.cluster_announce_port;
if (server.cluster_announce_bus_port)
@ -592,7 +593,7 @@ clusterLink *createClusterLink(clusterNode *node) {
link->sndbuf = sdsempty();
link->rcvbuf = sdsempty();
link->node = node;
link->fd = -1;
link->conn = NULL;
return link;
}
@ -600,23 +601,45 @@ clusterLink *createClusterLink(clusterNode *node) {
* This function will just make sure that the original node associated
* with this link will have the 'link' field set to NULL. */
void freeClusterLink(clusterLink *link) {
if (link->fd != -1) {
aeDeleteFileEvent(server.el, link->fd, AE_READABLE|AE_WRITABLE);
if (link->conn) {
connClose(link->conn);
link->conn = NULL;
}
sdsfree(link->sndbuf);
sdsfree(link->rcvbuf);
if (link->node)
link->node->link = NULL;
close(link->fd);
zfree(link);
}
static void clusterConnAcceptHandler(connection *conn) {
clusterLink *link;
if (connGetState(conn) != CONN_STATE_CONNECTED) {
serverLog(LL_VERBOSE,
"Error accepting cluster node connection: %s", connGetLastError(conn));
connClose(conn);
return;
}
/* Create a link object we use to handle the connection.
* It gets passed to the readable handler when data is available.
* Initiallly the link->node pointer is set to NULL as we don't know
* which node is, but the right node is references once we know the
* node identity. */
link = createClusterLink(NULL);
link->conn = conn;
connSetPrivateData(conn, link);
/* Register read handler */
connSetReadHandler(conn, clusterReadHandler);
}
#define MAX_CLUSTER_ACCEPTS_PER_CALL 1000
void clusterAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
int cport, cfd;
int max = MAX_CLUSTER_ACCEPTS_PER_CALL;
char cip[NET_IP_STR_LEN];
clusterLink *link;
UNUSED(el);
UNUSED(mask);
UNUSED(privdata);
@ -633,19 +656,24 @@ void clusterAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
"Error accepting cluster node: %s", server.neterr);
return;
}
anetNonBlock(NULL,cfd);
anetEnableTcpNoDelay(NULL,cfd);
connection *conn = server.tls_cluster ? connCreateAcceptedTLS(cfd,1) : connCreateAcceptedSocket(cfd);
connNonBlock(conn);
connEnableTcpNoDelay(conn);
/* Use non-blocking I/O for cluster messages. */
serverLog(LL_VERBOSE,"Accepted cluster node %s:%d", cip, cport);
/* Create a link object we use to handle the connection.
* It gets passed to the readable handler when data is available.
* Initiallly the link->node pointer is set to NULL as we don't know
* which node is, but the right node is references once we know the
* node identity. */
link = createClusterLink(NULL);
link->fd = cfd;
aeCreateFileEvent(server.el,cfd,AE_READABLE,clusterReadHandler,link);
serverLog(LL_VERBOSE,"Accepting cluster node connection from %s:%d", cip, cport);
/* Accept the connection now. connAccept() may call our handler directly
* or schedule it for later depending on connection implementation.
*/
if (connAccept(conn, clusterConnAcceptHandler) == C_ERR) {
serverLog(LL_VERBOSE,
"Error accepting cluster node connection: %s",
connGetLastError(conn));
connClose(conn);
return;
}
}
}
@ -1446,7 +1474,7 @@ void nodeIp2String(char *buf, clusterLink *link, char *announced_ip) {
memcpy(buf,announced_ip,NET_IP_STR_LEN);
buf[NET_IP_STR_LEN-1] = '\0'; /* We are not sure the input is sane. */
} else {
anetPeerToString(link->fd, buf, NET_IP_STR_LEN, NULL);
connPeerToString(link->conn, buf, NET_IP_STR_LEN, NULL);
}
}
@ -1750,7 +1778,7 @@ int clusterProcessPacket(clusterLink *link) {
{
char ip[NET_IP_STR_LEN];
if (anetSockName(link->fd,ip,sizeof(ip),NULL) != -1 &&
if (connSockName(link->conn,ip,sizeof(ip),NULL) != -1 &&
strcmp(ip,myself->ip))
{
memcpy(myself->ip,ip,NET_IP_STR_LEN);
@ -2117,35 +2145,76 @@ void handleLinkIOError(clusterLink *link) {
/* Send data. This is handled using a trivial send buffer that gets
* consumed by write(). We don't try to optimize this for speed too much
* as this is a very low traffic channel. */
void clusterWriteHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
clusterLink *link = (clusterLink*) privdata;
void clusterWriteHandler(connection *conn) {
clusterLink *link = connGetPrivateData(conn);
ssize_t nwritten;
UNUSED(el);
UNUSED(mask);
nwritten = write(fd, link->sndbuf, sdslen(link->sndbuf));
nwritten = connWrite(conn, link->sndbuf, sdslen(link->sndbuf));
if (nwritten <= 0) {
serverLog(LL_DEBUG,"I/O error writing to node link: %s",
(nwritten == -1) ? strerror(errno) : "short write");
(nwritten == -1) ? connGetLastError(conn) : "short write");
handleLinkIOError(link);
return;
}
sdsrange(link->sndbuf,nwritten,-1);
if (sdslen(link->sndbuf) == 0)
aeDeleteFileEvent(server.el, link->fd, AE_WRITABLE);
connSetWriteHandler(link->conn, NULL);
}
/* A connect handler that gets called when a connection to another node
* gets established.
*/
void clusterLinkConnectHandler(connection *conn) {
clusterLink *link = connGetPrivateData(conn);
clusterNode *node = link->node;
/* Check if connection succeeded */
if (connGetState(conn) != CONN_STATE_CONNECTED) {
serverLog(LL_VERBOSE, "Connection with Node %.40s at %s:%d failed: %s",
node->name, node->ip, node->cport,
connGetLastError(conn));
freeClusterLink(link);
return;
}
/* Register a read handler from now on */
connSetReadHandler(conn, clusterReadHandler);
/* Queue a PING in the new connection ASAP: this is crucial
* to avoid false positives in failure detection.
*
* If the node is flagged as MEET, we send a MEET message instead
* of a PING one, to force the receiver to add us in its node
* table. */
mstime_t old_ping_sent = node->ping_sent;
clusterSendPing(link, node->flags & CLUSTER_NODE_MEET ?
CLUSTERMSG_TYPE_MEET : CLUSTERMSG_TYPE_PING);
if (old_ping_sent) {
/* If there was an active ping before the link was
* disconnected, we want to restore the ping time, otherwise
* replaced by the clusterSendPing() call. */
node->ping_sent = old_ping_sent;
}
/* We can clear the flag after the first packet is sent.
* If we'll never receive a PONG, we'll never send new packets
* to this node. Instead after the PONG is received and we
* are no longer in meet/handshake status, we want to send
* normal PING packets. */
node->flags &= ~CLUSTER_NODE_MEET;
serverLog(LL_DEBUG,"Connecting with Node %.40s at %s:%d",
node->name, node->ip, node->cport);
}
/* Read data. Try to read the first field of the header first to check the
* full length of the packet. When a whole packet is in memory this function
* will call the function to process the packet. And so forth. */
void clusterReadHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
char buf[sizeof(clusterMsg)];
void clusterReadHandler(connection *conn) {
clusterMsg buf[1];
ssize_t nread;
clusterMsg *hdr;
clusterLink *link = (clusterLink*) privdata;
clusterLink *link = connGetPrivateData(conn);
unsigned int readlen, rcvbuflen;
UNUSED(el);
UNUSED(mask);
while(1) { /* Read as long as there is data to read. */
rcvbuflen = sdslen(link->rcvbuf);
@ -2173,13 +2242,13 @@ void clusterReadHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
if (readlen > sizeof(buf)) readlen = sizeof(buf);
}
nread = read(fd,buf,readlen);
if (nread == -1 && errno == EAGAIN) return; /* No more data ready. */
nread = connRead(conn,buf,readlen);
if (nread == -1 && (connGetState(conn) == CONN_STATE_CONNECTED)) return; /* No more data ready. */
if (nread <= 0) {
/* I/O error... */
serverLog(LL_DEBUG,"I/O error reading from node link: %s",
(nread == 0) ? "connection closed" : strerror(errno));
(nread == 0) ? "connection closed" : connGetLastError(conn));
handleLinkIOError(link);
return;
} else {
@ -2208,8 +2277,7 @@ void clusterReadHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
* from event handlers that will do stuff with the same link later. */
void clusterSendMessage(clusterLink *link, unsigned char *msg, size_t msglen) {
if (sdslen(link->sndbuf) == 0 && msglen != 0)
aeCreateFileEvent(server.el,link->fd,AE_WRITABLE|AE_BARRIER,
clusterWriteHandler,link);
connSetWriteHandlerWithBarrier(link->conn, clusterWriteHandler, 1);
link->sndbuf = sdscatlen(link->sndbuf, msg, msglen);
@ -2275,11 +2343,12 @@ void clusterBuildMessageHdr(clusterMsg *hdr, int type) {
}
/* Handle cluster-announce-port as well. */
int port = server.tls_cluster ? server.tls_port : server.port;
int announced_port = server.cluster_announce_port ?
server.cluster_announce_port : server.port;
server.cluster_announce_port : port;
int announced_cport = server.cluster_announce_bus_port ?
server.cluster_announce_bus_port :
(server.port + CLUSTER_PORT_INCR);
(port + CLUSTER_PORT_INCR);
memcpy(hdr->myslots,master->slots,sizeof(hdr->myslots));
memset(hdr->slaveof,0,CLUSTER_NAMELEN);
@ -2516,7 +2585,8 @@ void clusterBroadcastPong(int target) {
*
* If link is NULL, then the message is broadcasted to the whole cluster. */
void clusterSendPublish(clusterLink *link, robj *channel, robj *message) {
unsigned char buf[sizeof(clusterMsg)], *payload;
unsigned char *payload;
clusterMsg buf[1];
clusterMsg *hdr = (clusterMsg*) buf;
uint32_t totlen;
uint32_t channel_len, message_len;
@ -2536,7 +2606,7 @@ void clusterSendPublish(clusterLink *link, robj *channel, robj *message) {
/* Try to use the local buffer if possible */
if (totlen < sizeof(buf)) {
payload = buf;
payload = (unsigned char*)buf;
} else {
payload = zmalloc(totlen);
memcpy(payload,hdr,sizeof(*hdr));
@ -2553,7 +2623,7 @@ void clusterSendPublish(clusterLink *link, robj *channel, robj *message) {
decrRefCount(channel);
decrRefCount(message);
if (payload != buf) zfree(payload);
if (payload != (unsigned char*)buf) zfree(payload);
}
/* Send a FAIL message to all the nodes we are able to contact.
@ -2562,7 +2632,7 @@ void clusterSendPublish(clusterLink *link, robj *channel, robj *message) {
* we switch the node state to CLUSTER_NODE_FAIL and ask all the other
* nodes to do the same ASAP. */
void clusterSendFail(char *nodename) {
unsigned char buf[sizeof(clusterMsg)];
clusterMsg buf[1];
clusterMsg *hdr = (clusterMsg*) buf;
clusterBuildMessageHdr(hdr,CLUSTERMSG_TYPE_FAIL);
@ -2574,7 +2644,7 @@ void clusterSendFail(char *nodename) {
* slots configuration. The node name, slots bitmap, and configEpoch info
* are included. */
void clusterSendUpdate(clusterLink *link, clusterNode *node) {
unsigned char buf[sizeof(clusterMsg)];
clusterMsg buf[1];
clusterMsg *hdr = (clusterMsg*) buf;
if (link == NULL) return;
@ -2582,7 +2652,7 @@ void clusterSendUpdate(clusterLink *link, clusterNode *node) {
memcpy(hdr->data.update.nodecfg.nodename,node->name,CLUSTER_NAMELEN);
hdr->data.update.nodecfg.configEpoch = htonu64(node->configEpoch);
memcpy(hdr->data.update.nodecfg.slots,node->slots,sizeof(node->slots));
clusterSendMessage(link,buf,ntohl(hdr->totlen));
clusterSendMessage(link,(unsigned char*)buf,ntohl(hdr->totlen));
}
/* Send a MODULE message.
@ -2590,7 +2660,8 @@ void clusterSendUpdate(clusterLink *link, clusterNode *node) {
* If link is NULL, then the message is broadcasted to the whole cluster. */
void clusterSendModule(clusterLink *link, uint64_t module_id, uint8_t type,
unsigned char *payload, uint32_t len) {
unsigned char buf[sizeof(clusterMsg)], *heapbuf;
unsigned char *heapbuf;
clusterMsg buf[1];
clusterMsg *hdr = (clusterMsg*) buf;
uint32_t totlen;
@ -2605,7 +2676,7 @@ void clusterSendModule(clusterLink *link, uint64_t module_id, uint8_t type,
/* Try to use the local buffer if possible */
if (totlen < sizeof(buf)) {
heapbuf = buf;
heapbuf = (unsigned char*)buf;
} else {
heapbuf = zmalloc(totlen);
memcpy(heapbuf,hdr,sizeof(*hdr));
@ -2618,7 +2689,7 @@ void clusterSendModule(clusterLink *link, uint64_t module_id, uint8_t type,
else
clusterBroadcastMessage(heapbuf,totlen);
if (heapbuf != buf) zfree(heapbuf);
if (heapbuf != (unsigned char*)buf) zfree(heapbuf);
}
/* This function gets a cluster node ID string as target, the same way the nodes
@ -2662,7 +2733,7 @@ void clusterPropagatePublish(robj *channel, robj *message) {
* Note that we send the failover request to everybody, master and slave nodes,
* but only the masters are supposed to reply to our query. */
void clusterRequestFailoverAuth(void) {
unsigned char buf[sizeof(clusterMsg)];
clusterMsg buf[1];
clusterMsg *hdr = (clusterMsg*) buf;
uint32_t totlen;
@ -2678,7 +2749,7 @@ void clusterRequestFailoverAuth(void) {
/* Send a FAILOVER_AUTH_ACK message to the specified node. */
void clusterSendFailoverAuth(clusterNode *node) {
unsigned char buf[sizeof(clusterMsg)];
clusterMsg buf[1];
clusterMsg *hdr = (clusterMsg*) buf;
uint32_t totlen;
@ -2686,12 +2757,12 @@ void clusterSendFailoverAuth(clusterNode *node) {
clusterBuildMessageHdr(hdr,CLUSTERMSG_TYPE_FAILOVER_AUTH_ACK);
totlen = sizeof(clusterMsg)-sizeof(union clusterMsgData);
hdr->totlen = htonl(totlen);
clusterSendMessage(node->link,buf,totlen);
clusterSendMessage(node->link,(unsigned char*)buf,totlen);
}
/* Send a MFSTART message to the specified node. */
void clusterSendMFStart(clusterNode *node) {
unsigned char buf[sizeof(clusterMsg)];
clusterMsg buf[1];
clusterMsg *hdr = (clusterMsg*) buf;
uint32_t totlen;
@ -2699,7 +2770,7 @@ void clusterSendMFStart(clusterNode *node) {
clusterBuildMessageHdr(hdr,CLUSTERMSG_TYPE_MFSTART);
totlen = sizeof(clusterMsg)-sizeof(union clusterMsgData);
hdr->totlen = htonl(totlen);
clusterSendMessage(node->link,buf,totlen);
clusterSendMessage(node->link,(unsigned char*)buf,totlen);
}
/* Vote for the node asking for our vote if there are the conditions. */
@ -3382,13 +3453,11 @@ void clusterCron(void) {
}
if (node->link == NULL) {
int fd;
mstime_t old_ping_sent;
clusterLink *link;
fd = anetTcpNonBlockBindConnect(server.neterr, node->ip,
node->cport, NET_FIRST_BIND_ADDR);
if (fd == -1) {
clusterLink *link = createClusterLink(node);
link->conn = server.tls_cluster ? connCreateTLS() : connCreateSocket();
connSetPrivateData(link->conn, link);
if (connConnect(link->conn, node->ip, node->cport, NET_FIRST_BIND_ADDR,
clusterLinkConnectHandler) == -1) {
/* We got a synchronous error from connect before
* clusterSendPing() had a chance to be called.
* If node->ping_sent is zero, failure detection can't work,
@ -3398,37 +3467,11 @@ void clusterCron(void) {
serverLog(LL_DEBUG, "Unable to connect to "
"Cluster Node [%s]:%d -> %s", node->ip,
node->cport, server.neterr);
freeClusterLink(link);
continue;
}
link = createClusterLink(node);
link->fd = fd;
node->link = link;
aeCreateFileEvent(server.el,link->fd,AE_READABLE,
clusterReadHandler,link);
/* Queue a PING in the new connection ASAP: this is crucial
* to avoid false positives in failure detection.
*
* If the node is flagged as MEET, we send a MEET message instead
* of a PING one, to force the receiver to add us in its node
* table. */
old_ping_sent = node->ping_sent;
clusterSendPing(link, node->flags & CLUSTER_NODE_MEET ?
CLUSTERMSG_TYPE_MEET : CLUSTERMSG_TYPE_PING);
if (old_ping_sent) {
/* If there was an active ping before the link was
* disconnected, we want to restore the ping time, otherwise
* replaced by the clusterSendPing() call. */
node->ping_sent = old_ping_sent;
}
/* We can clear the flag after the first packet is sent.
* If we'll never receive a PONG, we'll never send new packets
* to this node. Instead after the PONG is received and we
* are no longer in meet/handshake status, we want to send
* normal PING packets. */
node->flags &= ~CLUSTER_NODE_MEET;
serverLog(LL_DEBUG,"Connecting with Node %.40s at %s:%d",
node->name, node->ip, node->cport);
}
}
dictReleaseIterator(di);
@ -4251,12 +4294,9 @@ NULL
}
} else if (!strcasecmp(c->argv[1]->ptr,"nodes") && c->argc == 2) {
/* CLUSTER NODES */
robj *o;
sds ci = clusterGenNodesDescription(0);
o = createObject(OBJ_STRING,ci);
addReplyBulk(c,o);
decrRefCount(o);
sds nodes = clusterGenNodesDescription(0);
addReplyVerbatim(c,nodes,sdslen(nodes),"txt");
sdsfree(nodes);
} else if (!strcasecmp(c->argv[1]->ptr,"myid") && c->argc == 2) {
/* CLUSTER MYID */
addReplyBulkCBuffer(c,myself->name, CLUSTER_NAMELEN);
@ -4498,10 +4538,8 @@ NULL
"cluster_stats_messages_received:%lld\r\n", tot_msg_received);
/* Produce the reply protocol. */
addReplySds(c,sdscatprintf(sdsempty(),"$%lu\r\n",
(unsigned long)sdslen(info)));
addReplySds(c,info);
addReply(c,shared.crlf);
addReplyVerbatim(c,info,sdslen(info),"txt");
sdsfree(info);
} else if (!strcasecmp(c->argv[1]->ptr,"saveconfig") && c->argc == 2) {
int retval = clusterSaveConfig(1);
@ -4776,7 +4814,7 @@ NULL
/* Generates a DUMP-format representation of the object 'o', adding it to the
* io stream pointed by 'rio'. This function can't fail. */
void createDumpPayload(rio *payload, robj *o) {
void createDumpPayload(rio *payload, robj *o, robj *key) {
unsigned char buf[2];
uint64_t crc;
@ -4784,7 +4822,7 @@ void createDumpPayload(rio *payload, robj *o) {
* byte followed by the serialized object. This is understood by RESTORE. */
rioInitWithBuffer(payload,sdsempty());
serverAssert(rdbSaveObjectType(payload,o));
serverAssert(rdbSaveObject(payload,o));
serverAssert(rdbSaveObject(payload,o,key));
/* Write the footer, this is how it looks like:
* ----------------+---------------------+---------------+
@ -4832,7 +4870,7 @@ int verifyDumpPayload(unsigned char *p, size_t len) {
* DUMP is actually not used by Redis Cluster but it is the obvious
* complement of RESTORE and can be useful for different applications. */
void dumpCommand(client *c) {
robj *o, *dumpobj;
robj *o;
rio payload;
/* Check if the key is here. */
@ -4842,12 +4880,10 @@ void dumpCommand(client *c) {
}
/* Create the DUMP encoded representation. */
createDumpPayload(&payload,o);
createDumpPayload(&payload,o,c->argv[1]);
/* Transfer to the client */
dumpobj = createObject(OBJ_STRING,payload.io.buffer.ptr);
addReplyBulk(c,dumpobj);
decrRefCount(dumpobj);
addReplyBulkSds(c,payload.io.buffer.ptr);
return;
}
@ -4915,7 +4951,7 @@ void restoreCommand(client *c) {
rioInitWithBuffer(&payload,c->argv[3]->ptr);
if (((type = rdbLoadObjectType(&payload)) == -1) ||
((obj = rdbLoadObject(type,&payload)) == NULL))
((obj = rdbLoadObject(type,&payload,c->argv[1])) == NULL))
{
addReplyError(c,"Bad data format");
return;
@ -4946,7 +4982,7 @@ void restoreCommand(client *c) {
#define MIGRATE_SOCKET_CACHE_TTL 10 /* close cached sockets after 10 sec. */
typedef struct migrateCachedSocket {
int fd;
connection *conn;
long last_dbid;
time_t last_use_time;
} migrateCachedSocket;
@ -4963,7 +4999,7 @@ typedef struct migrateCachedSocket {
* should be called so that the connection will be created from scratch
* the next time. */
migrateCachedSocket* migrateGetSocket(client *c, robj *host, robj *port, long timeout) {
int fd;
connection *conn;
sds name = sdsempty();
migrateCachedSocket *cs;
@ -4983,34 +5019,27 @@ migrateCachedSocket* migrateGetSocket(client *c, robj *host, robj *port, long ti
/* Too many items, drop one at random. */
dictEntry *de = dictGetRandomKey(server.migrate_cached_sockets);
cs = dictGetVal(de);
close(cs->fd);
connClose(cs->conn);
zfree(cs);
dictDelete(server.migrate_cached_sockets,dictGetKey(de));
}
/* Create the socket */
fd = anetTcpNonBlockConnect(server.neterr,c->argv[1]->ptr,
atoi(c->argv[2]->ptr));
if (fd == -1) {
sdsfree(name);
addReplyErrorFormat(c,"Can't connect to target node: %s",
server.neterr);
return NULL;
}
anetEnableTcpNoDelay(server.neterr,fd);
/* Check if it connects within the specified timeout. */
if ((aeWait(fd,AE_WRITABLE,timeout) & AE_WRITABLE) == 0) {
sdsfree(name);
conn = server.tls_cluster ? connCreateTLS() : connCreateSocket();
if (connBlockingConnect(conn, c->argv[1]->ptr, atoi(c->argv[2]->ptr), timeout)
!= C_OK) {
addReplySds(c,
sdsnew("-IOERR error or timeout connecting to the client\r\n"));
close(fd);
connClose(conn);
sdsfree(name);
return NULL;
}
connEnableTcpNoDelay(conn);
/* Add to the cache and return it to the caller. */
cs = zmalloc(sizeof(*cs));
cs->fd = fd;
cs->conn = conn;
cs->last_dbid = -1;
cs->last_use_time = server.unixtime;
dictAdd(server.migrate_cached_sockets,name,cs);
@ -5031,7 +5060,7 @@ void migrateCloseSocket(robj *host, robj *port) {
return;
}
close(cs->fd);
connClose(cs->conn);
zfree(cs);
dictDelete(server.migrate_cached_sockets,name);
sdsfree(name);
@ -5045,7 +5074,7 @@ void migrateCloseTimedoutSockets(void) {
migrateCachedSocket *cs = dictGetVal(de);
if ((server.unixtime - cs->last_use_time) > MIGRATE_SOCKET_CACHE_TTL) {
close(cs->fd);
connClose(cs->conn);
zfree(cs);
dictDelete(server.migrate_cached_sockets,dictGetKey(de));
}
@ -5203,7 +5232,7 @@ try_again:
/* Emit the payload argument, that is the serialized object using
* the DUMP format. */
createDumpPayload(&payload,ov[j]);
createDumpPayload(&payload,ov[j],kv[j]);
serverAssertWithInfo(c,NULL,
rioWriteBulkString(&cmd,payload.io.buffer.ptr,
sdslen(payload.io.buffer.ptr)));
@ -5227,7 +5256,7 @@ try_again:
while ((towrite = sdslen(buf)-pos) > 0) {
towrite = (towrite > (64*1024) ? (64*1024) : towrite);
nwritten = syncWrite(cs->fd,buf+pos,towrite,timeout);
nwritten = connSyncWrite(cs->conn,buf+pos,towrite,timeout);
if (nwritten != (signed)towrite) {
write_error = 1;
goto socket_err;
@ -5241,11 +5270,11 @@ try_again:
char buf2[1024]; /* Restore reply. */
/* Read the AUTH reply if needed. */
if (password && syncReadLine(cs->fd, buf0, sizeof(buf0), timeout) <= 0)
if (password && connSyncReadLine(cs->conn, buf0, sizeof(buf0), timeout) <= 0)
goto socket_err;
/* Read the SELECT reply if needed. */
if (select && syncReadLine(cs->fd, buf1, sizeof(buf1), timeout) <= 0)
if (select && connSyncReadLine(cs->conn, buf1, sizeof(buf1), timeout) <= 0)
goto socket_err;
/* Read the RESTORE replies. */
@ -5260,7 +5289,7 @@ try_again:
if (!copy) newargv = zmalloc(sizeof(robj*)*(num_keys+1));
for (j = 0; j < num_keys; j++) {
if (syncReadLine(cs->fd, buf2, sizeof(buf2), timeout) <= 0) {
if (connSyncReadLine(cs->conn, buf2, sizeof(buf2), timeout) <= 0) {
socket_error = 1;
break;
}

2
src/cluster.h
View File

@ -40,7 +40,7 @@ struct clusterNode;
/* clusterLink encapsulates everything needed to talk with a remote node. */
typedef struct clusterLink {
mstime_t ctime; /* Link creation time */
int fd; /* TCP socket file descriptor */
connection *conn; /* Connection to remote node */
sds sndbuf; /* Packet send buffer */
sds rcvbuf; /* Packet reception buffer */
struct clusterNode *node; /* Node related to this link if any, or NULL */

604
src/config.c
View File

@ -91,6 +91,13 @@ configEnum aof_fsync_enum[] = {
{NULL, 0}
};
configEnum repl_diskless_load_enum[] = {
{"disabled", REPL_DISKLESS_LOAD_DISABLED},
{"on-empty-db", REPL_DISKLESS_LOAD_WHEN_DB_EMPTY},
{"swapdb", REPL_DISKLESS_LOAD_SWAPDB},
{NULL, 0}
};
/* Output buffer limits presets. */
clientBufferLimitsConfig clientBufferLimitsDefaults[CLIENT_TYPE_OBUF_COUNT] = {
{0, 0, 0}, /* normal */
@ -98,6 +105,49 @@ clientBufferLimitsConfig clientBufferLimitsDefaults[CLIENT_TYPE_OBUF_COUNT] = {
{1024*1024*32, 1024*1024*8, 60} /* pubsub */
};
/* Configuration values that require no special handling to set, get, load or
* rewrite. */
typedef struct configYesNo {
const char *name; /* The user visible name of this config */
const char *alias; /* An alias that can also be used for this config */
int *config; /* The pointer to the server config this value is stored in */
const int modifiable; /* Can this value be updated by CONFIG SET? */
const int default_value; /* The default value of the config on rewrite */
} configYesNo;
configYesNo configs_yesno[] = {
/* Non-Modifiable */
{"rdbchecksum",NULL,&server.rdb_checksum,0,CONFIG_DEFAULT_RDB_CHECKSUM},
{"daemonize",NULL,&server.daemonize,0,0},
{"io-threads-do-reads",NULL,&server.io_threads_do_reads, 0, CONFIG_DEFAULT_IO_THREADS_DO_READS},
{"always-show-logo",NULL,&server.always_show_logo,0,CONFIG_DEFAULT_ALWAYS_SHOW_LOGO},
/* Modifiable */
{"protected-mode",NULL,&server.protected_mode,1,CONFIG_DEFAULT_PROTECTED_MODE},
{"rdbcompression",NULL,&server.rdb_compression,1,CONFIG_DEFAULT_RDB_COMPRESSION},
{"activerehashing",NULL,&server.activerehashing,1,CONFIG_DEFAULT_ACTIVE_REHASHING},
{"stop-writes-on-bgsave-error",NULL,&server.stop_writes_on_bgsave_err,1,CONFIG_DEFAULT_STOP_WRITES_ON_BGSAVE_ERROR},
{"dynamic-hz",NULL,&server.dynamic_hz,1,CONFIG_DEFAULT_DYNAMIC_HZ},
{"lazyfree-lazy-eviction",NULL,&server.lazyfree_lazy_eviction,1,CONFIG_DEFAULT_LAZYFREE_LAZY_EVICTION},
{"lazyfree-lazy-expire",NULL,&server.lazyfree_lazy_expire,1,CONFIG_DEFAULT_LAZYFREE_LAZY_EXPIRE},
{"lazyfree-lazy-server-del",NULL,&server.lazyfree_lazy_server_del,1,CONFIG_DEFAULT_LAZYFREE_LAZY_SERVER_DEL},
{"repl-disable-tcp-nodelay",NULL,&server.repl_disable_tcp_nodelay,1,CONFIG_DEFAULT_REPL_DISABLE_TCP_NODELAY},
{"repl-diskless-sync",NULL,&server.repl_diskless_sync,1,CONFIG_DEFAULT_REPL_DISKLESS_SYNC},
{"gopher-enabled",NULL,&server.gopher_enabled,1,CONFIG_DEFAULT_GOPHER_ENABLED},
{"aof-rewrite-incremental-fsync",NULL,&server.aof_rewrite_incremental_fsync,1,CONFIG_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC},
{"no-appendfsync-on-rewrite",NULL,&server.aof_no_fsync_on_rewrite,1,CONFIG_DEFAULT_AOF_NO_FSYNC_ON_REWRITE},
{"cluster-require-full-coverage",NULL,&server.cluster_require_full_coverage,CLUSTER_DEFAULT_REQUIRE_FULL_COVERAGE},
{"rdb-save-incremental-fsync",NULL,&server.rdb_save_incremental_fsync,1,CONFIG_DEFAULT_RDB_SAVE_INCREMENTAL_FSYNC},
{"aof-load-truncated",NULL,&server.aof_load_truncated,1,CONFIG_DEFAULT_AOF_LOAD_TRUNCATED},
{"aof-use-rdb-preamble",NULL,&server.aof_use_rdb_preamble,1,CONFIG_DEFAULT_AOF_USE_RDB_PREAMBLE},
{"cluster-replica-no-failover","cluster-slave-no-failover",&server.cluster_slave_no_failover,1,CLUSTER_DEFAULT_SLAVE_NO_FAILOVER},
{"replica-lazy-flush","slave-lazy-flush",&server.repl_slave_lazy_flush,1,CONFIG_DEFAULT_SLAVE_LAZY_FLUSH},
{"replica-serve-stale-data","slave-serve-stale-data",&server.repl_serve_stale_data,1,CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA},
{"replica-read-only","slave-read-only",&server.repl_slave_ro,1,CONFIG_DEFAULT_SLAVE_READ_ONLY},
{"replica-ignore-maxmemory","slave-ignore-maxmemory",&server.repl_slave_ignore_maxmemory,1,CONFIG_DEFAULT_SLAVE_IGNORE_MAXMEMORY},
{"jemalloc-bg-thread",NULL,&server.jemalloc_bg_thread,1,1},
{NULL, NULL, 0, 0}
};
/*-----------------------------------------------------------------------------
* Enum access functions
*----------------------------------------------------------------------------*/
@ -170,7 +220,7 @@ void queueLoadModule(sds path, sds *argv, int argc) {
}
void loadServerConfigFromString(char *config) {
char *err = NULL;
const char *err = NULL;
int linenum = 0, totlines, i;
int slaveof_linenum = 0;
sds *lines;
@ -201,6 +251,26 @@ void loadServerConfigFromString(char *config) {
}
sdstolower(argv[0]);
/* Iterate the configs that are standard */
int match = 0;
for (configYesNo *config = configs_yesno; config->name != NULL; config++) {
if ((!strcasecmp(argv[0],config->name) ||
(config->alias && !strcasecmp(argv[0],config->alias))) &&
(argc == 2))
{
if ((*(config->config) = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
match = 1;
break;
}
}
if (match) {
sdsfreesplitres(argv,argc);
continue;
}
/* Execute config directives */
if (!strcasecmp(argv[0],"timeout") && argc == 2) {
server.maxidletime = atoi(argv[1]);
@ -212,14 +282,6 @@ void loadServerConfigFromString(char *config) {
if (server.tcpkeepalive < 0) {
err = "Invalid tcp-keepalive value"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"protected-mode") && argc == 2) {
if ((server.protected_mode = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"gopher-enabled") && argc == 2) {
if ((server.gopher_enabled = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"port") && argc == 2) {
server.port = atoi(argv[1]);
if (server.port < 0 || server.port > 65535) {
@ -290,10 +352,6 @@ void loadServerConfigFromString(char *config) {
} else if (!strcasecmp(argv[0],"aclfile") && argc == 2) {
zfree(server.acl_filename);
server.acl_filename = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"always-show-logo") && argc == 2) {
if ((server.always_show_logo = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"syslog-enabled") && argc == 2) {
if ((server.syslog_enabled = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
@ -313,6 +371,11 @@ void loadServerConfigFromString(char *config) {
if (server.dbnum < 1) {
err = "Invalid number of databases"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"io-threads") && argc == 2) {
server.io_threads_num = atoi(argv[1]);
if (server.io_threads_num < 1 || server.io_threads_num > 512) {
err = "Invalid number of I/O threads"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"include") && argc == 2) {
loadServerConfig(argv[1],NULL);
} else if (!strcasecmp(argv[0],"maxclients") && argc == 2) {
@ -372,13 +435,11 @@ void loadServerConfigFromString(char *config) {
err = "repl-timeout must be 1 or greater";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"repl-disable-tcp-nodelay") && argc==2) {
if ((server.repl_disable_tcp_nodelay = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"repl-diskless-sync") && argc==2) {
if ((server.repl_diskless_sync = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
} else if (!strcasecmp(argv[0],"repl-diskless-load") && argc==2) {
server.repl_diskless_load = configEnumGetValue(repl_diskless_load_enum,argv[1]);
if (server.repl_diskless_load == INT_MIN) {
err = "argument must be 'disabled', 'on-empty-db', 'swapdb' or 'flushdb'";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"repl-diskless-sync-delay") && argc==2) {
server.repl_diskless_sync_delay = atoi(argv[1]);
@ -405,57 +466,6 @@ void loadServerConfigFromString(char *config) {
} else if (!strcasecmp(argv[0],"masterauth") && argc == 2) {
zfree(server.masterauth);
server.masterauth = argv[1][0] ? zstrdup(argv[1]) : NULL;
} else if ((!strcasecmp(argv[0],"slave-serve-stale-data") ||
!strcasecmp(argv[0],"replica-serve-stale-data"))
&& argc == 2)
{
if ((server.repl_serve_stale_data = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-read-only") ||
!strcasecmp(argv[0],"replica-read-only"))
&& argc == 2)
{
if ((server.repl_slave_ro = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-ignore-maxmemory") ||
!strcasecmp(argv[0],"replica-ignore-maxmemory"))
&& argc == 2)
{
if ((server.repl_slave_ignore_maxmemory = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"rdbcompression") && argc == 2) {
if ((server.rdb_compression = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"rdbchecksum") && argc == 2) {
if ((server.rdb_checksum = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"activerehashing") && argc == 2) {
if ((server.activerehashing = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"lazyfree-lazy-eviction") && argc == 2) {
if ((server.lazyfree_lazy_eviction = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"lazyfree-lazy-expire") && argc == 2) {
if ((server.lazyfree_lazy_expire = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"lazyfree-lazy-server-del") && argc == 2){
if ((server.lazyfree_lazy_server_del = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-lazy-flush") ||
!strcasecmp(argv[0],"replica-lazy-flush")) && argc == 2)
{
if ((server.repl_slave_lazy_flush = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"activedefrag") && argc == 2) {
if ((server.active_defrag_enabled = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
@ -465,25 +475,15 @@ void loadServerConfigFromString(char *config) {
err = "active defrag can't be enabled without proper jemalloc support"; goto loaderr;
#endif
}
} else if (!strcasecmp(argv[0],"daemonize") && argc == 2) {
if ((server.daemonize = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"dynamic-hz") && argc == 2) {
if ((server.dynamic_hz = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"hz") && argc == 2) {
server.config_hz = atoi(argv[1]);
if (server.config_hz < CONFIG_MIN_HZ) server.config_hz = CONFIG_MIN_HZ;
if (server.config_hz > CONFIG_MAX_HZ) server.config_hz = CONFIG_MAX_HZ;
} else if (!strcasecmp(argv[0],"appendonly") && argc == 2) {
int yes;
if ((yes = yesnotoi(argv[1])) == -1) {
if ((server.aof_enabled = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
server.aof_state = yes ? AOF_ON : AOF_OFF;
server.aof_state = server.aof_enabled ? AOF_ON : AOF_OFF;
} else if (!strcasecmp(argv[0],"appendfilename") && argc == 2) {
if (!pathIsBaseName(argv[1])) {
err = "appendfilename can't be a path, just a filename";
@ -491,11 +491,6 @@ void loadServerConfigFromString(char *config) {
}
zfree(server.aof_filename);
server.aof_filename = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"no-appendfsync-on-rewrite")
&& argc == 2) {
if ((server.aof_no_fsync_on_rewrite= yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"appendfsync") && argc == 2) {
server.aof_fsync = configEnumGetValue(aof_fsync_enum,argv[1]);
if (server.aof_fsync == INT_MIN) {
@ -514,27 +509,17 @@ void loadServerConfigFromString(char *config) {
argc == 2)
{
server.aof_rewrite_min_size = memtoll(argv[1],NULL);
} else if (!strcasecmp(argv[0],"aof-rewrite-incremental-fsync") &&
argc == 2)
{
if ((server.aof_rewrite_incremental_fsync =
yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
} else if (!strcasecmp(argv[0],"rdb-key-save-delay") && argc==2) {
server.rdb_key_save_delay = atoi(argv[1]);
if (server.rdb_key_save_delay < 0) {
err = "rdb-key-save-delay can't be negative";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"rdb-save-incremental-fsync") &&
argc == 2)
{
if ((server.rdb_save_incremental_fsync =
yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"aof-load-truncated") && argc == 2) {
if ((server.aof_load_truncated = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"aof-use-rdb-preamble") && argc == 2) {
if ((server.aof_use_rdb_preamble = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
} else if (!strcasecmp(argv[0],"key-load-delay") && argc==2) {
server.key_load_delay = atoi(argv[1]);
if (server.key_load_delay < 0) {
err = "key-load-delay can't be negative";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"requirepass") && argc == 2) {
if (strlen(argv[1]) > CONFIG_AUTHPASS_MAX_LEN) {
@ -669,13 +654,6 @@ void loadServerConfigFromString(char *config) {
{
err = "Invalid port"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"cluster-require-full-coverage") &&
argc == 2)
{
if ((server.cluster_require_full_coverage = yesnotoi(argv[1])) == -1)
{
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"cluster-node-timeout") && argc == 2) {
server.cluster_node_timeout = strtoll(argv[1],NULL,10);
if (server.cluster_node_timeout <= 0) {
@ -698,19 +676,14 @@ void loadServerConfigFromString(char *config) {
err = "cluster replica validity factor must be zero or positive";
goto loaderr;
}
} else if ((!strcasecmp(argv[0],"cluster-slave-no-failover") ||
!strcasecmp(argv[0],"cluster-replica-no-failover")) &&
argc == 2)
{
server.cluster_slave_no_failover = yesnotoi(argv[1]);
if (server.cluster_slave_no_failover == -1) {
err = "argument must be 'yes' or 'no'";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"lua-time-limit") && argc == 2) {
server.lua_time_limit = strtoll(argv[1],NULL,10);
} else if (!strcasecmp(argv[0],"lua-replicate-commands") && argc == 2) {
server.lua_always_replicate_commands = yesnotoi(argv[1]);
if (server.lua_always_replicate_commands == -1) {
err = "argument must be 'yes' or 'no'";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"slowlog-log-slower-than") &&
argc == 2)
{
@ -725,6 +698,17 @@ void loadServerConfigFromString(char *config) {
}
} else if (!strcasecmp(argv[0],"slowlog-max-len") && argc == 2) {
server.slowlog_max_len = strtoll(argv[1],NULL,10);
} else if (!strcasecmp(argv[0],"tracking-table-max-fill") &&
argc == 2)
{
server.tracking_table_max_fill = strtoll(argv[1],NULL,10);
if (server.tracking_table_max_fill > 100 ||
server.tracking_table_max_fill < 0)
{
err = "The tracking table fill percentage must be an "
"integer between 0 and 100";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"client-output-buffer-limit") &&
argc == 5)
{
@ -747,11 +731,6 @@ void loadServerConfigFromString(char *config) {
server.client_obuf_limits[class].hard_limit_bytes = hard;
server.client_obuf_limits[class].soft_limit_bytes = soft;
server.client_obuf_limits[class].soft_limit_seconds = soft_seconds;
} else if (!strcasecmp(argv[0],"stop-writes-on-bgsave-error") &&
argc == 2) {
if ((server.stop_writes_on_bgsave_err = yesnotoi(argv[1])) == -1) {
err = "argument must be 'yes' or 'no'"; goto loaderr;
}
} else if ((!strcasecmp(argv[0],"slave-priority") ||
!strcasecmp(argv[0],"replica-priority")) && argc == 2)
{
@ -824,6 +803,45 @@ void loadServerConfigFromString(char *config) {
err = sentinelHandleConfiguration(argv+1,argc-1);
if (err) goto loaderr;
}
#ifdef USE_OPENSSL
} else if (!strcasecmp(argv[0],"tls-port") && argc == 2) {
server.tls_port = atoi(argv[1]);
if (server.port < 0 || server.port > 65535) {
err = "Invalid tls-port"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"tls-cluster") && argc == 2) {
server.tls_cluster = yesnotoi(argv[1]);
} else if (!strcasecmp(argv[0],"tls-replication") && argc == 2) {
server.tls_replication = yesnotoi(argv[1]);
} else if (!strcasecmp(argv[0],"tls-auth-clients") && argc == 2) {
server.tls_auth_clients = yesnotoi(argv[1]);
} else if (!strcasecmp(argv[0],"tls-cert-file") && argc == 2) {
zfree(server.tls_ctx_config.cert_file);
server.tls_ctx_config.cert_file = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-key-file") && argc == 2) {
zfree(server.tls_ctx_config.key_file);
server.tls_ctx_config.key_file = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-dh-params-file") && argc == 2) {
zfree(server.tls_ctx_config.dh_params_file);
server.tls_ctx_config.dh_params_file = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-ca-cert-file") && argc == 2) {
zfree(server.tls_ctx_config.ca_cert_file);
server.tls_ctx_config.ca_cert_file = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-ca-cert-dir") && argc == 2) {
zfree(server.tls_ctx_config.ca_cert_dir);
server.tls_ctx_config.ca_cert_dir = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-protocols") && argc >= 2) {
zfree(server.tls_ctx_config.protocols);
server.tls_ctx_config.protocols = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-ciphers") && argc == 2) {
zfree(server.tls_ctx_config.ciphers);
server.tls_ctx_config.ciphers = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-ciphersuites") && argc == 2) {
zfree(server.tls_ctx_config.ciphersuites);
server.tls_ctx_config.ciphersuites = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"tls-prefer-server-ciphers") && argc == 2) {
server.tls_ctx_config.prefer_server_ciphers = yesnotoi(argv[1]);
#endif /* USE_OPENSSL */
} else {
err = "Bad directive or wrong number of arguments"; goto loaderr;
}
@ -932,6 +950,19 @@ void configSetCommand(client *c) {
serverAssertWithInfo(c,c->argv[3],sdsEncodedObject(c->argv[3]));
o = c->argv[3];
/* Iterate the configs that are standard */
for (configYesNo *config = configs_yesno; config->name != NULL; config++) {
if(config->modifiable && (!strcasecmp(c->argv[2]->ptr,config->name) ||
(config->alias && !strcasecmp(c->argv[2]->ptr,config->alias))))
{
int yn = yesnotoi(o->ptr);
if (yn == -1) goto badfmt;
*(config->config) = yn;
addReply(c,shared.ok);
return;
}
}
if (0) { /* this starts the config_set macros else-if chain. */
/* Special fields that can't be handled with general macros. */
@ -989,6 +1020,7 @@ void configSetCommand(client *c) {
int enable = yesnotoi(o->ptr);
if (enable == -1) goto badfmt;
server.aof_enabled = enable;
if (enable == 0 && server.aof_state != AOF_OFF) {
stopAppendOnly();
} else if (enable && server.aof_state == AOF_OFF) {
@ -1096,40 +1128,6 @@ void configSetCommand(client *c) {
/* Boolean fields.
* config_set_bool_field(name,var). */
} config_set_bool_field(
"rdbcompression", server.rdb_compression) {
} config_set_bool_field(
"repl-disable-tcp-nodelay",server.repl_disable_tcp_nodelay) {
} config_set_bool_field(
"repl-diskless-sync",server.repl_diskless_sync) {
} config_set_bool_field(
"cluster-require-full-coverage",server.cluster_require_full_coverage) {
} config_set_bool_field(
"cluster-slave-no-failover",server.cluster_slave_no_failover) {
} config_set_bool_field(
"cluster-replica-no-failover",server.cluster_slave_no_failover) {
} config_set_bool_field(
"aof-rewrite-incremental-fsync",server.aof_rewrite_incremental_fsync) {
} config_set_bool_field(
"rdb-save-incremental-fsync",server.rdb_save_incremental_fsync) {
} config_set_bool_field(
"aof-load-truncated",server.aof_load_truncated) {
} config_set_bool_field(
"aof-use-rdb-preamble",server.aof_use_rdb_preamble) {
} config_set_bool_field(
"slave-serve-stale-data",server.repl_serve_stale_data) {
} config_set_bool_field(
"replica-serve-stale-data",server.repl_serve_stale_data) {
} config_set_bool_field(
"slave-read-only",server.repl_slave_ro) {
} config_set_bool_field(
"replica-read-only",server.repl_slave_ro) {
} config_set_bool_field(
"slave-ignore-maxmemory",server.repl_slave_ignore_maxmemory) {
} config_set_bool_field(
"replica-ignore-maxmemory",server.repl_slave_ignore_maxmemory) {
} config_set_bool_field(
"activerehashing",server.activerehashing) {
} config_set_bool_field(
"activedefrag",server.active_defrag_enabled) {
#ifndef HAVE_DEFRAG
@ -1143,27 +1141,6 @@ void configSetCommand(client *c) {
return;
}
#endif
} config_set_bool_field(
"protected-mode",server.protected_mode) {
} config_set_bool_field(
"gopher-enabled",server.gopher_enabled) {
} config_set_bool_field(
"stop-writes-on-bgsave-error",server.stop_writes_on_bgsave_err) {
} config_set_bool_field(
"lazyfree-lazy-eviction",server.lazyfree_lazy_eviction) {
} config_set_bool_field(
"lazyfree-lazy-expire",server.lazyfree_lazy_expire) {
} config_set_bool_field(
"lazyfree-lazy-server-del",server.lazyfree_lazy_server_del) {
} config_set_bool_field(
"slave-lazy-flush",server.repl_slave_lazy_flush) {
} config_set_bool_field(
"replica-lazy-flush",server.repl_slave_lazy_flush) {
} config_set_bool_field(
"no-appendfsync-on-rewrite",server.aof_no_fsync_on_rewrite) {
} config_set_bool_field(
"dynamic-hz",server.dynamic_hz) {
/* Numerical fields.
* config_set_numerical_field(name,var,min,max) */
} config_set_numerical_field(
@ -1218,6 +1195,8 @@ void configSetCommand(client *c) {
"slowlog-max-len",ll,0,LONG_MAX) {
/* Cast to unsigned. */
server.slowlog_max_len = (unsigned long)ll;
} config_set_numerical_field(
"tracking-table-max-fill",server.tracking_table_max_fill,0,100) {
} config_set_numerical_field(
"latency-monitor-threshold",server.latency_monitor_threshold,0,LLONG_MAX){
} config_set_numerical_field(
@ -1234,6 +1213,10 @@ void configSetCommand(client *c) {
"slave-priority",server.slave_priority,0,INT_MAX) {
} config_set_numerical_field(
"replica-priority",server.slave_priority,0,INT_MAX) {
} config_set_numerical_field(
"rdb-key-save-delay",server.rdb_key_save_delay,0,LLONG_MAX) {
} config_set_numerical_field(
"key-load-delay",server.key_load_delay,0,LLONG_MAX) {
} config_set_numerical_field(
"slave-announce-port",server.slave_announce_port,0,65535) {
} config_set_numerical_field(
@ -1301,7 +1284,102 @@ void configSetCommand(client *c) {
"maxmemory-policy",server.maxmemory_policy,maxmemory_policy_enum) {
} config_set_enum_field(
"appendfsync",server.aof_fsync,aof_fsync_enum) {
} config_set_enum_field(
"repl-diskless-load",server.repl_diskless_load,repl_diskless_load_enum) {
#ifdef USE_OPENSSL
/* TLS fields. */
} config_set_special_field("tls-cert-file") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.cert_file = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-cert-file. Check server logs.");
return;
}
zfree(server.tls_ctx_config.cert_file);
server.tls_ctx_config.cert_file = zstrdup(o->ptr);
} config_set_special_field("tls-key-file") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.key_file = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-key-file. Check server logs.");
return;
}
zfree(server.tls_ctx_config.key_file);
server.tls_ctx_config.key_file = zstrdup(o->ptr);
} config_set_special_field("tls-dh-params-file") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.dh_params_file = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-dh-params-file. Check server logs.");
return;
}
zfree(server.tls_ctx_config.dh_params_file);
server.tls_ctx_config.dh_params_file = zstrdup(o->ptr);
} config_set_special_field("tls-ca-cert-file") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.ca_cert_file = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-ca-cert-file. Check server logs.");
return;
}
zfree(server.tls_ctx_config.ca_cert_file);
server.tls_ctx_config.ca_cert_file = zstrdup(o->ptr);
} config_set_special_field("tls-ca-cert-dir") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.ca_cert_dir = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-ca-cert-dir. Check server logs.");
return;
}
zfree(server.tls_ctx_config.ca_cert_dir);
server.tls_ctx_config.ca_cert_dir = zstrdup(o->ptr);
} config_set_bool_field("tls-auth-clients", server.tls_auth_clients) {
} config_set_bool_field("tls-replication", server.tls_replication) {
} config_set_bool_field("tls-cluster", server.tls_cluster) {
} config_set_special_field("tls-protocols") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.protocols = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-protocols. Check server logs.");
return;
}
zfree(server.tls_ctx_config.protocols);
server.tls_ctx_config.protocols = zstrdup(o->ptr);
} config_set_special_field("tls-ciphers") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.ciphers = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-ciphers. Check server logs.");
return;
}
zfree(server.tls_ctx_config.ciphers);
server.tls_ctx_config.ciphers = zstrdup(o->ptr);
} config_set_special_field("tls-ciphersuites") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.ciphersuites = (char *) o->ptr;
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c,
"Unable to configure tls-ciphersuites. Check server logs.");
return;
}
zfree(server.tls_ctx_config.ciphersuites);
server.tls_ctx_config.ciphersuites = zstrdup(o->ptr);
} config_set_special_field("tls-prefer-server-ciphers") {
redisTLSContextConfig tmpctx = server.tls_ctx_config;
tmpctx.prefer_server_ciphers = yesnotoi(o->ptr);
if (tlsConfigure(&tmpctx) == C_ERR) {
addReplyError(c, "Unable to reconfigure TLS. Check server logs.");
return;
}
server.tls_ctx_config.prefer_server_ciphers = tmpctx.prefer_server_ciphers;
#endif /* USE_OPENSSL */
/* Everyhing else is an error... */
} config_set_else {
addReplyErrorFormat(c,"Unsupported CONFIG parameter: %s",
@ -1375,6 +1453,16 @@ void configGetCommand(client *c) {
config_get_string_field("pidfile",server.pidfile);
config_get_string_field("slave-announce-ip",server.slave_announce_ip);
config_get_string_field("replica-announce-ip",server.slave_announce_ip);
#ifdef USE_OPENSSL
config_get_string_field("tls-cert-file",server.tls_ctx_config.cert_file);
config_get_string_field("tls-key-file",server.tls_ctx_config.key_file);
config_get_string_field("tls-dh-params-file",server.tls_ctx_config.dh_params_file);
config_get_string_field("tls-ca-cert-file",server.tls_ctx_config.ca_cert_file);
config_get_string_field("tls-ca-cert-dir",server.tls_ctx_config.ca_cert_dir);
config_get_string_field("tls-protocols",server.tls_ctx_config.protocols);
config_get_string_field("tls-ciphers",server.tls_ctx_config.ciphers);
config_get_string_field("tls-ciphersuites",server.tls_ctx_config.ciphersuites);
#endif
/* Numerical values */
config_get_numerical_field("maxmemory",server.maxmemory);
@ -1419,13 +1507,15 @@ void configGetCommand(client *c) {
server.slowlog_log_slower_than);
config_get_numerical_field("latency-monitor-threshold",
server.latency_monitor_threshold);
config_get_numerical_field("slowlog-max-len",
server.slowlog_max_len);
config_get_numerical_field("slowlog-max-len", server.slowlog_max_len);
config_get_numerical_field("tracking-table-max-fill", server.tracking_table_max_fill);
config_get_numerical_field("port",server.port);
config_get_numerical_field("tls-port",server.tls_port);
config_get_numerical_field("cluster-announce-port",server.cluster_announce_port);
config_get_numerical_field("cluster-announce-bus-port",server.cluster_announce_bus_port);
config_get_numerical_field("tcp-backlog",server.tcp_backlog);
config_get_numerical_field("databases",server.dbnum);
config_get_numerical_field("io-threads",server.io_threads_num);
config_get_numerical_field("repl-ping-slave-period",server.repl_ping_slave_period);
config_get_numerical_field("repl-ping-replica-period",server.repl_ping_slave_period);
config_get_numerical_field("repl-timeout",server.repl_timeout);
@ -1447,63 +1537,25 @@ void configGetCommand(client *c) {
config_get_numerical_field("cluster-slave-validity-factor",server.cluster_slave_validity_factor);
config_get_numerical_field("cluster-replica-validity-factor",server.cluster_slave_validity_factor);
config_get_numerical_field("repl-diskless-sync-delay",server.repl_diskless_sync_delay);
config_get_numerical_field("rdb-key-save-delay",server.rdb_key_save_delay);
config_get_numerical_field("key-load-delay",server.key_load_delay);
config_get_numerical_field("tcp-keepalive",server.tcpkeepalive);
/* Bool (yes/no) values */
config_get_bool_field("cluster-require-full-coverage",
server.cluster_require_full_coverage);
config_get_bool_field("cluster-slave-no-failover",
server.cluster_slave_no_failover);
config_get_bool_field("cluster-replica-no-failover",
server.cluster_slave_no_failover);
config_get_bool_field("no-appendfsync-on-rewrite",
server.aof_no_fsync_on_rewrite);
config_get_bool_field("slave-serve-stale-data",
server.repl_serve_stale_data);
config_get_bool_field("replica-serve-stale-data",
server.repl_serve_stale_data);
config_get_bool_field("slave-read-only",
server.repl_slave_ro);
config_get_bool_field("replica-read-only",
server.repl_slave_ro);
config_get_bool_field("slave-ignore-maxmemory",
server.repl_slave_ignore_maxmemory);
config_get_bool_field("replica-ignore-maxmemory",
server.repl_slave_ignore_maxmemory);
config_get_bool_field("stop-writes-on-bgsave-error",
server.stop_writes_on_bgsave_err);
config_get_bool_field("daemonize", server.daemonize);
config_get_bool_field("rdbcompression", server.rdb_compression);
config_get_bool_field("rdbchecksum", server.rdb_checksum);
config_get_bool_field("activerehashing", server.activerehashing);
config_get_bool_field("activedefrag", server.active_defrag_enabled);
config_get_bool_field("protected-mode", server.protected_mode);
config_get_bool_field("gopher-enabled", server.gopher_enabled);
config_get_bool_field("repl-disable-tcp-nodelay",
server.repl_disable_tcp_nodelay);
config_get_bool_field("repl-diskless-sync",
server.repl_diskless_sync);
config_get_bool_field("aof-rewrite-incremental-fsync",
server.aof_rewrite_incremental_fsync);
config_get_bool_field("rdb-save-incremental-fsync",
server.rdb_save_incremental_fsync);
config_get_bool_field("aof-load-truncated",
server.aof_load_truncated);
config_get_bool_field("aof-use-rdb-preamble",
server.aof_use_rdb_preamble);
config_get_bool_field("lazyfree-lazy-eviction",
server.lazyfree_lazy_eviction);
config_get_bool_field("lazyfree-lazy-expire",
server.lazyfree_lazy_expire);
config_get_bool_field("lazyfree-lazy-server-del",
server.lazyfree_lazy_server_del);
config_get_bool_field("slave-lazy-flush",
server.repl_slave_lazy_flush);
config_get_bool_field("replica-lazy-flush",
server.repl_slave_lazy_flush);
config_get_bool_field("dynamic-hz",
server.dynamic_hz);
/* Iterate the configs that are standard */
for (configYesNo *config = configs_yesno; config->name != NULL; config++) {
config_get_bool_field(config->name, *(config->config));
if (config->alias) {
config_get_bool_field(config->alias, *(config->config));
}
}
config_get_bool_field("activedefrag", server.active_defrag_enabled);
config_get_bool_field("tls-cluster",server.tls_cluster);
config_get_bool_field("tls-replication",server.tls_replication);
config_get_bool_field("tls-auth-clients",server.tls_auth_clients);
config_get_bool_field("tls-prefer-server-ciphers",
server.tls_ctx_config.prefer_server_ciphers);
/* Enum values */
config_get_enum_field("maxmemory-policy",
server.maxmemory_policy,maxmemory_policy_enum);
@ -1515,12 +1567,14 @@ void configGetCommand(client *c) {
server.aof_fsync,aof_fsync_enum);
config_get_enum_field("syslog-facility",
server.syslog_facility,syslog_facility_enum);
config_get_enum_field("repl-diskless-load",
server.repl_diskless_load,repl_diskless_load_enum);
/* Everything we can't handle with macros follows. */
if (stringmatch(pattern,"appendonly",1)) {
addReplyBulkCString(c,"appendonly");
addReplyBulkCString(c,server.aof_state == AOF_OFF ? "no" : "yes");
addReplyBulkCString(c,server.aof_enabled ? "yes" : "no");
matches++;
}
if (stringmatch(pattern,"dir",1)) {
@ -1591,12 +1645,10 @@ void configGetCommand(client *c) {
matches++;
}
if (stringmatch(pattern,"notify-keyspace-events",1)) {
robj *flagsobj = createObject(OBJ_STRING,
keyspaceEventsFlagsToString(server.notify_keyspace_events));
sds flags = keyspaceEventsFlagsToString(server.notify_keyspace_events);
addReplyBulkCString(c,"notify-keyspace-events");
addReplyBulk(c,flagsobj);
decrRefCount(flagsobj);
addReplyBulkSds(c,flags);
matches++;
}
if (stringmatch(pattern,"bind",1)) {
@ -1617,6 +1669,7 @@ void configGetCommand(client *c) {
}
matches++;
}
setDeferredMapLen(c,replylen,matches);
}
@ -1700,12 +1753,11 @@ void rewriteConfigMarkAsProcessed(struct rewriteConfigState *state, const char *
* If the old file does not exist at all, an empty state is returned. */
struct rewriteConfigState *rewriteConfigReadOldFile(char *path) {
FILE *fp = fopen(path,"r");
struct rewriteConfigState *state = zmalloc(sizeof(*state));
char buf[CONFIG_MAX_LINE+1];
int linenum = -1;
if (fp == NULL && errno != ENOENT) return NULL;
char buf[CONFIG_MAX_LINE+1];
int linenum = -1;
struct rewriteConfigState *state = zmalloc(sizeof(*state));
state->option_to_line = dictCreate(&optionToLineDictType,NULL);
state->rewritten = dictCreate(&optionSetDictType,NULL);
state->numlines = 0;
@ -1848,7 +1900,7 @@ void rewriteConfigBytesOption(struct rewriteConfigState *state, char *option, lo
}
/* Rewrite a yes/no option. */
void rewriteConfigYesNoOption(struct rewriteConfigState *state, char *option, int value, int defvalue) {
void rewriteConfigYesNoOption(struct rewriteConfigState *state, const char *option, int value, int defvalue) {
int force = value != defvalue;
sds line = sdscatprintf(sdsempty(),"%s %s",option,
value ? "yes" : "no");
@ -2218,9 +2270,13 @@ int rewriteConfig(char *path) {
/* Step 2: rewrite every single option, replacing or appending it inside
* the rewrite state. */
rewriteConfigYesNoOption(state,"daemonize",server.daemonize,0);
/* Iterate the configs that are standard */
for (configYesNo *config = configs_yesno; config->name != NULL; config++) {
rewriteConfigYesNoOption(state,config->name,*(config->config),config->default_value);
}
rewriteConfigStringOption(state,"pidfile",server.pidfile,CONFIG_DEFAULT_PID_FILE);
rewriteConfigNumericalOption(state,"port",server.port,CONFIG_DEFAULT_SERVER_PORT);
rewriteConfigNumericalOption(state,"tls-port",server.tls_port,CONFIG_DEFAULT_SERVER_TLS_PORT);
rewriteConfigNumericalOption(state,"cluster-announce-port",server.cluster_announce_port,CONFIG_DEFAULT_CLUSTER_ANNOUNCE_PORT);
rewriteConfigNumericalOption(state,"cluster-announce-bus-port",server.cluster_announce_bus_port,CONFIG_DEFAULT_CLUSTER_ANNOUNCE_BUS_PORT);
rewriteConfigNumericalOption(state,"tcp-backlog",server.tcp_backlog,CONFIG_DEFAULT_TCP_BACKLOG);
@ -2239,9 +2295,7 @@ int rewriteConfig(char *path) {
rewriteConfigSaveOption(state);
rewriteConfigUserOption(state);
rewriteConfigNumericalOption(state,"databases",server.dbnum,CONFIG_DEFAULT_DBNUM);
rewriteConfigYesNoOption(state,"stop-writes-on-bgsave-error",server.stop_writes_on_bgsave_err,CONFIG_DEFAULT_STOP_WRITES_ON_BGSAVE_ERROR);
rewriteConfigYesNoOption(state,"rdbcompression",server.rdb_compression,CONFIG_DEFAULT_RDB_COMPRESSION);
rewriteConfigYesNoOption(state,"rdbchecksum",server.rdb_checksum,CONFIG_DEFAULT_RDB_CHECKSUM);
rewriteConfigNumericalOption(state,"io-threads",server.dbnum,CONFIG_DEFAULT_IO_THREADS_NUM);
rewriteConfigStringOption(state,"dbfilename",server.rdb_filename,CONFIG_DEFAULT_RDB_FILENAME);
rewriteConfigDirOption(state);
rewriteConfigSlaveofOption(state,"replicaof");
@ -2249,15 +2303,11 @@ int rewriteConfig(char *path) {
rewriteConfigStringOption(state,"masteruser",server.masteruser,NULL);
rewriteConfigStringOption(state,"masterauth",server.masterauth,NULL);
rewriteConfigStringOption(state,"cluster-announce-ip",server.cluster_announce_ip,NULL);
rewriteConfigYesNoOption(state,"replica-serve-stale-data",server.repl_serve_stale_data,CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA);
rewriteConfigYesNoOption(state,"replica-read-only",server.repl_slave_ro,CONFIG_DEFAULT_SLAVE_READ_ONLY);
rewriteConfigYesNoOption(state,"replica-ignore-maxmemory",server.repl_slave_ignore_maxmemory,CONFIG_DEFAULT_SLAVE_IGNORE_MAXMEMORY);
rewriteConfigNumericalOption(state,"repl-ping-replica-period",server.repl_ping_slave_period,CONFIG_DEFAULT_REPL_PING_SLAVE_PERIOD);
rewriteConfigNumericalOption(state,"repl-timeout",server.repl_timeout,CONFIG_DEFAULT_REPL_TIMEOUT);
rewriteConfigBytesOption(state,"repl-backlog-size",server.repl_backlog_size,CONFIG_DEFAULT_REPL_BACKLOG_SIZE);
rewriteConfigBytesOption(state,"repl-backlog-ttl",server.repl_backlog_time_limit,CONFIG_DEFAULT_REPL_BACKLOG_TIME_LIMIT);
rewriteConfigYesNoOption(state,"repl-disable-tcp-nodelay",server.repl_disable_tcp_nodelay,CONFIG_DEFAULT_REPL_DISABLE_TCP_NODELAY);
rewriteConfigYesNoOption(state,"repl-diskless-sync",server.repl_diskless_sync,CONFIG_DEFAULT_REPL_DISKLESS_SYNC);
rewriteConfigEnumOption(state,"repl-diskless-load",server.repl_diskless_load,repl_diskless_load_enum,CONFIG_DEFAULT_REPL_DISKLESS_LOAD);
rewriteConfigNumericalOption(state,"repl-diskless-sync-delay",server.repl_diskless_sync_delay,CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY);
rewriteConfigNumericalOption(state,"replica-priority",server.slave_priority,CONFIG_DEFAULT_SLAVE_PRIORITY);
rewriteConfigNumericalOption(state,"min-replicas-to-write",server.repl_min_slaves_to_write,CONFIG_DEFAULT_MIN_SLAVES_TO_WRITE);
@ -2277,23 +2327,21 @@ int rewriteConfig(char *path) {
rewriteConfigNumericalOption(state,"active-defrag-cycle-min",server.active_defrag_cycle_min,CONFIG_DEFAULT_DEFRAG_CYCLE_MIN);
rewriteConfigNumericalOption(state,"active-defrag-cycle-max",server.active_defrag_cycle_max,CONFIG_DEFAULT_DEFRAG_CYCLE_MAX);
rewriteConfigNumericalOption(state,"active-defrag-max-scan-fields",server.active_defrag_max_scan_fields,CONFIG_DEFAULT_DEFRAG_MAX_SCAN_FIELDS);
rewriteConfigYesNoOption(state,"appendonly",server.aof_state != AOF_OFF,0);
rewriteConfigYesNoOption(state,"appendonly",server.aof_enabled,0);
rewriteConfigStringOption(state,"appendfilename",server.aof_filename,CONFIG_DEFAULT_AOF_FILENAME);
rewriteConfigEnumOption(state,"appendfsync",server.aof_fsync,aof_fsync_enum,CONFIG_DEFAULT_AOF_FSYNC);
rewriteConfigYesNoOption(state,"no-appendfsync-on-rewrite",server.aof_no_fsync_on_rewrite,CONFIG_DEFAULT_AOF_NO_FSYNC_ON_REWRITE);
rewriteConfigNumericalOption(state,"auto-aof-rewrite-percentage",server.aof_rewrite_perc,AOF_REWRITE_PERC);
rewriteConfigBytesOption(state,"auto-aof-rewrite-min-size",server.aof_rewrite_min_size,AOF_REWRITE_MIN_SIZE);
rewriteConfigNumericalOption(state,"lua-time-limit",server.lua_time_limit,LUA_SCRIPT_TIME_LIMIT);
rewriteConfigYesNoOption(state,"cluster-enabled",server.cluster_enabled,0);
rewriteConfigStringOption(state,"cluster-config-file",server.cluster_configfile,CONFIG_DEFAULT_CLUSTER_CONFIG_FILE);
rewriteConfigYesNoOption(state,"cluster-require-full-coverage",server.cluster_require_full_coverage,CLUSTER_DEFAULT_REQUIRE_FULL_COVERAGE);
rewriteConfigYesNoOption(state,"cluster-replica-no-failover",server.cluster_slave_no_failover,CLUSTER_DEFAULT_SLAVE_NO_FAILOVER);
rewriteConfigNumericalOption(state,"cluster-node-timeout",server.cluster_node_timeout,CLUSTER_DEFAULT_NODE_TIMEOUT);
rewriteConfigNumericalOption(state,"cluster-migration-barrier",server.cluster_migration_barrier,CLUSTER_DEFAULT_MIGRATION_BARRIER);
rewriteConfigNumericalOption(state,"cluster-replica-validity-factor",server.cluster_slave_validity_factor,CLUSTER_DEFAULT_SLAVE_VALIDITY);
rewriteConfigNumericalOption(state,"slowlog-log-slower-than",server.slowlog_log_slower_than,CONFIG_DEFAULT_SLOWLOG_LOG_SLOWER_THAN);
rewriteConfigNumericalOption(state,"latency-monitor-threshold",server.latency_monitor_threshold,CONFIG_DEFAULT_LATENCY_MONITOR_THRESHOLD);
rewriteConfigNumericalOption(state,"slowlog-max-len",server.slowlog_max_len,CONFIG_DEFAULT_SLOWLOG_MAX_LEN);
rewriteConfigNumericalOption(state,"tracking-table-max-fill",server.tracking_table_max_fill,CONFIG_DEFAULT_TRACKING_TABLE_MAX_FILL);
rewriteConfigNotifykeyspaceeventsOption(state);
rewriteConfigNumericalOption(state,"hash-max-ziplist-entries",server.hash_max_ziplist_entries,OBJ_HASH_MAX_ZIPLIST_ENTRIES);
rewriteConfigNumericalOption(state,"hash-max-ziplist-value",server.hash_max_ziplist_value,OBJ_HASH_MAX_ZIPLIST_VALUE);
@ -2305,22 +2353,26 @@ int rewriteConfig(char *path) {
rewriteConfigNumericalOption(state,"zset-max-ziplist-entries",server.zset_max_ziplist_entries,OBJ_ZSET_MAX_ZIPLIST_ENTRIES);
rewriteConfigNumericalOption(state,"zset-max-ziplist-value",server.zset_max_ziplist_value,OBJ_ZSET_MAX_ZIPLIST_VALUE);
rewriteConfigNumericalOption(state,"hll-sparse-max-bytes",server.hll_sparse_max_bytes,CONFIG_DEFAULT_HLL_SPARSE_MAX_BYTES);
rewriteConfigYesNoOption(state,"activerehashing",server.activerehashing,CONFIG_DEFAULT_ACTIVE_REHASHING);
rewriteConfigYesNoOption(state,"activedefrag",server.active_defrag_enabled,CONFIG_DEFAULT_ACTIVE_DEFRAG);
rewriteConfigYesNoOption(state,"protected-mode",server.protected_mode,CONFIG_DEFAULT_PROTECTED_MODE);
rewriteConfigYesNoOption(state,"gopher-enabled",server.gopher_enabled,CONFIG_DEFAULT_GOPHER_ENABLED);
rewriteConfigClientoutputbufferlimitOption(state);
rewriteConfigNumericalOption(state,"hz",server.config_hz,CONFIG_DEFAULT_HZ);
rewriteConfigYesNoOption(state,"aof-rewrite-incremental-fsync",server.aof_rewrite_incremental_fsync,CONFIG_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC);
rewriteConfigYesNoOption(state,"rdb-save-incremental-fsync",server.rdb_save_incremental_fsync,CONFIG_DEFAULT_RDB_SAVE_INCREMENTAL_FSYNC);
rewriteConfigYesNoOption(state,"aof-load-truncated",server.aof_load_truncated,CONFIG_DEFAULT_AOF_LOAD_TRUNCATED);
rewriteConfigYesNoOption(state,"aof-use-rdb-preamble",server.aof_use_rdb_preamble,CONFIG_DEFAULT_AOF_USE_RDB_PREAMBLE);
rewriteConfigEnumOption(state,"supervised",server.supervised_mode,supervised_mode_enum,SUPERVISED_NONE);
rewriteConfigYesNoOption(state,"lazyfree-lazy-eviction",server.lazyfree_lazy_eviction,CONFIG_DEFAULT_LAZYFREE_LAZY_EVICTION);
rewriteConfigYesNoOption(state,"lazyfree-lazy-expire",server.lazyfree_lazy_expire,CONFIG_DEFAULT_LAZYFREE_LAZY_EXPIRE);
rewriteConfigYesNoOption(state,"lazyfree-lazy-server-del",server.lazyfree_lazy_server_del,CONFIG_DEFAULT_LAZYFREE_LAZY_SERVER_DEL);
rewriteConfigYesNoOption(state,"replica-lazy-flush",server.repl_slave_lazy_flush,CONFIG_DEFAULT_SLAVE_LAZY_FLUSH);
rewriteConfigYesNoOption(state,"dynamic-hz",server.dynamic_hz,CONFIG_DEFAULT_DYNAMIC_HZ);
rewriteConfigNumericalOption(state,"rdb-key-save-delay",server.rdb_key_save_delay,CONFIG_DEFAULT_RDB_KEY_SAVE_DELAY);
rewriteConfigNumericalOption(state,"key-load-delay",server.key_load_delay,CONFIG_DEFAULT_KEY_LOAD_DELAY);
#ifdef USE_OPENSSL
rewriteConfigYesNoOption(state,"tls-cluster",server.tls_cluster,0);
rewriteConfigYesNoOption(state,"tls-replication",server.tls_replication,0);
rewriteConfigYesNoOption(state,"tls-auth-clients",server.tls_auth_clients,1);
rewriteConfigStringOption(state,"tls-cert-file",server.tls_ctx_config.cert_file,NULL);
rewriteConfigStringOption(state,"tls-key-file",server.tls_ctx_config.key_file,NULL);
rewriteConfigStringOption(state,"tls-dh-params-file",server.tls_ctx_config.dh_params_file,NULL);
rewriteConfigStringOption(state,"tls-ca-cert-file",server.tls_ctx_config.ca_cert_file,NULL);
rewriteConfigStringOption(state,"tls-ca-cert-dir",server.tls_ctx_config.ca_cert_dir,NULL);
rewriteConfigStringOption(state,"tls-protocols",server.tls_ctx_config.protocols,NULL);
rewriteConfigStringOption(state,"tls-ciphers",server.tls_ctx_config.ciphers,NULL);
rewriteConfigStringOption(state,"tls-ciphersuites",server.tls_ctx_config.ciphersuites,NULL);
rewriteConfigYesNoOption(state,"tls-prefer-server-ciphers",server.tls_ctx_config.prefer_server_ciphers,0);
#endif
/* Rewrite Sentinel config if in Sentinel mode. */
if (server.sentinel_mode) rewriteConfigSentinelOption(state);

407
src/connection.c Normal file
View File

@ -0,0 +1,407 @@
/*
* Copyright (c) 2019, Redis Labs
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "server.h"
#include "connhelpers.h"
/* The connections module provides a lean abstraction of network connections
* to avoid direct socket and async event management across the Redis code base.
*
* It does NOT provide advanced connection features commonly found in similar
* libraries such as complete in/out buffer management, throttling, etc. These
* functions remain in networking.c.
*
* The primary goal is to allow transparent handling of TCP and TLS based
* connections. To do so, connections have the following properties:
*
* 1. A connection may live before its corresponding socket exists. This
* allows various context and configuration setting to be handled before
* establishing the actual connection.
* 2. The caller may register/unregister logical read/write handlers to be
* called when the connection has data to read from/can accept writes.
* These logical handlers may or may not correspond to actual AE events,
* depending on the implementation (for TCP they are; for TLS they aren't).
*/
ConnectionType CT_Socket;
/* When a connection is created we must know its type already, but the
* underlying socket may or may not exist:
*
* - For accepted connections, it exists as we do not model the listen/accept
* part; So caller calls connCreateSocket() followed by connAccept().
* - For outgoing connections, the socket is created by the connection module
* itself; So caller calls connCreateSocket() followed by connConnect(),
* which registers a connect callback that fires on connected/error state
* (and after any transport level handshake was done).
*
* NOTE: An earlier version relied on connections being part of other structs
* and not independently allocated. This could lead to further optimizations
* like using container_of(), etc. However it was discontinued in favor of
* this approach for these reasons:
*
* 1. In some cases conns are created/handled outside the context of the
* containing struct, in which case it gets a bit awkward to copy them.
* 2. Future implementations may wish to allocate arbitrary data for the
* connection.
* 3. The container_of() approach is anyway risky because connections may
* be embedded in different structs, not just client.
*/
connection *connCreateSocket() {
connection *conn = zcalloc(sizeof(connection));
conn->type = &CT_Socket;
conn->fd = -1;
return conn;
}
/* Create a new socket-type connection that is already associated with
* an accepted connection.
*
* The socket is not read for I/O until connAccept() was called and
* invoked the connection-level accept handler.
*/
connection *connCreateAcceptedSocket(int fd) {
connection *conn = connCreateSocket();
conn->fd = fd;
conn->state = CONN_STATE_ACCEPTING;
return conn;
}
static int connSocketConnect(connection *conn, const char *addr, int port, const char *src_addr,
ConnectionCallbackFunc connect_handler) {
int fd = anetTcpNonBlockBestEffortBindConnect(NULL,addr,port,src_addr);
if (fd == -1) {
conn->state = CONN_STATE_ERROR;
conn->last_errno = errno;
return C_ERR;
}
conn->fd = fd;
conn->state = CONN_STATE_CONNECTING;
conn->conn_handler = connect_handler;
aeCreateFileEvent(server.el, conn->fd, AE_WRITABLE,
conn->type->ae_handler, conn);
return C_OK;
}
/* Returns true if a write handler is registered */
int connHasWriteHandler(connection *conn) {
return conn->write_handler != NULL;
}
/* Returns true if a read handler is registered */
int connHasReadHandler(connection *conn) {
return conn->read_handler != NULL;
}
/* Associate a private data pointer with the connection */
void connSetPrivateData(connection *conn, void *data) {
conn->private_data = data;
}
/* Get the associated private data pointer */
void *connGetPrivateData(connection *conn) {
return conn->private_data;
}
/* ------ Pure socket connections ------- */
/* A very incomplete list of implementation-specific calls. Much of the above shall
* move here as we implement additional connection types.
*/
/* Close the connection and free resources. */
static void connSocketClose(connection *conn) {
if (conn->fd != -1) {
aeDeleteFileEvent(server.el,conn->fd,AE_READABLE);
aeDeleteFileEvent(server.el,conn->fd,AE_WRITABLE);
close(conn->fd);
conn->fd = -1;
}
/* If called from within a handler, schedule the close but
* keep the connection until the handler returns.
*/
if (conn->flags & CONN_FLAG_IN_HANDLER) {
conn->flags |= CONN_FLAG_CLOSE_SCHEDULED;
return;
}
zfree(conn);
}
static int connSocketWrite(connection *conn, const void *data, size_t data_len) {
int ret = write(conn->fd, data, data_len);
if (ret < 0 && errno != EAGAIN) {
conn->last_errno = errno;
conn->state = CONN_STATE_ERROR;
}
return ret;
}
static int connSocketRead(connection *conn, void *buf, size_t buf_len) {
int ret = read(conn->fd, buf, buf_len);
if (!ret) {
conn->state = CONN_STATE_CLOSED;
} else if (ret < 0 && errno != EAGAIN) {
conn->last_errno = errno;
conn->state = CONN_STATE_ERROR;
}
return ret;
}
static int connSocketAccept(connection *conn, ConnectionCallbackFunc accept_handler) {
if (conn->state != CONN_STATE_ACCEPTING) return C_ERR;
conn->state = CONN_STATE_CONNECTED;
if (!callHandler(conn, accept_handler)) return C_ERR;
return C_OK;
}
/* Register a write handler, to be called when the connection is writable.
* If NULL, the existing handler is removed.
*
* The barrier flag indicates a write barrier is requested, resulting with
* CONN_FLAG_WRITE_BARRIER set. This will ensure that the write handler is
* always called before and not after the read handler in a single event
* loop.
*/
static int connSocketSetWriteHandler(connection *conn, ConnectionCallbackFunc func, int barrier) {
if (func == conn->write_handler) return C_OK;
conn->write_handler = func;
if (barrier)
conn->flags |= CONN_FLAG_WRITE_BARRIER;
else
conn->flags &= ~CONN_FLAG_WRITE_BARRIER;
if (!conn->write_handler)
aeDeleteFileEvent(server.el,conn->fd,AE_WRITABLE);
else
if (aeCreateFileEvent(server.el,conn->fd,AE_WRITABLE,
conn->type->ae_handler,conn) == AE_ERR) return C_ERR;
return C_OK;
}
/* Register a read handler, to be called when the connection is readable.
* If NULL, the existing handler is removed.
*/
static int connSocketSetReadHandler(connection *conn, ConnectionCallbackFunc func) {
if (func == conn->read_handler) return C_OK;
conn->read_handler = func;
if (!conn->read_handler)
aeDeleteFileEvent(server.el,conn->fd,AE_READABLE);
else
if (aeCreateFileEvent(server.el,conn->fd,
AE_READABLE,conn->type->ae_handler,conn) == AE_ERR) return C_ERR;
return C_OK;
}
static const char *connSocketGetLastError(connection *conn) {
return strerror(conn->last_errno);
}
static void connSocketEventHandler(struct aeEventLoop *el, int fd, void *clientData, int mask)
{
UNUSED(el);
UNUSED(fd);
connection *conn = clientData;
if (conn->state == CONN_STATE_CONNECTING &&
(mask & AE_WRITABLE) && conn->conn_handler) {
if (connGetSocketError(conn)) {
conn->last_errno = errno;
conn->state = CONN_STATE_ERROR;
} else {
conn->state = CONN_STATE_CONNECTED;
}
if (!conn->write_handler) aeDeleteFileEvent(server.el,conn->fd,AE_WRITABLE);
if (!callHandler(conn, conn->conn_handler)) return;
conn->conn_handler = NULL;
}
/* Normally we execute the readable event first, and the writable
* event later. This is useful as sometimes we may be able
* to serve the reply of a query immediately after processing the
* query.
*
* However if WRITE_BARRIER is set in the mask, our application is
* asking us to do the reverse: never fire the writable event
* after the readable. In such a case, we invert the calls.
* This is useful when, for instance, we want to do things
* in the beforeSleep() hook, like fsync'ing a file to disk,
* before replying to a client. */
int invert = conn->flags & CONN_FLAG_WRITE_BARRIER;
int call_write = (mask & AE_WRITABLE) && conn->write_handler;
int call_read = (mask & AE_READABLE) && conn->read_handler;
/* Handle normal I/O flows */
if (!invert && call_read) {
if (!callHandler(conn, conn->read_handler)) return;
}
/* Fire the writable event. */
if (call_write) {
if (!callHandler(conn, conn->write_handler)) return;
}
/* If we have to invert the call, fire the readable event now
* after the writable one. */
if (invert && call_read) {
if (!callHandler(conn, conn->read_handler)) return;
}
}
static int connSocketBlockingConnect(connection *conn, const char *addr, int port, long long timeout) {
int fd = anetTcpNonBlockConnect(NULL,addr,port);
if (fd == -1) {
conn->state = CONN_STATE_ERROR;
conn->last_errno = errno;
return C_ERR;
}
if ((aeWait(fd, AE_WRITABLE, timeout) & AE_WRITABLE) == 0) {
conn->state = CONN_STATE_ERROR;
conn->last_errno = ETIMEDOUT;
}
conn->fd = fd;
conn->state = CONN_STATE_CONNECTED;
return C_OK;
}
/* Connection-based versions of syncio.c functions.
* NOTE: This should ideally be refactored out in favor of pure async work.
*/
static ssize_t connSocketSyncWrite(connection *conn, char *ptr, ssize_t size, long long timeout) {
return syncWrite(conn->fd, ptr, size, timeout);
}
static ssize_t connSocketSyncRead(connection *conn, char *ptr, ssize_t size, long long timeout) {
return syncRead(conn->fd, ptr, size, timeout);
}
static ssize_t connSocketSyncReadLine(connection *conn, char *ptr, ssize_t size, long long timeout) {
return syncReadLine(conn->fd, ptr, size, timeout);
}
ConnectionType CT_Socket = {
.ae_handler = connSocketEventHandler,
.close = connSocketClose,
.write = connSocketWrite,
.read = connSocketRead,
.accept = connSocketAccept,
.connect = connSocketConnect,
.set_write_handler = connSocketSetWriteHandler,
.set_read_handler = connSocketSetReadHandler,
.get_last_error = connSocketGetLastError,
.blocking_connect = connSocketBlockingConnect,
.sync_write = connSocketSyncWrite,
.sync_read = connSocketSyncRead,
.sync_readline = connSocketSyncReadLine
};
int connGetSocketError(connection *conn) {
int sockerr = 0;
socklen_t errlen = sizeof(sockerr);
if (getsockopt(conn->fd, SOL_SOCKET, SO_ERROR, &sockerr, &errlen) == -1)
sockerr = errno;
return sockerr;
}
int connPeerToString(connection *conn, char *ip, size_t ip_len, int *port) {
return anetPeerToString(conn ? conn->fd : -1, ip, ip_len, port);
}
int connFormatPeer(connection *conn, char *buf, size_t buf_len) {
return anetFormatPeer(conn ? conn->fd : -1, buf, buf_len);
}
int connSockName(connection *conn, char *ip, size_t ip_len, int *port) {
return anetSockName(conn->fd, ip, ip_len, port);
}
int connBlock(connection *conn) {
if (conn->fd == -1) return C_ERR;
return anetBlock(NULL, conn->fd);
}
int connNonBlock(connection *conn) {
if (conn->fd == -1) return C_ERR;
return anetNonBlock(NULL, conn->fd);
}
int connEnableTcpNoDelay(connection *conn) {
if (conn->fd == -1) return C_ERR;
return anetEnableTcpNoDelay(NULL, conn->fd);
}
int connDisableTcpNoDelay(connection *conn) {
if (conn->fd == -1) return C_ERR;
return anetDisableTcpNoDelay(NULL, conn->fd);
}
int connKeepAlive(connection *conn, int interval) {
if (conn->fd == -1) return C_ERR;
return anetKeepAlive(NULL, conn->fd, interval);
}
int connSendTimeout(connection *conn, long long ms) {
return anetSendTimeout(NULL, conn->fd, ms);
}
int connRecvTimeout(connection *conn, long long ms) {
return anetRecvTimeout(NULL, conn->fd, ms);
}
int connGetState(connection *conn) {
return conn->state;
}
/* Return a text that describes the connection, suitable for inclusion
* in CLIENT LIST and similar outputs.
*
* For sockets, we always return "fd=<fdnum>" to maintain compatibility.
*/
const char *connGetInfo(connection *conn, char *buf, size_t buf_len) {
snprintf(buf, buf_len-1, "fd=%i", conn->fd);
return buf;
}

220
src/connection.h Normal file
View File

@ -0,0 +1,220 @@
/*
* Copyright (c) 2019, Redis Labs
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __REDIS_CONNECTION_H
#define __REDIS_CONNECTION_H
#define CONN_INFO_LEN 32
struct aeEventLoop;
typedef struct connection connection;
typedef enum {
CONN_STATE_NONE = 0,
CONN_STATE_CONNECTING,
CONN_STATE_ACCEPTING,
CONN_STATE_CONNECTED,
CONN_STATE_CLOSED,
CONN_STATE_ERROR
} ConnectionState;
#define CONN_FLAG_IN_HANDLER (1<<0) /* A handler execution is in progress */
#define CONN_FLAG_CLOSE_SCHEDULED (1<<1) /* Closed scheduled by a handler */
#define CONN_FLAG_WRITE_BARRIER (1<<2) /* Write barrier requested */
typedef void (*ConnectionCallbackFunc)(struct connection *conn);
typedef struct ConnectionType {
void (*ae_handler)(struct aeEventLoop *el, int fd, void *clientData, int mask);
int (*connect)(struct connection *conn, const char *addr, int port, const char *source_addr, ConnectionCallbackFunc connect_handler);
int (*write)(struct connection *conn, const void *data, size_t data_len);
int (*read)(struct connection *conn, void *buf, size_t buf_len);
void (*close)(struct connection *conn);
int (*accept)(struct connection *conn, ConnectionCallbackFunc accept_handler);
int (*set_write_handler)(struct connection *conn, ConnectionCallbackFunc handler, int barrier);
int (*set_read_handler)(struct connection *conn, ConnectionCallbackFunc handler);
const char *(*get_last_error)(struct connection *conn);
int (*blocking_connect)(struct connection *conn, const char *addr, int port, long long timeout);
ssize_t (*sync_write)(struct connection *conn, char *ptr, ssize_t size, long long timeout);
ssize_t (*sync_read)(struct connection *conn, char *ptr, ssize_t size, long long timeout);
ssize_t (*sync_readline)(struct connection *conn, char *ptr, ssize_t size, long long timeout);
} ConnectionType;
struct connection {
ConnectionType *type;
ConnectionState state;
int flags;
int last_errno;
void *private_data;
ConnectionCallbackFunc conn_handler;
ConnectionCallbackFunc write_handler;
ConnectionCallbackFunc read_handler;
int fd;
};
/* The connection module does not deal with listening and accepting sockets,
* so we assume we have a socket when an incoming connection is created.
*
* The fd supplied should therefore be associated with an already accept()ed
* socket.
*
* connAccept() may directly call accept_handler(), or return and call it
* at a later time. This behavior is a bit awkward but aims to reduce the need
* to wait for the next event loop, if no additional handshake is required.
*/
static inline int connAccept(connection *conn, ConnectionCallbackFunc accept_handler) {
return conn->type->accept(conn, accept_handler);
}
/* Establish a connection. The connect_handler will be called when the connection
* is established, or if an error has occured.
*
* The connection handler will be responsible to set up any read/write handlers
* as needed.
*
* If C_ERR is returned, the operation failed and the connection handler shall
* not be expected.
*/
static inline int connConnect(connection *conn, const char *addr, int port, const char *src_addr,
ConnectionCallbackFunc connect_handler) {
return conn->type->connect(conn, addr, port, src_addr, connect_handler);
}
/* Blocking connect.
*
* NOTE: This is implemented in order to simplify the transition to the abstract
* connections, but should probably be refactored out of cluster.c and replication.c,
* in favor of a pure async implementation.
*/
static inline int connBlockingConnect(connection *conn, const char *addr, int port, long long timeout) {
return conn->type->blocking_connect(conn, addr, port, timeout);
}
/* Write to connection, behaves the same as write(2).
*
* Like write(2), a short write is possible. A -1 return indicates an error.
*
* The caller should NOT rely on errno. Testing for an EAGAIN-like condition, use
* connGetState() to see if the connection state is still CONN_STATE_CONNECTED.
*/
static inline int connWrite(connection *conn, const void *data, size_t data_len) {
return conn->type->write(conn, data, data_len);
}
/* Read from the connection, behaves the same as read(2).
*
* Like read(2), a short read is possible. A return value of 0 will indicate the
* connection was closed, and -1 will indicate an error.
*
* The caller should NOT rely on errno. Testing for an EAGAIN-like condition, use
* connGetState() to see if the connection state is still CONN_STATE_CONNECTED.
*/
static inline int connRead(connection *conn, void *buf, size_t buf_len) {
return conn->type->read(conn, buf, buf_len);
}
/* Register a write handler, to be called when the connection is writable.
* If NULL, the existing handler is removed.
*/
static inline int connSetWriteHandler(connection *conn, ConnectionCallbackFunc func) {
return conn->type->set_write_handler(conn, func, 0);
}
/* Register a read handler, to be called when the connection is readable.
* If NULL, the existing handler is removed.
*/
static inline int connSetReadHandler(connection *conn, ConnectionCallbackFunc func) {
return conn->type->set_read_handler(conn, func);
}
/* Set a write handler, and possibly enable a write barrier, this flag is
* cleared when write handler is changed or removed.
* With barroer enabled, we never fire the event if the read handler already
* fired in the same event loop iteration. Useful when you want to persist
* things to disk before sending replies, and want to do that in a group fashion. */
static inline int connSetWriteHandlerWithBarrier(connection *conn, ConnectionCallbackFunc func, int barrier) {
return conn->type->set_write_handler(conn, func, barrier);
}
static inline void connClose(connection *conn) {
conn->type->close(conn);
}
/* Returns the last error encountered by the connection, as a string. If no error,
* a NULL is returned.
*/
static inline const char *connGetLastError(connection *conn) {
return conn->type->get_last_error(conn);
}
static inline ssize_t connSyncWrite(connection *conn, char *ptr, ssize_t size, long long timeout) {
return conn->type->sync_write(conn, ptr, size, timeout);
}
static inline ssize_t connSyncRead(connection *conn, char *ptr, ssize_t size, long long timeout) {
return conn->type->sync_read(conn, ptr, size, timeout);
}
static inline ssize_t connSyncReadLine(connection *conn, char *ptr, ssize_t size, long long timeout) {
return conn->type->sync_readline(conn, ptr, size, timeout);
}
connection *connCreateSocket();
connection *connCreateAcceptedSocket(int fd);
connection *connCreateTLS();
connection *connCreateAcceptedTLS(int fd, int require_auth);
void connSetPrivateData(connection *conn, void *data);
void *connGetPrivateData(connection *conn);
int connGetState(connection *conn);
int connHasWriteHandler(connection *conn);
int connHasReadHandler(connection *conn);
int connGetSocketError(connection *conn);
/* anet-style wrappers to conns */
int connBlock(connection *conn);
int connNonBlock(connection *conn);
int connEnableTcpNoDelay(connection *conn);
int connDisableTcpNoDelay(connection *conn);
int connKeepAlive(connection *conn, int interval);
int connSendTimeout(connection *conn, long long ms);
int connRecvTimeout(connection *conn, long long ms);
int connPeerToString(connection *conn, char *ip, size_t ip_len, int *port);
int connFormatPeer(connection *conn, char *buf, size_t buf_len);
int connSockName(connection *conn, char *ip, size_t ip_len, int *port);
const char *connGetInfo(connection *conn, char *buf, size_t buf_len);
/* Helpers for tls special considerations */
int tlsHasPendingData();
void tlsProcessPendingData();
#endif /* __REDIS_CONNECTION_H */

85
src/connhelpers.h Normal file
View File

@ -0,0 +1,85 @@
/*
* Copyright (c) 2019, Redis Labs
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __REDIS_CONNHELPERS_H
#define __REDIS_CONNHELPERS_H
#include "connection.h"
/* These are helper functions that are common to different connection
* implementations (currently sockets in connection.c and TLS in tls.c).
*
* Currently helpers implement the mechanisms for invoking connection
* handlers, tracking in-handler states and dealing with deferred
* destruction (if invoked by a handler).
*/
/* Called whenever a handler is invoked on a connection and sets the
* CONN_FLAG_IN_HANDLER flag to indicate we're in a handler context.
*
* An attempt to close a connection while CONN_FLAG_IN_HANDLER is
* set will result with deferred close, i.e. setting the CONN_FLAG_CLOSE_SCHEDULED
* instead of destructing it.
*/
static inline void enterHandler(connection *conn) {
conn->flags |= CONN_FLAG_IN_HANDLER;
}
/* Called whenever a handler returns. This unsets the CONN_FLAG_IN_HANDLER
* flag and performs actual close/destruction if a deferred close was
* scheduled by the handler.
*/
static inline int exitHandler(connection *conn) {
conn->flags &= ~CONN_FLAG_IN_HANDLER;
if (conn->flags & CONN_FLAG_CLOSE_SCHEDULED) {
connClose(conn);
return 0;
}
return 1;
}
/* Helper for connection implementations to call handlers:
* 1. Mark the handler in use.
* 2. Execute the handler (if set).
* 3. Mark the handler as NOT in use and perform deferred close if was
* requested by the handler at any time.
*/
static inline int callHandler(connection *conn, ConnectionCallbackFunc handler) {
conn->flags |= CONN_FLAG_IN_HANDLER;
if (handler) handler(conn);
conn->flags &= ~CONN_FLAG_IN_HANDLER;
if (conn->flags & CONN_FLAG_CLOSE_SCHEDULED) {
connClose(conn);
return 0;
}
return 1;
}
#endif /* __REDIS_CONNHELPERS_H */

89
src/db.c
View File

@ -60,10 +60,7 @@ robj *lookupKey(redisDb *db, robj *key, int flags) {
/* Update the access time for the ageing algorithm.
* Don't do it if we have a saving child, as this will trigger
* a copy on write madness. */
if (server.rdb_child_pid == -1 &&
server.aof_child_pid == -1 &&
!(flags & LOOKUP_NOTOUCH))
{
if (!hasActiveChildProcess() && !(flags & LOOKUP_NOTOUCH)){
if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
updateLFU(val);
} else {
@ -83,6 +80,7 @@ robj *lookupKey(redisDb *db, robj *key, int flags) {
* 1. A key gets expired if it reached it's TTL.
* 2. The key last access time is updated.
* 3. The global keys hits/misses stats are updated (reported in INFO).
* 4. If keyspace notifications are enabled, a "keymiss" notification is fired.
*
* This API should not be used when we write to the key after obtaining
* the object linked to the key, but only for read only operations.
@ -106,6 +104,7 @@ robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
* to return NULL ASAP. */
if (server.masterhost == NULL) {
server.stat_keyspace_misses++;
notifyKeyspaceEvent(NOTIFY_KEY_MISS, "keymiss", key, db->id);
return NULL;
}
@ -127,12 +126,15 @@ robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
server.current_client->cmd->flags & CMD_READONLY)
{
server.stat_keyspace_misses++;
notifyKeyspaceEvent(NOTIFY_KEY_MISS, "keymiss", key, db->id);
return NULL;
}
}
val = lookupKey(db,key,flags);
if (val == NULL)
if (val == NULL) {
server.stat_keyspace_misses++;
notifyKeyspaceEvent(NOTIFY_KEY_MISS, "keymiss", key, db->id);
}
else
server.stat_keyspace_hits++;
return val;
@ -339,7 +341,7 @@ robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o) {
* On success the fuction returns the number of keys removed from the
* database(s). Otherwise -1 is returned in the specific case the
* DB number is out of range, and errno is set to EINVAL. */
long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
long long emptyDbGeneric(redisDb *dbarray, int dbnum, int flags, void(callback)(void*)) {
int async = (flags & EMPTYDB_ASYNC);
long long removed = 0;
@ -348,6 +350,11 @@ long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
return -1;
}
/* Make sure the WATCHed keys are affected by the FLUSH* commands.
* Note that we need to call the function while the keys are still
* there. */
signalFlushedDb(dbnum);
int startdb, enddb;
if (dbnum == -1) {
startdb = 0;
@ -357,12 +364,12 @@ long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
}
for (int j = startdb; j <= enddb; j++) {
removed += dictSize(server.db[j].dict);
removed += dictSize(dbarray[j].dict);
if (async) {
emptyDbAsync(&server.db[j]);
emptyDbAsync(&dbarray[j]);
} else {
dictEmpty(server.db[j].dict,callback);
dictEmpty(server.db[j].expires,callback);
dictEmpty(dbarray[j].dict,callback);
dictEmpty(dbarray[j].expires,callback);
}
}
if (server.cluster_enabled) {
@ -376,6 +383,10 @@ long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
return removed;
}
long long emptyDb(int dbnum, int flags, void(callback)(void*)) {
return emptyDbGeneric(server.db, dbnum, flags, callback);
}
int selectDb(client *c, int id) {
if (id < 0 || id >= server.dbnum)
return C_ERR;
@ -383,6 +394,15 @@ int selectDb(client *c, int id) {
return C_OK;
}
long long dbTotalServerKeyCount() {
long long total = 0;
int j;
for (j = 0; j < server.dbnum; j++) {
total += dictSize(server.db[j].dict);
}
return total;
}
/*-----------------------------------------------------------------------------
* Hooks for key space changes.
*
@ -394,10 +414,12 @@ int selectDb(client *c, int id) {
void signalModifiedKey(redisDb *db, robj *key) {
touchWatchedKey(db,key);
trackingInvalidateKey(key);
}
void signalFlushedDb(int dbid) {
touchWatchedKeysOnFlush(dbid);
trackingInvalidateKeysOnFlush(dbid);
}
/*-----------------------------------------------------------------------------
@ -433,9 +455,15 @@ void flushdbCommand(client *c) {
int flags;
if (getFlushCommandFlags(c,&flags) == C_ERR) return;
signalFlushedDb(c->db->id);
server.dirty += emptyDb(c->db->id,flags,NULL);
addReply(c,shared.ok);
#if defined(USE_JEMALLOC)
/* jemalloc 5 doesn't release pages back to the OS when there's no traffic.
* for large databases, flushdb blocks for long anyway, so a bit more won't
* harm and this way the flush and purge will be synchroneus. */
if (!(flags & EMPTYDB_ASYNC))
jemalloc_purge();
#endif
}
/* FLUSHALL [ASYNC]
@ -445,7 +473,6 @@ void flushallCommand(client *c) {
int flags;
if (getFlushCommandFlags(c,&flags) == C_ERR) return;
signalFlushedDb(-1);
server.dirty += emptyDb(-1,flags,NULL);
addReply(c,shared.ok);
if (server.rdb_child_pid != -1) killRDBChild();
@ -459,6 +486,13 @@ void flushallCommand(client *c) {
server.dirty = saved_dirty;
}
server.dirty++;
#if defined(USE_JEMALLOC)
/* jemalloc 5 doesn't release pages back to the OS when there's no traffic.
* for large databases, flushdb blocks for long anyway, so a bit more won't
* harm and this way the flush and purge will be synchroneus. */
if (!(flags & EMPTYDB_ASYNC))
jemalloc_purge();
#endif
}
/* This command implements DEL and LAZYDEL. */
@ -608,7 +642,7 @@ int parseScanCursorOrReply(client *c, robj *o, unsigned long *cursor) {
}
/* This command implements SCAN, HSCAN and SSCAN commands.
* If object 'o' is passed, then it must be a Hash or Set object, otherwise
* If object 'o' is passed, then it must be a Hash, Set or Zset object, otherwise
* if 'o' is NULL the command will operate on the dictionary associated with
* the current database.
*
@ -624,6 +658,7 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) {
listNode *node, *nextnode;
long count = 10;
sds pat = NULL;
sds typename = NULL;
int patlen = 0, use_pattern = 0;
dict *ht;
@ -660,6 +695,10 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) {
use_pattern = !(pat[0] == '*' && patlen == 1);
i += 2;
} else if (!strcasecmp(c->argv[i]->ptr, "type") && o == NULL && j >= 2) {
/* SCAN for a particular type only applies to the db dict */
typename = c->argv[i+1]->ptr;
i+= 2;
} else {
addReply(c,shared.syntaxerr);
goto cleanup;
@ -754,6 +793,13 @@ void scanGenericCommand(client *c, robj *o, unsigned long cursor) {
}
}
/* Filter an element if it isn't the type we want. */
if (!filter && o == NULL && typename){
robj* typecheck = lookupKeyReadWithFlags(c->db, kobj, LOOKUP_NOTOUCH);
char* type = getObjectTypeName(typecheck);
if (strcasecmp((char*) typename, type)) filter = 1;
}
/* Filter element if it is an expired key. */
if (!filter && o == NULL && expireIfNeeded(c->db, kobj)) filter = 1;
@ -810,11 +856,8 @@ void lastsaveCommand(client *c) {
addReplyLongLong(c,server.lastsave);
}
void typeCommand(client *c) {
robj *o;
char *type;
o = lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH);
char* getObjectTypeName(robj *o) {
char* type;
if (o == NULL) {
type = "none";
} else {
@ -832,7 +875,13 @@ void typeCommand(client *c) {
default: type = "unknown"; break;
}
}
addReplyStatus(c,type);
return type;
}
void typeCommand(client *c) {
robj *o;
o = lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH);
addReplyStatus(c, getObjectTypeName(o));
}
void shutdownCommand(client *c) {
@ -994,7 +1043,7 @@ void scanDatabaseForReadyLists(redisDb *db) {
*
* Returns C_ERR if at least one of the DB ids are out of range, otherwise
* C_OK is returned. */
int dbSwapDatabases(int id1, int id2) {
int dbSwapDatabases(long id1, long id2) {
if (id1 < 0 || id1 >= server.dbnum ||
id2 < 0 || id2 >= server.dbnum) return C_ERR;
if (id1 == id2) return C_OK;

111
src/debug.c
View File

@ -297,6 +297,56 @@ void computeDatasetDigest(unsigned char *final) {
}
}
#ifdef USE_JEMALLOC
void mallctl_int(client *c, robj **argv, int argc) {
int ret;
/* start with the biggest size (int64), and if that fails, try smaller sizes (int32, bool) */
int64_t old = 0, val;
if (argc > 1) {
long long ll;
if (getLongLongFromObjectOrReply(c, argv[1], &ll, NULL) != C_OK)
return;
val = ll;
}
size_t sz = sizeof(old);
while (sz > 0) {
if ((ret=je_mallctl(argv[0]->ptr, &old, &sz, argc > 1? &val: NULL, argc > 1?sz: 0))) {
if (ret==EINVAL) {
/* size might be wrong, try a smaller one */
sz /= 2;
#if BYTE_ORDER == BIG_ENDIAN
val <<= 8*sz;
#endif
continue;
}
addReplyErrorFormat(c,"%s", strerror(ret));
return;
} else {
#if BYTE_ORDER == BIG_ENDIAN
old >>= 64 - 8*sz;
#endif
addReplyLongLong(c, old);
return;
}
}
addReplyErrorFormat(c,"%s", strerror(EINVAL));
}
void mallctl_string(client *c, robj **argv, int argc) {
int ret;
char *old;
size_t sz = sizeof(old);
/* for strings, it seems we need to first get the old value, before overriding it. */
if ((ret=je_mallctl(argv[0]->ptr, &old, &sz, NULL, 0))) {
addReplyErrorFormat(c,"%s", strerror(ret));
return;
}
addReplyBulkCString(c, old);
if(argc > 1)
je_mallctl(argv[0]->ptr, NULL, 0, &argv[1]->ptr, sizeof(char*));
}
#endif
void debugCommand(client *c) {
if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
const char *help[] = {
@ -319,10 +369,15 @@ void debugCommand(client *c) {
"SDSLEN <key> -- Show low level SDS string info representing key and value.",
"SEGFAULT -- Crash the server with sigsegv.",
"SET-ACTIVE-EXPIRE <0|1> -- Setting it to 0 disables expiring keys in background when they are not accessed (otherwise the Redis behavior). Setting it to 1 reenables back the default.",
"AOF-FLUSH-SLEEP <microsec> -- Server will sleep before flushing the AOF, this is used for testing",
"SLEEP <seconds> -- Stop the server for <seconds>. Decimals allowed.",
"STRUCTSIZE -- Return the size of different Redis core C structures.",
"ZIPLIST <key> -- Show low level info about the ziplist encoding.",
"STRINGMATCH-TEST -- Run a fuzz tester against the stringmatchlen() function.",
#ifdef USE_JEMALLOC
"MALLCTL <key> [<val>] -- Get or set a malloc tunning integer.",
"MALLCTL-STR <key> [<val>] -- Get or set a malloc tunning string.",
#endif
NULL
};
addReplyHelp(c, help);
@ -595,6 +650,11 @@ NULL
{
server.active_expire_enabled = atoi(c->argv[2]->ptr);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"aof-flush-sleep") &&
c->argc == 3)
{
server.aof_flush_sleep = atoi(c->argv[2]->ptr);
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"lua-always-replicate-commands") &&
c->argc == 3)
{
@ -638,7 +698,8 @@ NULL
dictGetStats(buf,sizeof(buf),server.db[dbid].expires);
stats = sdscat(stats,buf);
addReplyBulkSds(c,stats);
addReplyVerbatim(c,stats,sdslen(stats),"txt");
sdsfree(stats);
} else if (!strcasecmp(c->argv[1]->ptr,"htstats-key") && c->argc == 3) {
robj *o;
dict *ht = NULL;
@ -665,7 +726,7 @@ NULL
} else {
char buf[4096];
dictGetStats(buf,sizeof(buf),ht);
addReplyBulkCString(c,buf);
addReplyVerbatim(c,buf,strlen(buf),"txt");
}
} else if (!strcasecmp(c->argv[1]->ptr,"change-repl-id") && c->argc == 2) {
serverLog(LL_WARNING,"Changing replication IDs after receiving DEBUG change-repl-id");
@ -676,6 +737,14 @@ NULL
{
stringmatchlen_fuzz_test();
addReplyStatus(c,"Apparently Redis did not crash: test passed");
#ifdef USE_JEMALLOC
} else if(!strcasecmp(c->argv[1]->ptr,"mallctl") && c->argc >= 3) {
mallctl_int(c, c->argv+2, c->argc-2);
return;
} else if(!strcasecmp(c->argv[1]->ptr,"mallctl-str") && c->argc >= 3) {
mallctl_string(c, c->argv+2, c->argc-2);
return;
#endif
} else {
addReplySubcommandSyntaxError(c);
return;
@ -699,11 +768,12 @@ void _serverAssert(const char *estr, const char *file, int line) {
void _serverAssertPrintClientInfo(const client *c) {
int j;
char conninfo[CONN_INFO_LEN];
bugReportStart();
serverLog(LL_WARNING,"=== ASSERTION FAILED CLIENT CONTEXT ===");
serverLog(LL_WARNING,"client->flags = %d", c->flags);
serverLog(LL_WARNING,"client->fd = %d", c->fd);
serverLog(LL_WARNING,"client->flags = %llu", (unsigned long long) c->flags);
serverLog(LL_WARNING,"client->conn = %s", connGetInfo(c->conn, conninfo, sizeof(conninfo)));
serverLog(LL_WARNING,"client->argc = %d", c->argc);
for (j=0; j < c->argc; j++) {
char buf[128];
@ -1110,6 +1180,33 @@ void logRegisters(ucontext_t *uc) {
(unsigned long) uc->uc_mcontext.mc_cs
);
logStackContent((void**)uc->uc_mcontext.mc_rsp);
#elif defined(__aarch64__) /* Linux AArch64 */
serverLog(LL_WARNING,
"\n"
"X18:%016lx X19:%016lx\nX20:%016lx X21:%016lx\n"
"X22:%016lx X23:%016lx\nX24:%016lx X25:%016lx\n"
"X26:%016lx X27:%016lx\nX28:%016lx X29:%016lx\n"
"X30:%016lx\n"
"pc:%016lx sp:%016lx\npstate:%016lx fault_address:%016lx\n",
(unsigned long) uc->uc_mcontext.regs[18],
(unsigned long) uc->uc_mcontext.regs[19],
(unsigned long) uc->uc_mcontext.regs[20],
(unsigned long) uc->uc_mcontext.regs[21],
(unsigned long) uc->uc_mcontext.regs[22],
(unsigned long) uc->uc_mcontext.regs[23],
(unsigned long) uc->uc_mcontext.regs[24],
(unsigned long) uc->uc_mcontext.regs[25],
(unsigned long) uc->uc_mcontext.regs[26],
(unsigned long) uc->uc_mcontext.regs[27],
(unsigned long) uc->uc_mcontext.regs[28],
(unsigned long) uc->uc_mcontext.regs[29],
(unsigned long) uc->uc_mcontext.regs[30],
(unsigned long) uc->uc_mcontext.pc,
(unsigned long) uc->uc_mcontext.sp,
(unsigned long) uc->uc_mcontext.pstate,
(unsigned long) uc->uc_mcontext.fault_address
);
logStackContent((void**)uc->uc_mcontext.sp);
#else
serverLog(LL_WARNING,
" Dumping of registers not supported for this OS/arch");
@ -1337,6 +1434,12 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
/* Log dump of processor registers */
logRegisters(uc);
/* Log Modules INFO */
serverLogRaw(LL_WARNING|LL_RAW, "\n------ MODULES INFO OUTPUT ------\n");
infostring = modulesCollectInfo(sdsempty(), NULL, 1, 0);
serverLogRaw(LL_WARNING|LL_RAW, infostring);
sdsfree(infostring);
#if defined(HAVE_PROC_MAPS)
/* Test memory */
serverLogRaw(LL_WARNING|LL_RAW, "\n------ FAST MEMORY TEST ------\n");

10
src/defrag.c
View File

@ -47,7 +47,7 @@ int je_get_defrag_hint(void* ptr, int *bin_util, int *run_util);
/* forward declarations*/
void defragDictBucketCallback(void *privdata, dictEntry **bucketref);
dictEntry* replaceSateliteDictKeyPtrAndOrDefragDictEntry(dict *d, sds oldkey, sds newkey, unsigned int hash, long *defragged);
dictEntry* replaceSateliteDictKeyPtrAndOrDefragDictEntry(dict *d, sds oldkey, sds newkey, uint64_t hash, long *defragged);
/* Defrag helper for generic allocations.
*
@ -355,7 +355,7 @@ long activeDefragSdsListAndDict(list *l, dict *d, int dict_val_type) {
sdsele = ln->value;
if ((newsds = activeDefragSds(sdsele))) {
/* When defragging an sds value, we need to update the dict key */
unsigned int hash = dictGetHash(d, sdsele);
uint64_t hash = dictGetHash(d, sdsele);
replaceSateliteDictKeyPtrAndOrDefragDictEntry(d, sdsele, newsds, hash, &defragged);
ln->value = newsds;
defragged++;
@ -374,7 +374,7 @@ long activeDefragSdsListAndDict(list *l, dict *d, int dict_val_type) {
if ((newele = activeDefragStringOb(ele, &defragged)))
de->v.val = newele, defragged++;
} else if (dict_val_type == DEFRAG_SDS_DICT_VAL_VOID_PTR) {
void *newptr, *ptr = ln->value;
void *newptr, *ptr = dictGetVal(de);
if ((newptr = activeDefragAlloc(ptr)))
ln->value = newptr, defragged++;
}
@ -392,7 +392,7 @@ long activeDefragSdsListAndDict(list *l, dict *d, int dict_val_type) {
* moved. Return value is the the dictEntry if found, or NULL if not found.
* NOTE: this is very ugly code, but it let's us avoid the complication of
* doing a scan on another dict. */
dictEntry* replaceSateliteDictKeyPtrAndOrDefragDictEntry(dict *d, sds oldkey, sds newkey, unsigned int hash, long *defragged) {
dictEntry* replaceSateliteDictKeyPtrAndOrDefragDictEntry(dict *d, sds oldkey, sds newkey, uint64_t hash, long *defragged) {
dictEntry **deref = dictFindEntryRefByPtrAndHash(d, oldkey, hash);
if (deref) {
dictEntry *de = *deref;
@ -1039,7 +1039,7 @@ void activeDefragCycle(void) {
mstime_t latency;
int quit = 0;
if (server.aof_child_pid!=-1 || server.rdb_child_pid!=-1)
if (hasActiveChildProcess())
return; /* Defragging memory while there's a fork will just do damage. */
/* Once a second, check if we the fragmentation justfies starting a scan

9
src/evict.c
View File

@ -78,7 +78,7 @@ unsigned int getLRUClock(void) {
unsigned int LRU_CLOCK(void) {
unsigned int lruclock;
if (1000/server.hz <= LRU_CLOCK_RESOLUTION) {
atomicGet(server.lruclock,lruclock);
lruclock = server.lruclock;
} else {
lruclock = getLRUClock();
}
@ -444,6 +444,7 @@ int getMaxmemoryState(size_t *total, size_t *logical, size_t *tofree, float *lev
* Otehrwise if we are over the memory limit, but not enough memory
* was freed to return back under the limit, the function returns C_ERR. */
int freeMemoryIfNeeded(void) {
int keys_freed = 0;
/* By default replicas should ignore maxmemory
* and just be masters exact copies. */
if (server.masterhost && server.repl_slave_ignore_maxmemory) return C_OK;
@ -467,7 +468,7 @@ int freeMemoryIfNeeded(void) {
latencyStartMonitor(latency);
while (mem_freed < mem_tofree) {
int j, k, i, keys_freed = 0;
int j, k, i;
static unsigned int next_db = 0;
sds bestkey = NULL;
int bestdbid;
@ -598,9 +599,7 @@ int freeMemoryIfNeeded(void) {
mem_freed = mem_tofree;
}
}
}
if (!keys_freed) {
} else {
latencyEndMonitor(latency);
latencyAddSampleIfNeeded("eviction-cycle",latency);
goto cant_free; /* nothing to free... */

1
src/expire.c
View File

@ -64,6 +64,7 @@ int activeExpireCycleTryExpire(redisDb *db, dictEntry *de, long long now) {
dbSyncDelete(db,keyobj);
notifyKeyspaceEvent(NOTIFY_EXPIRED,
"expired",keyobj,db->id);
trackingInvalidateKey(keyobj);
decrRefCount(keyobj);
server.stat_expiredkeys++;
return 1;

12
src/geo.c
View File

@ -466,7 +466,7 @@ void georadiusGeneric(client *c, int flags) {
/* Look up the requested zset */
robj *zobj = NULL;
if ((zobj = lookupKeyReadOrReply(c, key, shared.null[c->resp])) == NULL ||
if ((zobj = lookupKeyReadOrReply(c, key, shared.emptyarray)) == NULL ||
checkType(c, zobj, OBJ_ZSET)) {
return;
}
@ -566,7 +566,7 @@ void georadiusGeneric(client *c, int flags) {
/* If no matching results, the user gets an empty reply. */
if (ga->used == 0 && storekey == NULL) {
addReplyNull(c);
addReply(c,shared.emptyarray);
geoArrayFree(ga);
return;
}
@ -734,14 +734,14 @@ void geohashCommand(client *c) {
r[1].max = 90;
geohashEncode(&r[0],&r[1],xy[0],xy[1],26,&hash);
char buf[12];
char buf[11];
int i;
for (i = 0; i < 11; i++) {
for (i = 0; i < 10; i++) {
int idx = (hash.bits >> (52-((i+1)*5))) & 0x1f;
buf[i] = geoalphabet[idx];
}
buf[11] = '\0';
addReplyBulkCBuffer(c,buf,11);
buf[10] = '\0';
addReplyBulkCBuffer(c,buf,10);
}
}
}

10
src/hyperloglog.c
View File

@ -700,7 +700,7 @@ int hllSparseSet(robj *o, long index, uint8_t count) {
p += oplen;
first += span;
}
if (span == 0) return -1; /* Invalid format. */
if (span == 0 || p >= end) return -1; /* Invalid format. */
next = HLL_SPARSE_IS_XZERO(p) ? p+2 : p+1;
if (next >= end) next = NULL;
@ -1014,8 +1014,8 @@ uint64_t hllCount(struct hllhdr *hdr, int *invalid) {
double m = HLL_REGISTERS;
double E;
int j;
/* Note that reghisto could be just HLL_Q+1, becuase this is the
* maximum frequency of the "000...1" sequence the hash function is
/* Note that reghisto size could be just HLL_Q+2, becuase HLL_Q+1 is
* the maximum frequency of the "000...1" sequence the hash function is
* able to return. However it is slow to check for sanity of the
* input: instead we history array at a safe size: overflows will
* just write data to wrong, but correctly allocated, places. */
@ -1242,7 +1242,7 @@ void pfcountCommand(client *c) {
if (o == NULL) continue; /* Assume empty HLL for non existing var.*/
if (isHLLObjectOrReply(c,o) != C_OK) return;
/* Merge with this HLL with our 'max' HHL by setting max[i]
/* Merge with this HLL with our 'max' HLL by setting max[i]
* to MAX(max[i],hll[i]). */
if (hllMerge(registers,o) == C_ERR) {
addReplySds(c,sdsnew(invalid_hll_err));
@ -1329,7 +1329,7 @@ void pfmergeCommand(client *c) {
hdr = o->ptr;
if (hdr->encoding == HLL_DENSE) use_dense = 1;
/* Merge with this HLL with our 'max' HHL by setting max[i]
/* Merge with this HLL with our 'max' HLL by setting max[i]
* to MAX(max[i],hll[i]). */
if (hllMerge(max,o) == C_ERR) {
addReplySds(c,sdsnew(invalid_hll_err));

4
src/latency.c
View File

@ -599,7 +599,7 @@ NULL
event = dictGetKey(de);
graph = latencyCommandGenSparkeline(event,ts);
addReplyBulkCString(c,graph);
addReplyVerbatim(c,graph,sdslen(graph),"txt");
sdsfree(graph);
} else if (!strcasecmp(c->argv[1]->ptr,"latest") && c->argc == 2) {
/* LATENCY LATEST */
@ -608,7 +608,7 @@ NULL
/* LATENCY DOCTOR */
sds report = createLatencyReport();
addReplyBulkCBuffer(c,report,sdslen(report));
addReplyVerbatim(c,report,sdslen(report),"txt");
sdsfree(report);
} else if (!strcasecmp(c->argv[1]->ptr,"reset") && c->argc >= 2) {
/* LATENCY RESET */

136
src/lolwut.c
View File

@ -34,8 +34,11 @@
*/
#include "server.h"
#include "lolwut.h"
#include <math.h>
void lolwut5Command(client *c);
void lolwut6Command(client *c);
/* The default target for LOLWUT if no matching version was found.
* This is what unstable versions of Redis will display. */
@ -43,14 +46,143 @@ void lolwutUnstableCommand(client *c) {
sds rendered = sdsnew("Redis ver. ");
rendered = sdscat(rendered,REDIS_VERSION);
rendered = sdscatlen(rendered,"\n",1);
addReplyBulkSds(c,rendered);
addReplyVerbatim(c,rendered,sdslen(rendered),"txt");
sdsfree(rendered);
}
/* LOLWUT [VERSION <version>] [... version specific arguments ...] */
void lolwutCommand(client *c) {
char *v = REDIS_VERSION;
if ((v[0] == '5' && v[1] == '.') ||
char verstr[64];
if (c->argc >= 3 && !strcasecmp(c->argv[1]->ptr,"version")) {
long ver;
if (getLongFromObjectOrReply(c,c->argv[2],&ver,NULL) != C_OK) return;
snprintf(verstr,sizeof(verstr),"%u.0.0",(unsigned int)ver);
v = verstr;
/* Adjust argv/argc to filter the "VERSION ..." option, since the
* specific LOLWUT version implementations don't know about it
* and expect their arguments. */
c->argv += 2;
c->argc -= 2;
}
if ((v[0] == '5' && v[1] == '.' && v[2] != '9') ||
(v[0] == '4' && v[1] == '.' && v[2] == '9'))
lolwut5Command(c);
else if ((v[0] == '6' && v[1] == '.' && v[2] != '9') ||
(v[0] == '5' && v[1] == '.' && v[2] == '9'))
lolwut6Command(c);
else
lolwutUnstableCommand(c);
/* Fix back argc/argv in case of VERSION argument. */
if (v == verstr) {
c->argv -= 2;
c->argc += 2;
}
}
/* ========================== LOLWUT Canvase ===============================
* Many LOWUT versions will likely print some computer art to the screen.
* This is the case with LOLWUT 5 and LOLWUT 6, so here there is a generic
* canvas implementation that can be reused. */
/* Allocate and return a new canvas of the specified size. */
lwCanvas *lwCreateCanvas(int width, int height, int bgcolor) {
lwCanvas *canvas = zmalloc(sizeof(*canvas));
canvas->width = width;
canvas->height = height;
canvas->pixels = zmalloc(width*height);
memset(canvas->pixels,bgcolor,width*height);
return canvas;
}
/* Free the canvas created by lwCreateCanvas(). */
void lwFreeCanvas(lwCanvas *canvas) {
zfree(canvas->pixels);
zfree(canvas);
}
/* Set a pixel to the specified color. Color is 0 or 1, where zero means no
* dot will be displyed, and 1 means dot will be displayed.
* Coordinates are arranged so that left-top corner is 0,0. You can write
* out of the size of the canvas without issues. */
void lwDrawPixel(lwCanvas *canvas, int x, int y, int color) {
if (x < 0 || x >= canvas->width ||
y < 0 || y >= canvas->height) return;
canvas->pixels[x+y*canvas->width] = color;
}
/* Return the value of the specified pixel on the canvas. */
int lwGetPixel(lwCanvas *canvas, int x, int y) {
if (x < 0 || x >= canvas->width ||
y < 0 || y >= canvas->height) return 0;
return canvas->pixels[x+y*canvas->width];
}
/* Draw a line from x1,y1 to x2,y2 using the Bresenham algorithm. */
void lwDrawLine(lwCanvas *canvas, int x1, int y1, int x2, int y2, int color) {
int dx = abs(x2-x1);
int dy = abs(y2-y1);
int sx = (x1 < x2) ? 1 : -1;
int sy = (y1 < y2) ? 1 : -1;
int err = dx-dy, e2;
while(1) {
lwDrawPixel(canvas,x1,y1,color);
if (x1 == x2 && y1 == y2) break;
e2 = err*2;
if (e2 > -dy) {
err -= dy;
x1 += sx;
}
if (e2 < dx) {
err += dx;
y1 += sy;
}
}
}
/* Draw a square centered at the specified x,y coordinates, with the specified
* rotation angle and size. In order to write a rotated square, we use the
* trivial fact that the parametric equation:
*
* x = sin(k)
* y = cos(k)
*
* Describes a circle for values going from 0 to 2*PI. So basically if we start
* at 45 degrees, that is k = PI/4, with the first point, and then we find
* the other three points incrementing K by PI/2 (90 degrees), we'll have the
* points of the square. In order to rotate the square, we just start with
* k = PI/4 + rotation_angle, and we are done.
*
* Of course the vanilla equations above will describe the square inside a
* circle of radius 1, so in order to draw larger squares we'll have to
* multiply the obtained coordinates, and then translate them. However this
* is much simpler than implementing the abstract concept of 2D shape and then
* performing the rotation/translation transformation, so for LOLWUT it's
* a good approach. */
void lwDrawSquare(lwCanvas *canvas, int x, int y, float size, float angle, int color) {
int px[4], py[4];
/* Adjust the desired size according to the fact that the square inscribed
* into a circle of radius 1 has the side of length SQRT(2). This way
* size becomes a simple multiplication factor we can use with our
* coordinates to magnify them. */
size /= 1.4142135623;
size = round(size);
/* Compute the four points. */
float k = M_PI/4 + angle;
for (int j = 0; j < 4; j++) {
px[j] = round(sin(k) * size + x);
py[j] = round(cos(k) * size + y);
k += M_PI/2;
}
/* Draw the square. */
for (int j = 0; j < 4; j++)
lwDrawLine(canvas,px[j],py[j],px[(j+1)%4],py[(j+1)%4],color);
}

49
src/lolwut.h Normal file
View File

@ -0,0 +1,49 @@
/*
* Copyright (c) 2018-2019, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
/* This structure represents our canvas. Drawing functions will take a pointer
* to a canvas to write to it. Later the canvas can be rendered to a string
* suitable to be printed on the screen, using unicode Braille characters. */
/* This represents a very simple generic canvas in order to draw stuff.
* It's up to each LOLWUT versions to translate what they draw to the
* screen, depending on the result to accomplish. */
typedef struct lwCanvas {
int width;
int height;
char *pixels;
} lwCanvas;
/* Drawing functions implemented inside lolwut.c. */
lwCanvas *lwCreateCanvas(int width, int height, int bgcolor);
void lwFreeCanvas(lwCanvas *canvas);
void lwDrawPixel(lwCanvas *canvas, int x, int y, int color);
int lwGetPixel(lwCanvas *canvas, int x, int y);
void lwDrawLine(lwCanvas *canvas, int x1, int y1, int x2, int y2, int color);
void lwDrawSquare(lwCanvas *canvas, int x, int y, float size, float angle, int color);

119
src/lolwut5.c
View File

@ -34,17 +34,9 @@
*/
#include "server.h"
#include "lolwut.h"
#include <math.h>
/* This structure represents our canvas. Drawing functions will take a pointer
* to a canvas to write to it. Later the canvas can be rendered to a string
* suitable to be printed on the screen, using unicode Braille characters. */
typedef struct lwCanvas {
int width;
int height;
char *pixels;
} lwCanvas;
/* Translate a group of 8 pixels (2x4 vertical rectangle) to the corresponding
* braille character. The byte should correspond to the pixels arranged as
* follows, where 0 is the least significant bit, and 7 the most significant
@ -69,104 +61,6 @@ void lwTranslatePixelsGroup(int byte, char *output) {
output[2] = 0x80 | (code & 0x3F); /* 10-xxxxxx */
}
/* Allocate and return a new canvas of the specified size. */
lwCanvas *lwCreateCanvas(int width, int height) {
lwCanvas *canvas = zmalloc(sizeof(*canvas));
canvas->width = width;
canvas->height = height;
canvas->pixels = zmalloc(width*height);
memset(canvas->pixels,0,width*height);
return canvas;
}
/* Free the canvas created by lwCreateCanvas(). */
void lwFreeCanvas(lwCanvas *canvas) {
zfree(canvas->pixels);
zfree(canvas);
}
/* Set a pixel to the specified color. Color is 0 or 1, where zero means no
* dot will be displyed, and 1 means dot will be displayed.
* Coordinates are arranged so that left-top corner is 0,0. You can write
* out of the size of the canvas without issues. */
void lwDrawPixel(lwCanvas *canvas, int x, int y, int color) {
if (x < 0 || x >= canvas->width ||
y < 0 || y >= canvas->height) return;
canvas->pixels[x+y*canvas->width] = color;
}
/* Return the value of the specified pixel on the canvas. */
int lwGetPixel(lwCanvas *canvas, int x, int y) {
if (x < 0 || x >= canvas->width ||
y < 0 || y >= canvas->height) return 0;
return canvas->pixels[x+y*canvas->width];
}
/* Draw a line from x1,y1 to x2,y2 using the Bresenham algorithm. */
void lwDrawLine(lwCanvas *canvas, int x1, int y1, int x2, int y2, int color) {
int dx = abs(x2-x1);
int dy = abs(y2-y1);
int sx = (x1 < x2) ? 1 : -1;
int sy = (y1 < y2) ? 1 : -1;
int err = dx-dy, e2;
while(1) {
lwDrawPixel(canvas,x1,y1,color);
if (x1 == x2 && y1 == y2) break;
e2 = err*2;
if (e2 > -dy) {
err -= dy;
x1 += sx;
}
if (e2 < dx) {
err += dx;
y1 += sy;
}
}
}
/* Draw a square centered at the specified x,y coordinates, with the specified
* rotation angle and size. In order to write a rotated square, we use the
* trivial fact that the parametric equation:
*
* x = sin(k)
* y = cos(k)
*
* Describes a circle for values going from 0 to 2*PI. So basically if we start
* at 45 degrees, that is k = PI/4, with the first point, and then we find
* the other three points incrementing K by PI/2 (90 degrees), we'll have the
* points of the square. In order to rotate the square, we just start with
* k = PI/4 + rotation_angle, and we are done.
*
* Of course the vanilla equations above will describe the square inside a
* circle of radius 1, so in order to draw larger squares we'll have to
* multiply the obtained coordinates, and then translate them. However this
* is much simpler than implementing the abstract concept of 2D shape and then
* performing the rotation/translation transformation, so for LOLWUT it's
* a good approach. */
void lwDrawSquare(lwCanvas *canvas, int x, int y, float size, float angle) {
int px[4], py[4];
/* Adjust the desired size according to the fact that the square inscribed
* into a circle of radius 1 has the side of length SQRT(2). This way
* size becomes a simple multiplication factor we can use with our
* coordinates to magnify them. */
size /= 1.4142135623;
size = round(size);
/* Compute the four points. */
float k = M_PI/4 + angle;
for (int j = 0; j < 4; j++) {
px[j] = round(sin(k) * size + x);
py[j] = round(cos(k) * size + y);
k += M_PI/2;
}
/* Draw the square. */
for (int j = 0; j < 4; j++)
lwDrawLine(canvas,px[j],py[j],px[(j+1)%4],py[(j+1)%4],1);
}
/* Schotter, the output of LOLWUT of Redis 5, is a computer graphic art piece
* generated by Georg Nees in the 60s. It explores the relationship between
* caos and order.
@ -180,7 +74,7 @@ lwCanvas *lwDrawSchotter(int console_cols, int squares_per_row, int squares_per_
int padding = canvas_width > 4 ? 2 : 0;
float square_side = (float)(canvas_width-padding*2) / squares_per_row;
int canvas_height = square_side * squares_per_col + padding*2;
lwCanvas *canvas = lwCreateCanvas(canvas_width, canvas_height);
lwCanvas *canvas = lwCreateCanvas(canvas_width, canvas_height, 0);
for (int y = 0; y < squares_per_col; y++) {
for (int x = 0; x < squares_per_row; x++) {
@ -200,7 +94,7 @@ lwCanvas *lwDrawSchotter(int console_cols, int squares_per_row, int squares_per_
sx += r2*square_side/3;
sy += r3*square_side/3;
}
lwDrawSquare(canvas,sx,sy,square_side,angle);
lwDrawSquare(canvas,sx,sy,square_side,angle,1);
}
}
@ -212,7 +106,7 @@ lwCanvas *lwDrawSchotter(int console_cols, int squares_per_row, int squares_per_
* logical canvas. The actual returned string will require a terminal that is
* width/2 large and height/4 tall in order to hold the whole image without
* overflowing or scrolling, since each Barille character is 2x4. */
sds lwRenderCanvas(lwCanvas *canvas) {
static sds renderCanvas(lwCanvas *canvas) {
sds text = sdsempty();
for (int y = 0; y < canvas->height; y += 4) {
for (int x = 0; x < canvas->width; x += 2) {
@ -272,11 +166,12 @@ void lolwut5Command(client *c) {
/* Generate some computer art and reply. */
lwCanvas *canvas = lwDrawSchotter(cols,squares_per_row,squares_per_col);
sds rendered = lwRenderCanvas(canvas);
sds rendered = renderCanvas(canvas);
rendered = sdscat(rendered,
"\nGeorg Nees - schotter, plotter on paper, 1968. Redis ver. ");
rendered = sdscat(rendered,REDIS_VERSION);
rendered = sdscatlen(rendered,"\n",1);
addReplyBulkSds(c,rendered);
addReplyVerbatim(c,rendered,sdslen(rendered),"txt");
sdsfree(rendered);
lwFreeCanvas(canvas);
}

200
src/lolwut6.c Normal file
View File

@ -0,0 +1,200 @@
/*
* Copyright (c) 2019, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* ----------------------------------------------------------------------------
*
* This file implements the LOLWUT command. The command should do something
* fun and interesting, and should be replaced by a new implementation at
* each new version of Redis.
*
* Thanks to Michele Hiki Falcone for the original image that ispired
* the image, part of his game, Plaguemon.
*
* Thanks to the Shhh computer art collective for the help in tuning the
* output to have a better artistic effect.
*/
#include "server.h"
#include "lolwut.h"
/* Render the canvas using the four gray levels of the standard color
* terminal: they match very well to the grayscale display of the gameboy. */
static sds renderCanvas(lwCanvas *canvas) {
sds text = sdsempty();
for (int y = 0; y < canvas->height; y++) {
for (int x = 0; x < canvas->width; x++) {
int color = lwGetPixel(canvas,x,y);
char *ce; /* Color escape sequence. */
/* Note that we set both the foreground and background color.
* This way we are able to get a more consistent result among
* different terminals implementations. */
switch(color) {
case 0: ce = "0;30;40m"; break; /* Black */
case 1: ce = "0;90;100m"; break; /* Gray 1 */
case 2: ce = "0;37;47m"; break; /* Gray 2 */
case 3: ce = "0;97;107m"; break; /* White */
}
text = sdscatprintf(text,"\033[%s \033[0m",ce);
}
if (y != canvas->height-1) text = sdscatlen(text,"\n",1);
}
return text;
}
/* Draw a skyscraper on the canvas, according to the parameters in the
* 'skyscraper' structure. Window colors are random and are always one
* of the two grays. */
struct skyscraper {
int xoff; /* X offset. */
int width; /* Pixels width. */
int height; /* Pixels height. */
int windows; /* Draw windows if true. */
int color; /* Color of the skyscraper. */
};
void generateSkyscraper(lwCanvas *canvas, struct skyscraper *si) {
int starty = canvas->height-1;
int endy = starty - si->height + 1;
for (int y = starty; y >= endy; y--) {
for (int x = si->xoff; x < si->xoff+si->width; x++) {
/* The roof is four pixels less wide. */
if (y == endy && (x <= si->xoff+1 || x >= si->xoff+si->width-2))
continue;
int color = si->color;
/* Alter the color if this is a place where we want to
* draw a window. We check that we are in the inner part of the
* skyscraper, so that windows are far from the borders. */
if (si->windows &&
x > si->xoff+1 &&
x < si->xoff+si->width-2 &&
y > endy+1 &&
y < starty-1)
{
/* Calculate the x,y position relative to the start of
* the window area. */
int relx = x - (si->xoff+1);
int rely = y - (endy+1);
/* Note that we want the windows to be two pixels wide
* but just one pixel tall, because terminal "pixels"
* (characters) are not square. */
if (relx/2 % 2 && rely % 2) {
do {
color = 1 + rand() % 2;
} while (color == si->color);
/* Except we want adjacent pixels creating the same
* window to be the same color. */
if (relx % 2) color = lwGetPixel(canvas,x-1,y);
}
}
lwDrawPixel(canvas,x,y,color);
}
}
}
/* Generate a skyline inspired by the parallax backgrounds of 8 bit games. */
void generateSkyline(lwCanvas *canvas) {
struct skyscraper si;
/* First draw the background skyscraper without windows, using the
* two different grays. We use two passes to make sure that the lighter
* ones are always in the background. */
for (int color = 2; color >= 1; color--) {
si.color = color;
for (int offset = -10; offset < canvas->width;) {
offset += rand() % 8;
si.xoff = offset;
si.width = 10 + rand()%9;
if (color == 2)
si.height = canvas->height/2 + rand()%canvas->height/2;
else
si.height = canvas->height/2 + rand()%canvas->height/3;
si.windows = 0;
generateSkyscraper(canvas, &si);
if (color == 2)
offset += si.width/2;
else
offset += si.width+1;
}
}
/* Now draw the foreground skyscraper with the windows. */
si.color = 0;
for (int offset = -10; offset < canvas->width;) {
offset += rand() % 8;
si.xoff = offset;
si.width = 5 + rand()%14;
if (si.width % 4) si.width += (si.width % 3);
si.height = canvas->height/3 + rand()%canvas->height/2;
si.windows = 1;
generateSkyscraper(canvas, &si);
offset += si.width+5;
}
}
/* The LOLWUT 6 command:
*
* LOLWUT [columns] [rows]
*
* By default the command uses 80 columns, 40 squares per row
* per column.
*/
void lolwut6Command(client *c) {
long cols = 80;
long rows = 20;
/* Parse the optional arguments if any. */
if (c->argc > 1 &&
getLongFromObjectOrReply(c,c->argv[1],&cols,NULL) != C_OK)
return;
if (c->argc > 2 &&
getLongFromObjectOrReply(c,c->argv[2],&rows,NULL) != C_OK)
return;
/* Limits. We want LOLWUT to be always reasonably fast and cheap to execute
* so we have maximum number of columns, rows, and output resulution. */
if (cols < 1) cols = 1;
if (cols > 1000) cols = 1000;
if (rows < 1) rows = 1;
if (rows > 1000) rows = 1000;
/* Generate the city skyline and reply. */
lwCanvas *canvas = lwCreateCanvas(cols,rows,3);
generateSkyline(canvas);
sds rendered = renderCanvas(canvas);
rendered = sdscat(rendered,
"\nDedicated to the 8 bit game developers of past and present.\n"
"Original 8 bit image from Plaguemon by hikikomori. Redis ver. ");
rendered = sdscat(rendered,REDIS_VERSION);
rendered = sdscatlen(rendered,"\n",1);
addReplyVerbatim(c,rendered,sdslen(rendered),"txt");
sdsfree(rendered);
lwFreeCanvas(canvas);
}

1020
src/module.c
View File

File diff suppressed because it is too large Load Diff

1
src/modules/Makefile
View File

@ -46,7 +46,6 @@ hellotimer.so: hellotimer.xo
hellodict.xo: ../redismodule.h
hellodict.so: hellodict.xo
$(LD) -o $@ $< $(SHOBJ_LDFLAGS) $(LIBS) -lc
testmodule.xo: ../redismodule.h

29
src/modules/testmodule.c
View File

@ -109,9 +109,9 @@ int TestStringPrintf(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (argc < 3) {
return RedisModule_WrongArity(ctx);
}
RedisModuleString *s = RedisModule_CreateStringPrintf(ctx,
"Got %d args. argv[1]: %s, argv[2]: %s",
argc,
RedisModuleString *s = RedisModule_CreateStringPrintf(ctx,
"Got %d args. argv[1]: %s, argv[2]: %s",
argc,
RedisModule_StringPtrLen(argv[1], NULL),
RedisModule_StringPtrLen(argv[2], NULL)
);
@ -133,7 +133,7 @@ int TestUnlink(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
RedisModuleKey *k = RedisModule_OpenKey(ctx, RedisModule_CreateStringPrintf(ctx, "unlinked"), REDISMODULE_WRITE | REDISMODULE_READ);
if (!k) return failTest(ctx, "Could not create key");
if (REDISMODULE_ERR == RedisModule_StringSet(k, RedisModule_CreateStringPrintf(ctx, "Foobar"))) {
return failTest(ctx, "Could not set string value");
}
@ -152,7 +152,7 @@ int TestUnlink(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
return failTest(ctx, "Could not verify key to be unlinked");
}
return RedisModule_ReplyWithSimpleString(ctx, "OK");
}
int NotifyCallback(RedisModuleCtx *ctx, int type, const char *event,
@ -188,6 +188,10 @@ int TestNotifications(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
RedisModule_Call(ctx, "LPUSH", "cc", "l", "y");
RedisModule_Call(ctx, "LPUSH", "cc", "l", "y");
/* Miss some keys intentionally so we will get a "keymiss" notification. */
RedisModule_Call(ctx, "GET", "c", "nosuchkey");
RedisModule_Call(ctx, "SMEMBERS", "c", "nosuchkey");
size_t sz;
const char *rep;
RedisModuleCallReply *r = RedisModule_Call(ctx, "HGET", "cc", "notifications", "foo");
@ -225,6 +229,16 @@ int TestNotifications(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
FAIL("Wrong reply for l");
}
r = RedisModule_Call(ctx, "HGET", "cc", "notifications", "nosuchkey");
if (r == NULL || RedisModule_CallReplyType(r) != REDISMODULE_REPLY_STRING) {
FAIL("Wrong or no reply for nosuchkey");
} else {
rep = RedisModule_CallReplyStringPtr(r, &sz);
if (sz != 1 || *rep != '2') {
FAIL("Got reply '%.*s'. expected '2'", sz, rep);
}
}
RedisModule_Call(ctx, "FLUSHDB", "");
return RedisModule_ReplyWithSimpleString(ctx, "OK");
@ -423,7 +437,7 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
if (RedisModule_CreateCommand(ctx,"test.ctxflags",
TestCtxFlags,"readonly",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
if (RedisModule_CreateCommand(ctx,"test.unlink",
TestUnlink,"write deny-oom",1,1,1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
@ -435,7 +449,8 @@ int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc)
RedisModule_SubscribeToKeyspaceEvents(ctx,
REDISMODULE_NOTIFY_HASH |
REDISMODULE_NOTIFY_SET |
REDISMODULE_NOTIFY_STRING,
REDISMODULE_NOTIFY_STRING |
REDISMODULE_NOTIFY_KEY_MISS,
NotifyCallback);
if (RedisModule_CreateCommand(ctx,"test.notify",
TestNotifications,"write deny-oom",1,1,1) == REDISMODULE_ERR)

14
src/multi.c
View File

@ -175,7 +175,19 @@ void execCommand(client *c) {
must_propagate = 1;
}
call(c,server.loading ? CMD_CALL_NONE : CMD_CALL_FULL);
int acl_retval = ACLCheckCommandPerm(c);
if (acl_retval != ACL_OK) {
addReplyErrorFormat(c,
"-NOPERM ACLs rules changed between the moment the "
"transaction was accumulated and the EXEC call. "
"This command is no longer allowed for the "
"following reason: %s",
(acl_retval == ACL_DENIED_CMD) ?
"no permission to execute the command or subcommand" :
"no permission to touch the specified keys");
} else {
call(c,server.loading ? CMD_CALL_NONE : CMD_CALL_FULL);
}
/* Commands may alter argc/argv, restore mstate. */
c->mstate.commands[j].argc = c->argc;

794
src/networking.c
View File

File diff suppressed because it is too large Load Diff

6
src/notify.c
View File

@ -55,6 +55,7 @@ int keyspaceEventsStringToFlags(char *classes) {
case 'K': flags |= NOTIFY_KEYSPACE; break;
case 'E': flags |= NOTIFY_KEYEVENT; break;
case 't': flags |= NOTIFY_STREAM; break;
case 'm': flags |= NOTIFY_KEY_MISS; break;
default: return -1;
}
}
@ -81,6 +82,7 @@ sds keyspaceEventsFlagsToString(int flags) {
if (flags & NOTIFY_EXPIRED) res = sdscatlen(res,"x",1);
if (flags & NOTIFY_EVICTED) res = sdscatlen(res,"e",1);
if (flags & NOTIFY_STREAM) res = sdscatlen(res,"t",1);
if (flags & NOTIFY_KEY_MISS) res = sdscatlen(res,"m",1);
}
if (flags & NOTIFY_KEYSPACE) res = sdscatlen(res,"K",1);
if (flags & NOTIFY_KEYEVENT) res = sdscatlen(res,"E",1);
@ -100,12 +102,12 @@ void notifyKeyspaceEvent(int type, char *event, robj *key, int dbid) {
int len = -1;
char buf[24];
/* If any modules are interested in events, notify the module system now.
/* If any modules are interested in events, notify the module system now.
* This bypasses the notifications configuration, but the module engine
* will only call event subscribers if the event type matches the types
* they are interested in. */
moduleNotifyKeyspaceEvent(type, event, key, dbid);
/* If notifications for this class of events are off, return ASAP. */
if (!(server.notify_keyspace_events & type)) return;

43
src/object.c
View File

@ -467,10 +467,15 @@ robj *tryObjectEncoding(robj *o) {
incrRefCount(shared.integers[value]);
return shared.integers[value];
} else {
if (o->encoding == OBJ_ENCODING_RAW) sdsfree(o->ptr);
o->encoding = OBJ_ENCODING_INT;
o->ptr = (void*) value;
return o;
if (o->encoding == OBJ_ENCODING_RAW) {
sdsfree(o->ptr);
o->encoding = OBJ_ENCODING_INT;
o->ptr = (void*) value;
return o;
} else if (o->encoding == OBJ_ENCODING_EMBSTR) {
decrRefCount(o);
return createStringObjectFromLongLongForValue(value);
}
}
}
@ -834,7 +839,9 @@ size_t objectComputeSize(robj *o, size_t sample_size) {
d = ((zset*)o->ptr)->dict;
zskiplist *zsl = ((zset*)o->ptr)->zsl;
zskiplistNode *znode = zsl->header->level[0].forward;
asize = sizeof(*o)+sizeof(zset)+(sizeof(struct dictEntry*)*dictSlots(d));
asize = sizeof(*o)+sizeof(zset)+sizeof(zskiplist)+sizeof(dict)+
(sizeof(struct dictEntry*)*dictSlots(d))+
zmalloc_size(zsl->header);
while(znode != NULL && samples < sample_size) {
elesize += sdsAllocSize(znode->ele);
elesize += sizeof(struct dictEntry) + zmalloc_size(znode);
@ -1433,30 +1440,20 @@ NULL
#if defined(USE_JEMALLOC)
sds info = sdsempty();
je_malloc_stats_print(inputCatSds, &info, NULL);
addReplyBulkSds(c, info);
addReplyVerbatim(c,info,sdslen(info),"txt");
sdsfree(info);
#else
addReplyBulkCString(c,"Stats not supported for the current allocator");
#endif
} else if (!strcasecmp(c->argv[1]->ptr,"doctor") && c->argc == 2) {
sds report = getMemoryDoctorReport();
addReplyBulkSds(c,report);
addReplyVerbatim(c,report,sdslen(report),"txt");
sdsfree(report);
} else if (!strcasecmp(c->argv[1]->ptr,"purge") && c->argc == 2) {
#if defined(USE_JEMALLOC)
char tmp[32];
unsigned narenas = 0;
size_t sz = sizeof(unsigned);
if (!je_mallctl("arenas.narenas", &narenas, &sz, NULL, 0)) {
sprintf(tmp, "arena.%d.purge", narenas);
if (!je_mallctl(tmp, NULL, 0, NULL, 0)) {
addReply(c, shared.ok);
return;
}
}
addReplyError(c, "Error purging dirty pages");
#else
addReply(c, shared.ok);
/* Nothing to do for other allocators. */
#endif
if (jemalloc_purge() == 0)
addReply(c, shared.ok);
else
addReplyError(c, "Error purging dirty pages");
} else {
addReplyErrorFormat(c, "Unknown subcommand or wrong number of arguments for '%s'. Try MEMORY HELP", (char*)c->argv[1]->ptr);
}

3
src/rax.c
View File

@ -1791,7 +1791,8 @@ int raxCompare(raxIterator *iter, const char *op, unsigned char *key, size_t key
if (eq && key_len == iter->key_len) return 1;
else if (lt) return iter->key_len < key_len;
else if (gt) return iter->key_len > key_len;
} if (cmp > 0) {
return 0;
} else if (cmp > 0) {
return gt ? 1 : 0;
} else /* (cmp < 0) */ {
return lt ? 1 : 0;

558
src/rdb.c
View File

@ -42,30 +42,41 @@
#include <sys/stat.h>
#include <sys/param.h>
#define rdbExitReportCorruptRDB(...) rdbCheckThenExit(__LINE__,__VA_ARGS__)
/* This macro is called when the internal RDB stracture is corrupt */
#define rdbExitReportCorruptRDB(...) rdbReportError(1, __LINE__,__VA_ARGS__)
/* This macro is called when RDB read failed (possibly a short read) */
#define rdbReportReadError(...) rdbReportError(0, __LINE__,__VA_ARGS__)
char* rdbFileBeingLoaded = NULL; /* used for rdb checking on read error */
extern int rdbCheckMode;
void rdbCheckError(const char *fmt, ...);
void rdbCheckSetError(const char *fmt, ...);
void rdbCheckThenExit(int linenum, char *reason, ...) {
void rdbReportError(int corruption_error, int linenum, char *reason, ...) {
va_list ap;
char msg[1024];
int len;
len = snprintf(msg,sizeof(msg),
"Internal error in RDB reading function at rdb.c:%d -> ", linenum);
"Internal error in RDB reading offset %llu, function at rdb.c:%d -> ",
(unsigned long long)server.loading_loaded_bytes, linenum);
va_start(ap,reason);
vsnprintf(msg+len,sizeof(msg)-len,reason,ap);
va_end(ap);
if (!rdbCheckMode) {
serverLog(LL_WARNING, "%s", msg);
char *argv[2] = {"",server.rdb_filename};
redis_check_rdb_main(2,argv,NULL);
if (rdbFileBeingLoaded || corruption_error) {
serverLog(LL_WARNING, "%s", msg);
char *argv[2] = {"",rdbFileBeingLoaded};
redis_check_rdb_main(2,argv,NULL);
} else {
serverLog(LL_WARNING, "%s. Failure loading rdb format from socket, assuming connection error, resuming operation.", msg);
return;
}
} else {
rdbCheckError("%s",msg);
}
serverLog(LL_WARNING, "Terminating server after rdb file reading failure.");
exit(1);
}
@ -75,18 +86,6 @@ static int rdbWriteRaw(rio *rdb, void *p, size_t len) {
return len;
}
/* This is just a wrapper for the low level function rioRead() that will
* automatically abort if it is not possible to read the specified amount
* of bytes. */
void rdbLoadRaw(rio *rdb, void *buf, uint64_t len) {
if (rioRead(rdb,buf,len) == 0) {
rdbExitReportCorruptRDB(
"Impossible to read %llu bytes in rdbLoadRaw()",
(unsigned long long) len);
return; /* Not reached. */
}
}
int rdbSaveType(rio *rdb, unsigned char type) {
return rdbWriteRaw(rdb,&type,1);
}
@ -102,10 +101,12 @@ int rdbLoadType(rio *rdb) {
/* This is only used to load old databases stored with the RDB_OPCODE_EXPIRETIME
* opcode. New versions of Redis store using the RDB_OPCODE_EXPIRETIME_MS
* opcode. */
* opcode. On error -1 is returned, however this could be a valid time, so
* to check for loading errors the caller should call rioGetReadError() after
* calling this function. */
time_t rdbLoadTime(rio *rdb) {
int32_t t32;
rdbLoadRaw(rdb,&t32,4);
if (rioRead(rdb,&t32,4) == 0) return -1;
return (time_t)t32;
}
@ -125,10 +126,14 @@ int rdbSaveMillisecondTime(rio *rdb, long long t) {
* after upgrading to Redis version 5 they will no longer be able to load their
* own old RDB files. Because of that, we instead fix the function only for new
* RDB versions, and load older RDB versions as we used to do in the past,
* allowing big endian systems to load their own old RDB files. */
* allowing big endian systems to load their own old RDB files.
*
* On I/O error the function returns LLONG_MAX, however if this is also a
* valid stored value, the caller should use rioGetReadError() to check for
* errors after calling this function. */
long long rdbLoadMillisecondTime(rio *rdb, int rdbver) {
int64_t t64;
rdbLoadRaw(rdb,&t64,8);
if (rioRead(rdb,&t64,8) == 0) return LLONG_MAX;
if (rdbver >= 9) /* Check the top comment of this function. */
memrev64ifbe(&t64); /* Convert in big endian if the system is BE. */
return (long long)t64;
@ -255,7 +260,7 @@ int rdbEncodeInteger(long long value, unsigned char *enc) {
/* Loads an integer-encoded object with the specified encoding type "enctype".
* The returned value changes according to the flags, see
* rdbGenerincLoadStringObject() for more info. */
* rdbGenericLoadStringObject() for more info. */
void *rdbLoadIntegerObject(rio *rdb, int enctype, int flags, size_t *lenptr) {
int plain = flags & RDB_LOAD_PLAIN;
int sds = flags & RDB_LOAD_SDS;
@ -277,8 +282,8 @@ void *rdbLoadIntegerObject(rio *rdb, int enctype, int flags, size_t *lenptr) {
v = enc[0]|(enc[1]<<8)|(enc[2]<<16)|(enc[3]<<24);
val = (int32_t)v;
} else {
val = 0; /* anti-warning */
rdbExitReportCorruptRDB("Unknown RDB integer encoding type %d",enctype);
return NULL; /* Never reached. */
}
if (plain || sds) {
char buf[LONG_STR_SIZE], *p;
@ -381,8 +386,7 @@ void *rdbLoadLzfStringObject(rio *rdb, int flags, size_t *lenptr) {
/* Load the compressed representation and uncompress it to target. */
if (rioRead(rdb,c,clen) == 0) goto err;
if (lzf_decompress(c,clen,val,len) == 0) {
if (rdbCheckMode) rdbCheckSetError("Invalid LZF compressed string");
goto err;
rdbExitReportCorruptRDB("Invalid LZF compressed string");
}
zfree(c);
@ -496,6 +500,7 @@ void *rdbGenericLoadStringObject(rio *rdb, int flags, size_t *lenptr) {
return rdbLoadLzfStringObject(rdb,flags,lenptr);
default:
rdbExitReportCorruptRDB("Unknown RDB string encoding type %d",len);
return NULL; /* Never reached. */
}
}
@ -751,7 +756,7 @@ size_t rdbSaveStreamConsumers(rio *rdb, streamCG *cg) {
/* Save a Redis object.
* Returns -1 on error, number of bytes written on success. */
ssize_t rdbSaveObject(rio *rdb, robj *o) {
ssize_t rdbSaveObject(rio *rdb, robj *o, robj *key) {
ssize_t n = 0, nwritten = 0;
if (o->type == OBJ_STRING) {
@ -966,7 +971,6 @@ ssize_t rdbSaveObject(rio *rdb, robj *o) {
RedisModuleIO io;
moduleValue *mv = o->ptr;
moduleType *mt = mv->type;
moduleInitIOContext(io,mt,rdb);
/* Write the "module" identifier as prefix, so that we'll be able
* to call the right module during loading. */
@ -975,10 +979,13 @@ ssize_t rdbSaveObject(rio *rdb, robj *o) {
io.bytes += retval;
/* Then write the module-specific representation + EOF marker. */
moduleInitIOContext(io,mt,rdb,key);
mt->rdb_save(&io,mv->value);
retval = rdbSaveLen(rdb,RDB_MODULE_OPCODE_EOF);
if (retval == -1) return -1;
io.bytes += retval;
if (retval == -1)
io.error = 1;
else
io.bytes += retval;
if (io.ctx) {
moduleFreeContext(io.ctx);
@ -996,7 +1003,7 @@ ssize_t rdbSaveObject(rio *rdb, robj *o) {
* this length with very little changes to the code. In the future
* we could switch to a faster solution. */
size_t rdbSavedObjectLen(robj *o) {
ssize_t len = rdbSaveObject(NULL,o);
ssize_t len = rdbSaveObject(NULL,o,NULL);
serverAssertWithInfo(NULL,o,len != -1);
return len;
}
@ -1038,7 +1045,12 @@ int rdbSaveKeyValuePair(rio *rdb, robj *key, robj *val, long long expiretime) {
/* Save type, key, value */
if (rdbSaveObjectType(rdb,val) == -1) return -1;
if (rdbSaveStringObject(rdb,key) == -1) return -1;
if (rdbSaveObject(rdb,val) == -1) return -1;
if (rdbSaveObject(rdb,val,key) == -1) return -1;
/* Delay return if required (for testing) */
if (server.rdb_key_save_delay)
usleep(server.rdb_key_save_delay);
return 1;
}
@ -1091,6 +1103,45 @@ int rdbSaveInfoAuxFields(rio *rdb, int flags, rdbSaveInfo *rsi) {
return 1;
}
ssize_t rdbSaveSingleModuleAux(rio *rdb, int when, moduleType *mt) {
/* Save a module-specific aux value. */
RedisModuleIO io;
int retval = rdbSaveType(rdb, RDB_OPCODE_MODULE_AUX);
/* Write the "module" identifier as prefix, so that we'll be able
* to call the right module during loading. */
retval = rdbSaveLen(rdb,mt->id);
if (retval == -1) return -1;
io.bytes += retval;
/* write the 'when' so that we can provide it on loading. add a UINT opcode
* for backwards compatibility, everything after the MT needs to be prefixed
* by an opcode. */
retval = rdbSaveLen(rdb,RDB_MODULE_OPCODE_UINT);
if (retval == -1) return -1;
io.bytes += retval;
retval = rdbSaveLen(rdb,when);
if (retval == -1) return -1;
io.bytes += retval;
/* Then write the module-specific representation + EOF marker. */
moduleInitIOContext(io,mt,rdb,NULL);
mt->aux_save(&io,when);
retval = rdbSaveLen(rdb,RDB_MODULE_OPCODE_EOF);
if (retval == -1)
io.error = 1;
else
io.bytes += retval;
if (io.ctx) {
moduleFreeContext(io.ctx);
zfree(io.ctx);
}
if (io.error)
return -1;
return io.bytes;
}
/* Produces a dump of the database in RDB format sending it to the specified
* Redis I/O channel. On success C_OK is returned, otherwise C_ERR
* is returned and part of the output, or all the output, can be
@ -1112,6 +1163,7 @@ int rdbSaveRio(rio *rdb, int *error, int flags, rdbSaveInfo *rsi) {
snprintf(magic,sizeof(magic),"REDIS%04d",RDB_VERSION);
if (rdbWriteRaw(rdb,magic,9) == -1) goto werr;
if (rdbSaveInfoAuxFields(rdb,flags,rsi) == -1) goto werr;
if (rdbSaveModulesAux(rdb, REDISMODULE_AUX_BEFORE_RDB) == -1) goto werr;
for (j = 0; j < server.dbnum; j++) {
redisDb *db = server.db+j;
@ -1173,6 +1225,8 @@ int rdbSaveRio(rio *rdb, int *error, int flags, rdbSaveInfo *rsi) {
di = NULL; /* So that we don't release it again on error. */
}
if (rdbSaveModulesAux(rdb, REDISMODULE_AUX_AFTER_RDB) == -1) goto werr;
/* EOF opcode */
if (rdbSaveType(rdb,RDB_OPCODE_EOF) == -1) goto werr;
@ -1281,40 +1335,25 @@ werr:
int rdbSaveBackground(char *filename, rdbSaveInfo *rsi) {
pid_t childpid;
long long start;
if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) return C_ERR;
if (hasActiveChildProcess()) return C_ERR;
server.dirty_before_bgsave = server.dirty;
server.lastbgsave_try = time(NULL);
openChildInfoPipe();
start = ustime();
if ((childpid = fork()) == 0) {
if ((childpid = redisFork()) == 0) {
int retval;
/* Child */
closeListeningSockets(0);
redisSetProcTitle("redis-rdb-bgsave");
retval = rdbSave(filename,rsi);
if (retval == C_OK) {
size_t private_dirty = zmalloc_get_private_dirty(-1);
if (private_dirty) {
serverLog(LL_NOTICE,
"RDB: %zu MB of memory used by copy-on-write",
private_dirty/(1024*1024));
}
server.child_info_data.cow_size = private_dirty;
sendChildInfo(CHILD_INFO_TYPE_RDB);
sendChildCOWInfo(CHILD_INFO_TYPE_RDB, "RDB");
}
exitFromChild((retval == C_OK) ? 0 : 1);
} else {
/* Parent */
server.stat_fork_time = ustime()-start;
server.stat_fork_rate = (double) zmalloc_used_memory() * 1000000 / server.stat_fork_time / (1024*1024*1024); /* GB per second. */
latencyAddSampleIfNeeded("fork",server.stat_fork_time/1000);
if (childpid == -1) {
closeChildInfoPipe();
server.lastbgsave_status = C_ERR;
@ -1326,7 +1365,6 @@ int rdbSaveBackground(char *filename, rdbSaveInfo *rsi) {
server.rdb_save_time_start = time(NULL);
server.rdb_child_pid = childpid;
server.rdb_child_type = RDB_CHILD_TYPE_DISK;
updateDictResizePolicy();
return C_OK;
}
return C_OK; /* unreached */
@ -1380,7 +1418,7 @@ robj *rdbLoadCheckModuleValue(rio *rdb, char *modulename) {
/* Load a Redis object of the specified type from the specified file.
* On success a newly allocated object is returned, otherwise NULL. */
robj *rdbLoadObject(int rdbtype, rio *rdb) {
robj *rdbLoadObject(int rdbtype, rio *rdb, robj *key) {
robj *o = NULL, *ele, *dec;
uint64_t len;
unsigned int i;
@ -1632,6 +1670,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
hashTypeConvert(o, OBJ_ENCODING_HT);
break;
default:
/* totally unreachable */
rdbExitReportCorruptRDB("Unknown RDB encoding type %d",rdbtype);
break;
}
@ -1639,6 +1678,11 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
o = createStreamObject();
stream *s = o->ptr;
uint64_t listpacks = rdbLoadLen(rdb,NULL);
if (listpacks == RDB_LENERR) {
rdbReportReadError("Stream listpacks len loading failed.");
decrRefCount(o);
return NULL;
}
while(listpacks--) {
/* Get the master ID, the one we'll use as key of the radix tree
@ -1646,7 +1690,9 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
* relatively to this ID. */
sds nodekey = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL);
if (nodekey == NULL) {
rdbExitReportCorruptRDB("Stream master ID loading failed: invalid encoding or I/O error.");
rdbReportReadError("Stream master ID loading failed: invalid encoding or I/O error.");
decrRefCount(o);
return NULL;
}
if (sdslen(nodekey) != sizeof(streamID)) {
rdbExitReportCorruptRDB("Stream node key entry is not the "
@ -1656,7 +1702,12 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
/* Load the listpack. */
unsigned char *lp =
rdbGenericLoadStringObject(rdb,RDB_LOAD_PLAIN,NULL);
if (lp == NULL) return NULL;
if (lp == NULL) {
rdbReportReadError("Stream listpacks loading failed.");
sdsfree(nodekey);
decrRefCount(o);
return NULL;
}
unsigned char *first = lpFirst(lp);
if (first == NULL) {
/* Serialized listpacks should never be empty, since on
@ -1674,12 +1725,24 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
}
/* Load total number of items inside the stream. */
s->length = rdbLoadLen(rdb,NULL);
/* Load the last entry ID. */
s->last_id.ms = rdbLoadLen(rdb,NULL);
s->last_id.seq = rdbLoadLen(rdb,NULL);
if (rioGetReadError(rdb)) {
rdbReportReadError("Stream object metadata loading failed.");
decrRefCount(o);
return NULL;
}
/* Consumer groups loading */
size_t cgroups_count = rdbLoadLen(rdb,NULL);
uint64_t cgroups_count = rdbLoadLen(rdb,NULL);
if (cgroups_count == RDB_LENERR) {
rdbReportReadError("Stream cgroup count loading failed.");
decrRefCount(o);
return NULL;
}
while(cgroups_count--) {
/* Get the consumer group name and ID. We can then create the
* consumer group ASAP and populate its structure as
@ -1687,11 +1750,21 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
streamID cg_id;
sds cgname = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL);
if (cgname == NULL) {
rdbExitReportCorruptRDB(
rdbReportReadError(
"Error reading the consumer group name from Stream");
decrRefCount(o);
return NULL;
}
cg_id.ms = rdbLoadLen(rdb,NULL);
cg_id.seq = rdbLoadLen(rdb,NULL);
if (rioGetReadError(rdb)) {
rdbReportReadError("Stream cgroup ID loading failed.");
sdsfree(cgname);
decrRefCount(o);
return NULL;
}
streamCG *cgroup = streamCreateCG(s,cgname,sdslen(cgname),&cg_id);
if (cgroup == NULL)
rdbExitReportCorruptRDB("Duplicated consumer group name %s",
@ -1703,13 +1776,28 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
* owner, since consumers for this group and their messages will
* be read as a next step. So for now leave them not resolved
* and later populate it. */
size_t pel_size = rdbLoadLen(rdb,NULL);
uint64_t pel_size = rdbLoadLen(rdb,NULL);
if (pel_size == RDB_LENERR) {
rdbReportReadError("Stream PEL size loading failed.");
decrRefCount(o);
return NULL;
}
while(pel_size--) {
unsigned char rawid[sizeof(streamID)];
rdbLoadRaw(rdb,rawid,sizeof(rawid));
if (rioRead(rdb,rawid,sizeof(rawid)) == 0) {
rdbReportReadError("Stream PEL ID loading failed.");
decrRefCount(o);
return NULL;
}
streamNACK *nack = streamCreateNACK(NULL);
nack->delivery_time = rdbLoadMillisecondTime(rdb,RDB_VERSION);
nack->delivery_count = rdbLoadLen(rdb,NULL);
if (rioGetReadError(rdb)) {
rdbReportReadError("Stream PEL NACK loading failed.");
decrRefCount(o);
streamFreeNACK(nack);
return NULL;
}
if (!raxInsert(cgroup->pel,rawid,sizeof(rawid),nack,NULL))
rdbExitReportCorruptRDB("Duplicated gobal PEL entry "
"loading stream consumer group");
@ -1717,24 +1805,47 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
/* Now that we loaded our global PEL, we need to load the
* consumers and their local PELs. */
size_t consumers_num = rdbLoadLen(rdb,NULL);
uint64_t consumers_num = rdbLoadLen(rdb,NULL);
if (consumers_num == RDB_LENERR) {
rdbReportReadError("Stream consumers num loading failed.");
decrRefCount(o);
return NULL;
}
while(consumers_num--) {
sds cname = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL);
if (cname == NULL) {
rdbExitReportCorruptRDB(
"Error reading the consumer name from Stream group");
rdbReportReadError(
"Error reading the consumer name from Stream group.");
decrRefCount(o);
return NULL;
}
streamConsumer *consumer = streamLookupConsumer(cgroup,cname,
1);
sdsfree(cname);
consumer->seen_time = rdbLoadMillisecondTime(rdb,RDB_VERSION);
if (rioGetReadError(rdb)) {
rdbReportReadError("Stream short read reading seen time.");
decrRefCount(o);
return NULL;
}
/* Load the PEL about entries owned by this specific
* consumer. */
pel_size = rdbLoadLen(rdb,NULL);
if (pel_size == RDB_LENERR) {
rdbReportReadError(
"Stream consumer PEL num loading failed.");
decrRefCount(o);
return NULL;
}
while(pel_size--) {
unsigned char rawid[sizeof(streamID)];
rdbLoadRaw(rdb,rawid,sizeof(rawid));
if (rioRead(rdb,rawid,sizeof(rawid)) == 0) {
rdbReportReadError(
"Stream short read reading PEL streamID.");
decrRefCount(o);
return NULL;
}
streamNACK *nack = raxFind(cgroup->pel,rawid,sizeof(rawid));
if (nack == raxNotFound)
rdbExitReportCorruptRDB("Consumer entry not found in "
@ -1753,6 +1864,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
}
} else if (rdbtype == RDB_TYPE_MODULE || rdbtype == RDB_TYPE_MODULE_2) {
uint64_t moduleid = rdbLoadLen(rdb,NULL);
if (rioGetReadError(rdb)) return NULL;
moduleType *mt = moduleTypeLookupModuleByID(moduleid);
char name[10];
@ -1767,7 +1879,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
exit(1);
}
RedisModuleIO io;
moduleInitIOContext(io,mt,rdb);
moduleInitIOContext(io,mt,rdb,key);
io.ver = (rdbtype == RDB_TYPE_MODULE) ? 1 : 2;
/* Call the rdb_load method of the module providing the 10 bit
* encoding version in the lower 10 bits of the module ID. */
@ -1780,6 +1892,11 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
/* Module v2 serialization has an EOF mark at the end. */
if (io.ver == 2) {
uint64_t eof = rdbLoadLen(rdb,NULL);
if (eof == RDB_LENERR) {
o = createModuleObject(mt,ptr); /* creating just in order to easily destroy */
decrRefCount(o);
return NULL;
}
if (eof != RDB_MODULE_OPCODE_EOF) {
serverLog(LL_WARNING,"The RDB file contains module data for the module '%s' that is not terminated by the proper module value EOF marker", name);
exit(1);
@ -1793,25 +1910,31 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
}
o = createModuleObject(mt,ptr);
} else {
rdbExitReportCorruptRDB("Unknown RDB encoding type %d",rdbtype);
rdbReportReadError("Unknown RDB encoding type %d",rdbtype);
return NULL;
}
return o;
}
/* Mark that we are loading in the global state and setup the fields
* needed to provide loading stats. */
void startLoading(FILE *fp) {
struct stat sb;
void startLoading(size_t size) {
/* Load the DB */
server.loading = 1;
server.loading_start_time = time(NULL);
server.loading_loaded_bytes = 0;
if (fstat(fileno(fp), &sb) == -1) {
server.loading_total_bytes = 0;
} else {
server.loading_total_bytes = sb.st_size;
}
server.loading_total_bytes = size;
}
/* Mark that we are loading in the global state and setup the fields
* needed to provide loading stats.
* 'filename' is optional and used for rdb-check on error */
void startLoadingFile(FILE *fp, char* filename) {
struct stat sb;
if (fstat(fileno(fp), &sb) == -1)
sb.st_size = 0;
rdbFileBeingLoaded = filename;
startLoading(sb.st_size);
}
/* Refresh the loading progress info */
@ -1824,6 +1947,7 @@ void loadingProgress(off_t pos) {
/* Loading finished */
void stopLoading(void) {
server.loading = 0;
rdbFileBeingLoaded = NULL;
}
/* Track loading progress in order to serve client's from time to time
@ -1886,11 +2010,13 @@ int rdbLoadRio(rio *rdb, rdbSaveInfo *rsi, int loading_aof) {
* load the actual type, and continue. */
expiretime = rdbLoadTime(rdb);
expiretime *= 1000;
if (rioGetReadError(rdb)) goto eoferr;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_EXPIRETIME_MS) {
/* EXPIRETIME_MS: milliseconds precision expire times introduced
* with RDB v3. Like EXPIRETIME but no with more precision. */
expiretime = rdbLoadMillisecondTime(rdb,rdbver);
if (rioGetReadError(rdb)) goto eoferr;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_FREQ) {
/* FREQ: LFU frequency. */
@ -1991,15 +2117,15 @@ int rdbLoadRio(rio *rdb, rdbSaveInfo *rsi, int loading_aof) {
decrRefCount(auxval);
continue; /* Read type again. */
} else if (type == RDB_OPCODE_MODULE_AUX) {
/* This is just for compatibility with the future: we have plans
* to add the ability for modules to store anything in the RDB
* file, like data that is not related to the Redis key space.
* Such data will potentially be stored both before and after the
* RDB keys-values section. For this reason since RDB version 9,
* we have the ability to read a MODULE_AUX opcode followed by an
* identifier of the module, and a serialized value in "MODULE V2"
* format. */
/* Load module data that is not related to the Redis key space.
* Such data can be potentially be stored both before and after the
* RDB keys-values section. */
uint64_t moduleid = rdbLoadLen(rdb,NULL);
int when_opcode = rdbLoadLen(rdb,NULL);
int when = rdbLoadLen(rdb,NULL);
if (rioGetReadError(rdb)) goto eoferr;
if (when_opcode != RDB_MODULE_OPCODE_UINT)
rdbReportReadError("bad when_opcode");
moduleType *mt = moduleTypeLookupModuleByID(moduleid);
char name[10];
moduleTypeNameByID(name,moduleid);
@ -2009,21 +2135,44 @@ int rdbLoadRio(rio *rdb, rdbSaveInfo *rsi, int loading_aof) {
serverLog(LL_WARNING,"The RDB file contains AUX module data I can't load: no matching module '%s'", name);
exit(1);
} else if (!rdbCheckMode && mt != NULL) {
/* This version of Redis actually does not know what to do
* with modules AUX data... */
serverLog(LL_WARNING,"The RDB file contains AUX module data I can't load for the module '%s'. Probably you want to use a newer version of Redis which implements aux data callbacks", name);
exit(1);
if (!mt->aux_load) {
/* Module doesn't support AUX. */
serverLog(LL_WARNING,"The RDB file contains module AUX data, but the module '%s' doesn't seem to support it.", name);
exit(1);
}
RedisModuleIO io;
moduleInitIOContext(io,mt,rdb,NULL);
io.ver = 2;
/* Call the rdb_load method of the module providing the 10 bit
* encoding version in the lower 10 bits of the module ID. */
if (mt->aux_load(&io,moduleid&1023, when) || io.error) {
moduleTypeNameByID(name,moduleid);
serverLog(LL_WARNING,"The RDB file contains module AUX data for the module type '%s', that the responsible module is not able to load. Check for modules log above for additional clues.", name);
exit(1);
}
if (io.ctx) {
moduleFreeContext(io.ctx);
zfree(io.ctx);
}
uint64_t eof = rdbLoadLen(rdb,NULL);
if (eof != RDB_MODULE_OPCODE_EOF) {
serverLog(LL_WARNING,"The RDB file contains module AUX data for the module '%s' that is not terminated by the proper module value EOF marker", name);
exit(1);
}
continue;
} else {
/* RDB check mode. */
robj *aux = rdbLoadCheckModuleValue(rdb,name);
decrRefCount(aux);
continue; /* Read next opcode. */
}
}
/* Read key */
if ((key = rdbLoadStringObject(rdb)) == NULL) goto eoferr;
/* Read value */
if ((val = rdbLoadObject(type,rdb)) == NULL) goto eoferr;
if ((val = rdbLoadObject(type,rdb,key)) == NULL) goto eoferr;
/* Check if the key already expired. This function is used when loading
* an RDB file from disk, either at startup, or when an RDB was
* received from the master. In the latter case, the master is
@ -2046,6 +2195,8 @@ int rdbLoadRio(rio *rdb, rdbSaveInfo *rsi, int loading_aof) {
* own reference. */
decrRefCount(key);
}
if (server.key_load_delay)
usleep(server.key_load_delay);
/* Reset the state that is key-specified and is populated by
* opcodes before the key, so that we start from scratch again. */
@ -2070,10 +2221,15 @@ int rdbLoadRio(rio *rdb, rdbSaveInfo *rsi, int loading_aof) {
}
return C_OK;
eoferr: /* unexpected end of file is handled here with a fatal exit */
serverLog(LL_WARNING,"Short read or OOM loading DB. Unrecoverable error, aborting now.");
rdbExitReportCorruptRDB("Unexpected EOF reading RDB file");
return C_ERR; /* Just to avoid warning */
/* Unexpected end of file is handled here calling rdbReportReadError():
* this will in turn either abort Redis in most cases, or if we are loading
* the RDB file from a socket during initial SYNC (diskless replica mode),
* we'll report the error to the caller, so that we can retry. */
eoferr:
serverLog(LL_WARNING,
"Short read or OOM loading DB. Unrecoverable error, aborting now.");
rdbReportReadError("Unexpected EOF reading RDB file");
return C_ERR;
}
/* Like rdbLoadRio() but takes a filename instead of a rio stream. The
@ -2089,7 +2245,7 @@ int rdbLoad(char *filename, rdbSaveInfo *rsi) {
int retval;
if ((fp = fopen(filename,"r")) == NULL) return C_ERR;
startLoading(fp);
startLoadingFile(fp, filename);
rioInitWithFile(&rdb,fp);
retval = rdbLoadRio(&rdb,rsi,0);
fclose(fp);
@ -2136,8 +2292,6 @@ void backgroundSaveDoneHandlerDisk(int exitcode, int bysignal) {
* This function covers the case of RDB -> Salves socket transfers for
* diskless replication. */
void backgroundSaveDoneHandlerSocket(int exitcode, int bysignal) {
uint64_t *ok_slaves;
if (!bysignal && exitcode == 0) {
serverLog(LL_NOTICE,
"Background RDB transfer terminated with success");
@ -2151,79 +2305,6 @@ void backgroundSaveDoneHandlerSocket(int exitcode, int bysignal) {
server.rdb_child_type = RDB_CHILD_TYPE_NONE;
server.rdb_save_time_start = -1;
/* If the child returns an OK exit code, read the set of slave client
* IDs and the associated status code. We'll terminate all the slaves
* in error state.
*
* If the process returned an error, consider the list of slaves that
* can continue to be empty, so that it's just a special case of the
* normal code path. */
ok_slaves = zmalloc(sizeof(uint64_t)); /* Make space for the count. */
ok_slaves[0] = 0;
if (!bysignal && exitcode == 0) {
int readlen = sizeof(uint64_t);
if (read(server.rdb_pipe_read_result_from_child, ok_slaves, readlen) ==
readlen)
{
readlen = ok_slaves[0]*sizeof(uint64_t)*2;
/* Make space for enough elements as specified by the first
* uint64_t element in the array. */
ok_slaves = zrealloc(ok_slaves,sizeof(uint64_t)+readlen);
if (readlen &&
read(server.rdb_pipe_read_result_from_child, ok_slaves+1,
readlen) != readlen)
{
ok_slaves[0] = 0;
}
}
}
close(server.rdb_pipe_read_result_from_child);
close(server.rdb_pipe_write_result_to_parent);
/* We can continue the replication process with all the slaves that
* correctly received the full payload. Others are terminated. */
listNode *ln;
listIter li;
listRewind(server.slaves,&li);
while((ln = listNext(&li))) {
client *slave = ln->value;
if (slave->replstate == SLAVE_STATE_WAIT_BGSAVE_END) {
uint64_t j;
int errorcode = 0;
/* Search for the slave ID in the reply. In order for a slave to
* continue the replication process, we need to find it in the list,
* and it must have an error code set to 0 (which means success). */
for (j = 0; j < ok_slaves[0]; j++) {
if (slave->id == ok_slaves[2*j+1]) {
errorcode = ok_slaves[2*j+2];
break; /* Found in slaves list. */
}
}
if (j == ok_slaves[0] || errorcode != 0) {
serverLog(LL_WARNING,
"Closing slave %s: child->slave RDB transfer failed: %s",
replicationGetSlaveName(slave),
(errorcode == 0) ? "RDB transfer child aborted"
: strerror(errorcode));
freeClient(slave);
} else {
serverLog(LL_WARNING,
"Slave %s correctly received the streamed RDB file.",
replicationGetSlaveName(slave));
/* Restore the socket as non-blocking. */
anetNonBlock(NULL,slave->fd);
anetSendTimeout(NULL,slave->fd,0);
}
}
}
zfree(ok_slaves);
updateSlavesWaitingBgsave((!bysignal && exitcode == 0) ? C_OK : C_ERR, RDB_CHILD_TYPE_SOCKET);
}
@ -2255,120 +2336,61 @@ void killRDBChild(void) {
/* Spawn an RDB child that writes the RDB to the sockets of the slaves
* that are currently in SLAVE_STATE_WAIT_BGSAVE_START state. */
int rdbSaveToSlavesSockets(rdbSaveInfo *rsi) {
int *fds;
uint64_t *clientids;
int numfds;
listNode *ln;
listIter li;
pid_t childpid;
long long start;
int pipefds[2];
if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) return C_ERR;
if (hasActiveChildProcess()) return C_ERR;
/* Before to fork, create a pipe that will be used in order to
* send back to the parent the IDs of the slaves that successfully
* received all the writes. */
/* Even if the previous fork child exited, don't start a new one until we
* drained the pipe. */
if (server.rdb_pipe_conns) return C_ERR;
/* Before to fork, create a pipe that is used to transfer the rdb bytes to
* the parent, we can't let it write directly to the sockets, since in case
* of TLS we must let the parent handle a continuous TLS state when the
* child terminates and parent takes over. */
if (pipe(pipefds) == -1) return C_ERR;
server.rdb_pipe_read_result_from_child = pipefds[0];
server.rdb_pipe_write_result_to_parent = pipefds[1];
server.rdb_pipe_read = pipefds[0];
server.rdb_pipe_write = pipefds[1];
anetNonBlock(NULL, server.rdb_pipe_read);
/* Collect the file descriptors of the slaves we want to transfer
/* Collect the connections of the replicas we want to transfer
* the RDB to, which are i WAIT_BGSAVE_START state. */
fds = zmalloc(sizeof(int)*listLength(server.slaves));
/* We also allocate an array of corresponding client IDs. This will
* be useful for the child process in order to build the report
* (sent via unix pipe) that will be sent to the parent. */
clientids = zmalloc(sizeof(uint64_t)*listLength(server.slaves));
numfds = 0;
server.rdb_pipe_conns = zmalloc(sizeof(connection *)*listLength(server.slaves));
server.rdb_pipe_numconns = 0;
server.rdb_pipe_numconns_writing = 0;
listRewind(server.slaves,&li);
while((ln = listNext(&li))) {
client *slave = ln->value;
if (slave->replstate == SLAVE_STATE_WAIT_BGSAVE_START) {
clientids[numfds] = slave->id;
fds[numfds++] = slave->fd;
server.rdb_pipe_conns[server.rdb_pipe_numconns++] = slave->conn;
replicationSetupSlaveForFullResync(slave,getPsyncInitialOffset());
/* Put the socket in blocking mode to simplify RDB transfer.
* We'll restore it when the children returns (since duped socket
* will share the O_NONBLOCK attribute with the parent). */
anetBlock(NULL,slave->fd);
anetSendTimeout(NULL,slave->fd,server.repl_timeout*1000);
}
}
/* Create the child process. */
openChildInfoPipe();
start = ustime();
if ((childpid = fork()) == 0) {
if ((childpid = redisFork()) == 0) {
/* Child */
int retval;
rio slave_sockets;
rio rdb;
rioInitWithFdset(&slave_sockets,fds,numfds);
zfree(fds);
rioInitWithFd(&rdb,server.rdb_pipe_write);
closeListeningSockets(0);
redisSetProcTitle("redis-rdb-to-slaves");
retval = rdbSaveRioWithEOFMark(&slave_sockets,NULL,rsi);
if (retval == C_OK && rioFlush(&slave_sockets) == 0)
retval = rdbSaveRioWithEOFMark(&rdb,NULL,rsi);
if (retval == C_OK && rioFlush(&rdb) == 0)
retval = C_ERR;
if (retval == C_OK) {
size_t private_dirty = zmalloc_get_private_dirty(-1);
if (private_dirty) {
serverLog(LL_NOTICE,
"RDB: %zu MB of memory used by copy-on-write",
private_dirty/(1024*1024));
}
server.child_info_data.cow_size = private_dirty;
sendChildInfo(CHILD_INFO_TYPE_RDB);
/* If we are returning OK, at least one slave was served
* with the RDB file as expected, so we need to send a report
* to the parent via the pipe. The format of the message is:
*
* <len> <slave[0].id> <slave[0].error> ...
*
* len, slave IDs, and slave errors, are all uint64_t integers,
* so basically the reply is composed of 64 bits for the len field
* plus 2 additional 64 bit integers for each entry, for a total
* of 'len' entries.
*
* The 'id' represents the slave's client ID, so that the master
* can match the report with a specific slave, and 'error' is
* set to 0 if the replication process terminated with a success
* or the error code if an error occurred. */
void *msg = zmalloc(sizeof(uint64_t)*(1+2*numfds));
uint64_t *len = msg;
uint64_t *ids = len+1;
int j, msglen;
*len = numfds;
for (j = 0; j < numfds; j++) {
*ids++ = clientids[j];
*ids++ = slave_sockets.io.fdset.state[j];
}
/* Write the message to the parent. If we have no good slaves or
* we are unable to transfer the message to the parent, we exit
* with an error so that the parent will abort the replication
* process with all the childre that were waiting. */
msglen = sizeof(uint64_t)*(1+2*numfds);
if (*len == 0 ||
write(server.rdb_pipe_write_result_to_parent,msg,msglen)
!= msglen)
{
retval = C_ERR;
}
zfree(msg);
sendChildCOWInfo(CHILD_INFO_TYPE_RDB, "RDB");
}
zfree(clientids);
rioFreeFdset(&slave_sockets);
rioFreeFd(&rdb);
close(server.rdb_pipe_write); /* wake up the reader, tell it we're done. */
exitFromChild((retval == C_OK) ? 0 : 1);
} else {
/* Parent */
@ -2382,32 +2404,28 @@ int rdbSaveToSlavesSockets(rdbSaveInfo *rsi) {
listRewind(server.slaves,&li);
while((ln = listNext(&li))) {
client *slave = ln->value;
int j;
for (j = 0; j < numfds; j++) {
if (slave->id == clientids[j]) {
slave->replstate = SLAVE_STATE_WAIT_BGSAVE_START;
break;
}
if (slave->replstate == SLAVE_STATE_WAIT_BGSAVE_END) {
slave->replstate = SLAVE_STATE_WAIT_BGSAVE_START;
}
}
close(pipefds[0]);
close(pipefds[1]);
close(server.rdb_pipe_write);
close(server.rdb_pipe_read);
zfree(server.rdb_pipe_conns);
server.rdb_pipe_conns = NULL;
server.rdb_pipe_numconns = 0;
server.rdb_pipe_numconns_writing = 0;
closeChildInfoPipe();
} else {
server.stat_fork_time = ustime()-start;
server.stat_fork_rate = (double) zmalloc_used_memory() * 1000000 / server.stat_fork_time / (1024*1024*1024); /* GB per second. */
latencyAddSampleIfNeeded("fork",server.stat_fork_time/1000);
serverLog(LL_NOTICE,"Background RDB transfer started by pid %d",
childpid);
server.rdb_save_time_start = time(NULL);
server.rdb_child_pid = childpid;
server.rdb_child_type = RDB_CHILD_TYPE_SOCKET;
updateDictResizePolicy();
close(server.rdb_pipe_write); /* close write in parent so that it can detect the close on the child. */
if (aeCreateFileEvent(server.el, server.rdb_pipe_read, AE_READABLE, rdbPipeReadHandler,NULL) == AE_ERR) {
serverPanic("Unrecoverable error creating server.rdb_pipe_read file event.");
}
}
zfree(clientids);
zfree(fds);
return (childpid == -1) ? C_ERR : C_OK;
}
return C_OK; /* Unreached. */
@ -2447,15 +2465,15 @@ void bgsaveCommand(client *c) {
if (server.rdb_child_pid != -1) {
addReplyError(c,"Background save already in progress");
} else if (server.aof_child_pid != -1) {
} else if (hasActiveChildProcess()) {
if (schedule) {
server.rdb_bgsave_scheduled = 1;
addReplyStatus(c,"Background saving scheduled");
} else {
addReplyError(c,
"An AOF log rewriting in progress: can't BGSAVE right now. "
"Use BGSAVE SCHEDULE in order to schedule a BGSAVE whenever "
"possible.");
"Another child process is active (AOF?): can't BGSAVE right now. "
"Use BGSAVE SCHEDULE in order to schedule a BGSAVE whenever "
"possible.");
}
} else if (rdbSaveBackground(server.rdb_filename,rsiptr) == C_OK) {
addReplyStatus(c,"Background saving started");

5
src/rdb.h
View File

@ -140,11 +140,12 @@ int rdbSaveBackground(char *filename, rdbSaveInfo *rsi);
int rdbSaveToSlavesSockets(rdbSaveInfo *rsi);
void rdbRemoveTempFile(pid_t childpid);
int rdbSave(char *filename, rdbSaveInfo *rsi);
ssize_t rdbSaveObject(rio *rdb, robj *o);
ssize_t rdbSaveObject(rio *rdb, robj *o, robj *key);
size_t rdbSavedObjectLen(robj *o);
robj *rdbLoadObject(int type, rio *rdb);
robj *rdbLoadObject(int type, rio *rdb, robj *key);
void backgroundSaveDoneHandler(int exitcode, int bysignal);
int rdbSaveKeyValuePair(rio *rdb, robj *key, robj *val, long long expiretime);
ssize_t rdbSaveSingleModuleAux(rio *rdb, int when, moduleType *mt);
robj *rdbLoadStringObject(rio *rdb);
ssize_t rdbSaveStringObject(rio *rdb, robj *obj);
ssize_t rdbSaveRawString(rio *rdb, unsigned char *s, size_t len);

38
src/redis-benchmark.c
View File

@ -104,6 +104,7 @@ static struct config {
int is_fetching_slots;
int is_updating_slots;
int slots_last_update;
int enable_tracking;
/* Thread mutexes to be used as fallbacks by atomicvar.h */
pthread_mutex_t requests_issued_mutex;
pthread_mutex_t requests_finished_mutex;
@ -254,6 +255,19 @@ static redisConfig *getRedisConfig(const char *ip, int port,
else fprintf(stderr,"%s: %s\n",hostsocket,err);
goto fail;
}
if(config.auth) {
void *authReply = NULL;
redisAppendCommand(c, "AUTH %s", config.auth);
if (REDIS_OK != redisGetReply(c, &authReply)) goto fail;
if (reply) freeReplyObject(reply);
reply = ((redisReply *) authReply);
if (reply->type == REDIS_REPLY_ERROR) {
fprintf(stderr, "ERROR: %s\n", reply->str);
goto fail;
}
}
redisAppendCommand(c, "CONFIG GET %s", "save");
redisAppendCommand(c, "CONFIG GET %s", "appendonly");
int i = 0;
@ -620,6 +634,14 @@ static client createClient(char *cmd, size_t len, client from, int thread_id) {
c->prefix_pending++;
}
if (config.enable_tracking) {
char *buf = NULL;
int len = redisFormatCommand(&buf, "CLIENT TRACKING on");
c->obuf = sdscatlen(c->obuf, buf, len);
free(buf);
c->prefix_pending++;
}
/* If a DB number different than zero is selected, prefix our request
* buffer with the SELECT command, that will be discarded the first
* time the replies are received, so if the client is reused the
@ -1192,7 +1214,7 @@ static int fetchClusterSlotsConfiguration(client c) {
assert(reply->type == REDIS_REPLY_ARRAY);
for (i = 0; i < reply->elements; i++) {
redisReply *r = reply->element[i];
assert(r->type = REDIS_REPLY_ARRAY);
assert(r->type == REDIS_REPLY_ARRAY);
assert(r->elements >= 3);
int from, to, slot;
from = r->element[0]->integer;
@ -1294,7 +1316,7 @@ int parseOptions(int argc, const char **argv) {
if (*p < '0' || *p > '9') goto invalid;
}
config.randomkeys = 1;
config.randomkeys_keyspacelen = atoi(argv[++i]);
config.randomkeys_keyspacelen = atoi(next);
if (config.randomkeys_keyspacelen < 0)
config.randomkeys_keyspacelen = 0;
} else if (!strcmp(argv[i],"-q")) {
@ -1337,6 +1359,8 @@ int parseOptions(int argc, const char **argv) {
} else if (config.num_threads < 0) config.num_threads = 0;
} else if (!strcmp(argv[i],"--cluster")) {
config.cluster_mode = 1;
} else if (!strcmp(argv[i],"--enable-tracking")) {
config.enable_tracking = 1;
} else if (!strcmp(argv[i],"--help")) {
exit_status = 0;
goto usage;
@ -1367,6 +1391,7 @@ usage:
" --dbnum <db> SELECT the specified db number (default 0)\n"
" --threads <num> Enable multi-thread mode.\n"
" --cluster Enable cluster mode.\n"
" --enable-tracking Send CLIENT TRACKING on before starting benchmark.\n"
" -k <boolean> 1=keep alive 0=reconnect (default 1)\n"
" -r <keyspacelen> Use random keys for SET/GET/INCR, random values for SADD\n"
" Using this option the benchmark will expand the string __rand_int__\n"
@ -1491,6 +1516,7 @@ int main(int argc, const char **argv) {
config.is_fetching_slots = 0;
config.is_updating_slots = 0;
config.slots_last_update = 0;
config.enable_tracking = 0;
i = parseOptions(argc,argv);
argc -= i;
@ -1527,7 +1553,10 @@ int main(int argc, const char **argv) {
if (node->name) printf("%s ", node->name);
printf("%s:%d\n", node->ip, node->port);
node->redis_config = getRedisConfig(node->ip, node->port, NULL);
if (node->redis_config == NULL) exit(1);
if (node->redis_config == NULL) {
fprintf(stderr, "WARN: could not fetch node CONFIG %s:%d\n",
node->ip, node->port);
}
}
printf("\n");
/* Automatically set thread number to node count if not specified
@ -1537,7 +1566,8 @@ int main(int argc, const char **argv) {
} else {
config.redis_config =
getRedisConfig(config.hostip, config.hostport, config.hostsocket);
if (config.redis_config == NULL) exit(1);
if (config.redis_config == NULL)
fprintf(stderr, "WARN: could not fetch server CONFIG\n");
}
if (config.num_threads > 0) {

6
src/redis-check-aof.c
View File

@ -33,11 +33,11 @@
#define ERROR(...) { \
char __buf[1024]; \
sprintf(__buf, __VA_ARGS__); \
sprintf(error, "0x%16llx: %s", (long long)epos, __buf); \
snprintf(__buf, sizeof(__buf), __VA_ARGS__); \
snprintf(error, sizeof(error), "0x%16llx: %s", (long long)epos, __buf); \
}
static char error[1024];
static char error[1044];
static off_t epos;
int consumeNewline(char *buf) {

12
src/redis-check-rdb.c
View File

@ -202,7 +202,7 @@ int redis_check_rdb(char *rdbfilename, FILE *fp) {
}
expiretime = -1;
startLoading(fp);
startLoadingFile(fp, rdbfilename);
while(1) {
robj *key, *val;
@ -216,14 +216,16 @@ int redis_check_rdb(char *rdbfilename, FILE *fp) {
/* EXPIRETIME: load an expire associated with the next key
* to load. Note that after loading an expire we need to
* load the actual type, and continue. */
if ((expiretime = rdbLoadTime(&rdb)) == -1) goto eoferr;
expiretime = rdbLoadTime(&rdb);
expiretime *= 1000;
if (rioGetReadError(&rdb)) goto eoferr;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_EXPIRETIME_MS) {
/* EXPIRETIME_MS: milliseconds precision expire times introduced
* with RDB v3. Like EXPIRETIME but no with more precision. */
rdbstate.doing = RDB_CHECK_DOING_READ_EXPIRE;
if ((expiretime = rdbLoadMillisecondTime(&rdb, rdbver)) == -1) goto eoferr;
expiretime = rdbLoadMillisecondTime(&rdb, rdbver);
if (rioGetReadError(&rdb)) goto eoferr;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_FREQ) {
/* FREQ: LFU frequency. */
@ -285,7 +287,7 @@ int redis_check_rdb(char *rdbfilename, FILE *fp) {
rdbstate.keys++;
/* Read value */
rdbstate.doing = RDB_CHECK_DOING_READ_OBJECT_VALUE;
if ((val = rdbLoadObject(type,&rdb)) == NULL) goto eoferr;
if ((val = rdbLoadObject(type,&rdb,key)) == NULL) goto eoferr;
/* Check if the key already expired. */
if (expiretime != -1 && expiretime < now)
rdbstate.already_expired++;
@ -314,6 +316,7 @@ int redis_check_rdb(char *rdbfilename, FILE *fp) {
}
if (closefile) fclose(fp);
stopLoading();
return 0;
eoferr: /* unexpected end of file is handled here with a fatal exit */
@ -324,6 +327,7 @@ eoferr: /* unexpected end of file is handled here with a fatal exit */
}
err:
if (closefile) fclose(fp);
stopLoading();
return 1;
}

240
src/redis-cli.c
View File

@ -47,6 +47,10 @@
#include <math.h>
#include <hiredis.h>
#ifdef USE_OPENSSL
#include <openssl/ssl.h>
#include <hiredis_ssl.h>
#endif
#include <sds.h> /* use sds.h from hiredis, so that only one set of sds functions will be present in the binary */
#include "dict.h"
#include "adlist.h"
@ -188,6 +192,12 @@ static struct config {
char *hostip;
int hostport;
char *hostsocket;
int tls;
char *sni;
char *cacert;
char *cacertdir;
char *cert;
char *key;
long repeat;
long interval;
int dbnum;
@ -218,6 +228,7 @@ static struct config {
int hotkeys;
int stdinarg; /* get last arg from stdin. (-x option) */
char *auth;
char *user;
int output; /* output mode, see OUTPUT_* defines */
sds mb_delim;
char prompt[128];
@ -230,6 +241,7 @@ static struct config {
int verbose;
clusterManagerCommand cluster_manager_command;
int no_auth_warning;
int resp3;
} config;
/* User preferences. */
@ -728,8 +740,13 @@ static int cliAuth(void) {
redisReply *reply;
if (config.auth == NULL) return REDIS_OK;
reply = redisCommand(context,"AUTH %s",config.auth);
if (config.user == NULL)
reply = redisCommand(context,"AUTH %s",config.auth);
else
reply = redisCommand(context,"AUTH %s %s",config.user,config.auth);
if (reply != NULL) {
if (reply->type == REDIS_REPLY_ERROR)
fprintf(stderr,"Warning: AUTH failed\n");
freeReplyObject(reply);
return REDIS_OK;
}
@ -751,6 +768,86 @@ static int cliSelect(void) {
return REDIS_ERR;
}
/* Wrapper around redisSecureConnection to avoid hiredis_ssl dependencies if
* not building with TLS support.
*/
static int cliSecureConnection(redisContext *c, const char **err) {
#ifdef USE_OPENSSL
static SSL_CTX *ssl_ctx = NULL;
if (!ssl_ctx) {
ssl_ctx = SSL_CTX_new(SSLv23_client_method());
if (!ssl_ctx) {
*err = "Failed to create SSL_CTX";
goto error;
}
SSL_CTX_set_options(ssl_ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3);
SSL_CTX_set_verify(ssl_ctx, SSL_VERIFY_PEER, NULL);
if (config.cacert || config.cacertdir) {
if (!SSL_CTX_load_verify_locations(ssl_ctx, config.cacert, config.cacertdir)) {
*err = "Invalid CA Certificate File/Directory";
goto error;
}
} else {
if (!SSL_CTX_set_default_verify_paths(ssl_ctx)) {
*err = "Failed to use default CA paths";
goto error;
}
}
if (config.cert && !SSL_CTX_use_certificate_chain_file(ssl_ctx, config.cert)) {
*err = "Invalid client certificate";
goto error;
}
if (config.key && !SSL_CTX_use_PrivateKey_file(ssl_ctx, config.key, SSL_FILETYPE_PEM)) {
*err = "Invalid private key";
goto error;
}
}
SSL *ssl = SSL_new(ssl_ctx);
if (!ssl) {
*err = "Failed to create SSL object";
return REDIS_ERR;
}
if (config.sni && !SSL_set_tlsext_host_name(ssl, config.sni)) {
*err = "Failed to configure SNI";
SSL_free(ssl);
return REDIS_ERR;
}
return redisInitiateSSL(c, ssl);
error:
SSL_CTX_free(ssl_ctx);
ssl_ctx = NULL;
return REDIS_ERR;
#else
(void) c;
(void) err;
return REDIS_OK;
#endif
}
/* Select RESP3 mode if redis-cli was started with the -3 option. */
static int cliSwitchProto(void) {
redisReply *reply;
if (config.resp3 == 0) return REDIS_OK;
reply = redisCommand(context,"HELLO 3");
if (reply != NULL) {
int result = REDIS_OK;
if (reply->type == REDIS_REPLY_ERROR) result = REDIS_ERR;
freeReplyObject(reply);
return result;
}
return REDIS_ERR;
}
/* Connect to the server. It is possible to pass certain flags to the function:
* CC_FORCE: The connection is performed even if there is already
* a connected socket.
@ -767,6 +864,16 @@ static int cliConnect(int flags) {
context = redisConnectUnix(config.hostsocket);
}
if (!context->err && config.tls) {
const char *err = NULL;
if (cliSecureConnection(context, &err) == REDIS_ERR && err) {
fprintf(stderr, "Could not negotiate a TLS connection: %s\n", err);
context = NULL;
redisFree(context);
return REDIS_ERR;
}
}
if (context->err) {
if (!(flags & CC_QUIET)) {
fprintf(stderr,"Could not connect to Redis at ");
@ -782,17 +889,20 @@ static int cliConnect(int flags) {
return REDIS_ERR;
}
/* Set aggressive KEEP_ALIVE socket option in the Redis context socket
* in order to prevent timeouts caused by the execution of long
* commands. At the same time this improves the detection of real
* errors. */
anetKeepAlive(NULL, context->fd, REDIS_CLI_KEEPALIVE_INTERVAL);
/* Do AUTH and select the right DB. */
/* Do AUTH, select the right DB, switch to RESP3 if needed. */
if (cliAuth() != REDIS_OK)
return REDIS_ERR;
if (cliSelect() != REDIS_OK)
return REDIS_ERR;
if (cliSwitchProto() != REDIS_OK)
return REDIS_ERR;
}
return REDIS_OK;
}
@ -819,10 +929,17 @@ static sds cliFormatReplyTTY(redisReply *r, char *prefix) {
out = sdscatprintf(out,"(double) %s\n",r->str);
break;
case REDIS_REPLY_STRING:
case REDIS_REPLY_VERB:
/* If you are producing output for the standard output we want
* a more interesting output with quoted characters and so forth */
out = sdscatrepr(out,r->str,r->len);
out = sdscat(out,"\n");
* a more interesting output with quoted characters and so forth,
* unless it's a verbatim string type. */
if (r->type == REDIS_REPLY_STRING) {
out = sdscatrepr(out,r->str,r->len);
out = sdscat(out,"\n");
} else {
out = sdscatlen(out,r->str,r->len);
out = sdscat(out,"\n");
}
break;
case REDIS_REPLY_NIL:
out = sdscat(out,"(nil)\n");
@ -961,6 +1078,7 @@ static sds cliFormatReplyRaw(redisReply *r) {
break;
case REDIS_REPLY_STATUS:
case REDIS_REPLY_STRING:
case REDIS_REPLY_VERB:
if (r->type == REDIS_REPLY_STATUS && config.eval_ldb) {
/* The Lua debugger replies with arrays of simple (status)
* strings. We colorize the output for more fun if this
@ -980,9 +1098,15 @@ static sds cliFormatReplyRaw(redisReply *r) {
out = sdscatlen(out,r->str,r->len);
}
break;
case REDIS_REPLY_BOOL:
out = sdscat(out,r->integer ? "(true)" : "(false)");
break;
case REDIS_REPLY_INTEGER:
out = sdscatprintf(out,"%lld",r->integer);
break;
case REDIS_REPLY_DOUBLE:
out = sdscatprintf(out,"%s",r->str);
break;
case REDIS_REPLY_ARRAY:
for (i = 0; i < r->elements; i++) {
if (i > 0) out = sdscat(out,config.mb_delim);
@ -991,6 +1115,19 @@ static sds cliFormatReplyRaw(redisReply *r) {
sdsfree(tmp);
}
break;
case REDIS_REPLY_MAP:
for (i = 0; i < r->elements; i += 2) {
if (i > 0) out = sdscat(out,config.mb_delim);
tmp = cliFormatReplyRaw(r->element[i]);
out = sdscatlen(out,tmp,sdslen(tmp));
sdsfree(tmp);
out = sdscatlen(out," ",1);
tmp = cliFormatReplyRaw(r->element[i+1]);
out = sdscatlen(out,tmp,sdslen(tmp));
sdsfree(tmp);
}
break;
default:
fprintf(stderr,"Unknown reply type: %d\n", r->type);
exit(1);
@ -1013,13 +1150,21 @@ static sds cliFormatReplyCSV(redisReply *r) {
case REDIS_REPLY_INTEGER:
out = sdscatprintf(out,"%lld",r->integer);
break;
case REDIS_REPLY_DOUBLE:
out = sdscatprintf(out,"%s",r->str);
break;
case REDIS_REPLY_STRING:
case REDIS_REPLY_VERB:
out = sdscatrepr(out,r->str,r->len);
break;
case REDIS_REPLY_NIL:
out = sdscat(out,"NIL");
out = sdscat(out,"NULL");
break;
case REDIS_REPLY_BOOL:
out = sdscat(out,r->integer ? "true" : "false");
break;
case REDIS_REPLY_ARRAY:
case REDIS_REPLY_MAP: /* CSV has no map type, just output flat list. */
for (i = 0; i < r->elements; i++) {
sds tmp = cliFormatReplyCSV(r->element[i]);
out = sdscatlen(out,tmp,sdslen(tmp));
@ -1213,7 +1358,8 @@ static int cliSendCommand(int argc, char **argv, long repeat) {
if (!strcasecmp(command,"select") && argc == 2 && config.last_cmd_type != REDIS_REPLY_ERROR) {
config.dbnum = atoi(argv[1]);
cliRefreshPrompt();
} else if (!strcasecmp(command,"auth") && argc == 2) {
} else if (!strcasecmp(command,"auth") && (argc == 2 || argc == 3))
{
cliSelect();
}
}
@ -1245,6 +1391,13 @@ static redisReply *reconnectingRedisCommand(redisContext *c, const char *fmt, ..
redisFree(c);
c = redisConnect(config.hostip,config.hostport);
if (!c->err && config.tls) {
const char *err = NULL;
if (cliSecureConnection(c, &err) == REDIS_ERR && err) {
fprintf(stderr, "TLS Error: %s\n", err);
exit(1);
}
}
usleep(1000000);
}
@ -1296,8 +1449,12 @@ static int parseOptions(int argc, char **argv) {
config.dbnum = atoi(argv[++i]);
} else if (!strcmp(argv[i], "--no-auth-warning")) {
config.no_auth_warning = 1;
} else if (!strcmp(argv[i],"-a") && !lastarg) {
} else if ((!strcmp(argv[i],"-a") || !strcmp(argv[i],"--pass"))
&& !lastarg)
{
config.auth = argv[++i];
} else if (!strcmp(argv[i],"--user") && !lastarg) {
config.user = argv[++i];
} else if (!strcmp(argv[i],"-u") && !lastarg) {
parseRedisUri(argv[++i]);
} else if (!strcmp(argv[i],"--raw")) {
@ -1434,11 +1591,27 @@ static int parseOptions(int argc, char **argv) {
} else if (!strcmp(argv[i],"--cluster-search-multiple-owners")) {
config.cluster_manager_command.flags |=
CLUSTER_MANAGER_CMD_FLAG_CHECK_OWNERS;
#ifdef USE_OPENSSL
} else if (!strcmp(argv[i],"--tls")) {
config.tls = 1;
} else if (!strcmp(argv[i],"--sni")) {
config.sni = argv[++i];
} else if (!strcmp(argv[i],"--cacertdir")) {
config.cacertdir = argv[++i];
} else if (!strcmp(argv[i],"--cacert")) {
config.cacert = argv[++i];
} else if (!strcmp(argv[i],"--cert")) {
config.cert = argv[++i];
} else if (!strcmp(argv[i],"--key")) {
config.key = argv[++i];
#endif
} else if (!strcmp(argv[i],"-v") || !strcmp(argv[i], "--version")) {
sds version = cliVersion();
printf("redis-cli %s\n", version);
sdsfree(version);
exit(0);
} else if (!strcmp(argv[i],"-3")) {
config.resp3 = 1;
} else if (CLUSTER_MANAGER_MODE() && argv[i][0] != '-') {
if (config.cluster_manager_command.argc == 0) {
int j = i + 1;
@ -1514,14 +1687,26 @@ static void usage(void) {
" You can also use the " REDIS_CLI_AUTH_ENV " environment\n"
" variable to pass this password more safely\n"
" (if both are used, this argument takes predecence).\n"
" -user <username> Used to send ACL style 'AUTH username pass'. Needs -a.\n"
" -pass <password> Alias of -a for consistency with the new --user option.\n"
" -u <uri> Server URI.\n"
" -r <repeat> Execute specified command N times.\n"
" -i <interval> When -r is used, waits <interval> seconds per command.\n"
" It is possible to specify sub-second times like -i 0.1.\n"
" -n <db> Database number.\n"
" -3 Start session in RESP3 protocol mode.\n"
" -x Read last argument from STDIN.\n"
" -d <delimiter> Multi-bulk delimiter in for raw formatting (default: \\n).\n"
" -c Enable cluster mode (follow -ASK and -MOVED redirections).\n"
#ifdef USE_OPENSSL
" --tls Establish a secure TLS connection.\n"
" --cacert CA Certificate file to verify with.\n"
" --cacertdir Directory where trusted CA certificates are stored.\n"
" If neither cacert nor cacertdir are specified, the default\n"
" system-wide trusted root certs configuration will apply.\n"
" --cert Client certificate to authenticate with.\n"
" --key Private key file to authenticate with.\n"
#endif
" --raw Use raw formatting for replies (default when STDOUT is\n"
" not a tty).\n"
" --no-raw Force formatted output even when STDOUT is not a tty.\n"
@ -1533,7 +1718,9 @@ static void usage(void) {
" --csv is specified, or if you redirect the output to a non\n"
" TTY, it samples the latency for 1 second (you can use\n"
" -i to change the interval), then produces a single output\n"
" and exits.\n"
" and exits.\n",version);
fprintf(stderr,
" --latency-history Like --latency but tracking latency changes over time.\n"
" Default time interval is 15 sec. Change it using -i.\n"
" --latency-dist Shows latency as a spectrum, requires xterm 256 colors.\n"
@ -1544,7 +1731,9 @@ static void usage(void) {
" --pipe Transfer raw Redis protocol from stdin to server.\n"
" --pipe-timeout <n> In --pipe mode, abort with error if after sending all data.\n"
" no reply is received within <n> seconds.\n"
" Default timeout: %d. Use 0 to wait forever.\n"
" Default timeout: %d. Use 0 to wait forever.\n",
REDIS_CLI_DEFAULT_PIPE_TIMEOUT);
fprintf(stderr,
" --bigkeys Sample Redis keys looking for keys with many elements (complexity).\n"
" --memkeys Sample Redis keys looking for keys consuming a lot of memory.\n"
" --memkeys-samples <n> Sample Redis keys looking for keys consuming a lot of memory.\n"
@ -1567,8 +1756,7 @@ static void usage(void) {
" line interface.\n"
" --help Output this help and exit.\n"
" --version Output version and exit.\n"
"\n",
version, REDIS_CLI_DEFAULT_PIPE_TIMEOUT);
"\n");
/* Using another fprintf call to avoid -Woverlength-strings compile warning */
fprintf(stderr,
"Cluster Manager Commands:\n"
@ -2336,6 +2524,15 @@ cleanup:
static int clusterManagerNodeConnect(clusterManagerNode *node) {
if (node->context) redisFree(node->context);
node->context = redisConnect(node->ip, node->port);
if (!node->context->err && config.tls) {
const char *err = NULL;
if (cliSecureConnection(node->context, &err) == REDIS_ERR && err) {
fprintf(stderr,"TLS Error: %s\n", err);
redisFree(node->context);
node->context = NULL;
return 0;
}
}
if (node->context->err) {
fprintf(stderr,"Could not connect to Redis at ");
fprintf(stderr,"%s:%d: %s\n", node->ip, node->port,
@ -2350,7 +2547,12 @@ static int clusterManagerNodeConnect(clusterManagerNode *node) {
* errors. */
anetKeepAlive(NULL, node->context->fd, REDIS_CLI_KEEPALIVE_INTERVAL);
if (config.auth) {
redisReply *reply = redisCommand(node->context,"AUTH %s",config.auth);
redisReply *reply;
if (config.user == NULL)
reply = redisCommand(node->context,"AUTH %s", config.auth);
else
reply = redisCommand(node->context,"AUTH %s %s",
config.user,config.auth);
int ok = clusterManagerCheckRedisReply(node, reply, NULL);
if (reply != NULL) freeReplyObject(reply);
if (!ok) return 0;
@ -3222,7 +3424,7 @@ static redisReply *clusterManagerMigrateKeysInReply(clusterManagerNode *source,
redisReply *entry = reply->element[i];
size_t idx = i + offset;
assert(entry->type == REDIS_REPLY_STRING);
argv[idx] = (char *) sdsnew(entry->str);
argv[idx] = (char *) sdsnewlen(entry->str, entry->len);
argv_len[idx] = entry->len;
if (dots) dots[i] = '.';
}
@ -6724,6 +6926,7 @@ static void pipeMode(void) {
/* Handle the readable state: we can read replies from the server. */
if (mask & AE_READABLE) {
ssize_t nread;
int read_error = 0;
/* Read from socket and feed the hiredis reader. */
do {
@ -6731,7 +6934,8 @@ static void pipeMode(void) {
if (nread == -1 && errno != EAGAIN && errno != EINTR) {
fprintf(stderr, "Error reading from the server: %s\n",
strerror(errno));
exit(1);
read_error = 1;
break;
}
if (nread > 0) {
redisReaderFeed(reader,ibuf,nread);
@ -6764,6 +6968,11 @@ static void pipeMode(void) {
freeReplyObject(reply);
}
} while(reply);
/* Abort on read errors. We abort here because it is important
* to consume replies even after a read error: this way we can
* show a potential problem to the user. */
if (read_error) exit(1);
}
/* Handle the writable state: we can send protocol to the server. */
@ -7671,6 +7880,7 @@ int main(int argc, char **argv) {
config.hotkeys = 0;
config.stdinarg = 0;
config.auth = NULL;
config.user = NULL;
config.eval = NULL;
config.eval_ldb = 0;
config.eval_ldb_end = 0;

99
src/redismodule.h
View File

@ -87,7 +87,24 @@
#define REDISMODULE_CTX_FLAGS_OOM_WARNING (1<<11)
/* The command was sent over the replication link. */
#define REDISMODULE_CTX_FLAGS_REPLICATED (1<<12)
/* Redis is currently loading either from AOF or RDB. */
#define REDISMODULE_CTX_FLAGS_LOADING (1<<13)
/* The replica has no link with its master, note that
* there is the inverse flag as well:
*
* REDISMODULE_CTX_FLAGS_REPLICA_IS_ONLINE
*
* The two flags are exclusive, one or the other can be set. */
#define REDISMODULE_CTX_FLAGS_REPLICA_IS_STALE (1<<14)
/* The replica is trying to connect with the master.
* (REPL_STATE_CONNECT and REPL_STATE_CONNECTING states) */
#define REDISMODULE_CTX_FLAGS_REPLICA_IS_CONNECTING (1<<15)
/* THe replica is receiving an RDB file from its master. */
#define REDISMODULE_CTX_FLAGS_REPLICA_IS_TRANSFERRING (1<<16)
/* The replica is online, receiving updates from its master. */
#define REDISMODULE_CTX_FLAGS_REPLICA_IS_ONLINE (1<<17)
/* There is currently some background process active. */
#define REDISMODULE_CTX_FLAGS_ACTIVE_CHILD (1<<18)
#define REDISMODULE_NOTIFY_GENERIC (1<<2) /* g */
#define REDISMODULE_NOTIFY_STRING (1<<3) /* $ */
@ -98,7 +115,8 @@
#define REDISMODULE_NOTIFY_EXPIRED (1<<8) /* x */
#define REDISMODULE_NOTIFY_EVICTED (1<<9) /* e */
#define REDISMODULE_NOTIFY_STREAM (1<<10) /* t */
#define REDISMODULE_NOTIFY_ALL (REDISMODULE_NOTIFY_GENERIC | REDISMODULE_NOTIFY_STRING | REDISMODULE_NOTIFY_LIST | REDISMODULE_NOTIFY_SET | REDISMODULE_NOTIFY_HASH | REDISMODULE_NOTIFY_ZSET | REDISMODULE_NOTIFY_EXPIRED | REDISMODULE_NOTIFY_EVICTED | REDISMODULE_NOTIFY_STREAM) /* A */
#define REDISMODULE_NOTIFY_KEY_MISS (1<<11) /* m */
#define REDISMODULE_NOTIFY_ALL (REDISMODULE_NOTIFY_GENERIC | REDISMODULE_NOTIFY_STRING | REDISMODULE_NOTIFY_LIST | REDISMODULE_NOTIFY_SET | REDISMODULE_NOTIFY_HASH | REDISMODULE_NOTIFY_ZSET | REDISMODULE_NOTIFY_EXPIRED | REDISMODULE_NOTIFY_EVICTED | REDISMODULE_NOTIFY_STREAM | REDISMODULE_NOTIFY_KEY_MISS) /* A */
/* A special pointer that we can use between the core and the module to signal
@ -126,12 +144,24 @@
#define REDISMODULE_NOT_USED(V) ((void) V)
/* Bit flags for aux_save_triggers and the aux_load and aux_save callbacks */
#define REDISMODULE_AUX_BEFORE_RDB (1<<0)
#define REDISMODULE_AUX_AFTER_RDB (1<<1)
/* This type represents a timer handle, and is returned when a timer is
* registered and used in order to invalidate a timer. It's just a 64 bit
* number, because this is how each timer is represented inside the radix tree
* of timers that are going to expire, sorted by expire time. */
typedef uint64_t RedisModuleTimerID;
/* CommandFilter Flags */
/* Do filter RedisModule_Call() commands initiated by module itself. */
#define REDISMODULE_CMDFILTER_NOSELF (1<<0)
/* Declare that the module can handle errors with RedisModule_SetModuleOptions. */
#define REDISMODULE_OPTIONS_HANDLE_IO_ERRORS (1<<0)
/* ------------------------- End of common defines ------------------------ */
#ifndef REDISMODULE_CORE
@ -150,20 +180,28 @@ typedef struct RedisModuleBlockedClient RedisModuleBlockedClient;
typedef struct RedisModuleClusterInfo RedisModuleClusterInfo;
typedef struct RedisModuleDict RedisModuleDict;
typedef struct RedisModuleDictIter RedisModuleDictIter;
typedef struct RedisModuleCommandFilterCtx RedisModuleCommandFilterCtx;
typedef struct RedisModuleCommandFilter RedisModuleCommandFilter;
typedef struct RedisModuleInfoCtx RedisModuleInfoCtx;
typedef int (*RedisModuleCmdFunc)(RedisModuleCtx *ctx, RedisModuleString **argv, int argc);
typedef void (*RedisModuleDisconnectFunc)(RedisModuleCtx *ctx, RedisModuleBlockedClient *bc);
typedef int (*RedisModuleNotificationFunc)(RedisModuleCtx *ctx, int type, const char *event, RedisModuleString *key);
typedef void *(*RedisModuleTypeLoadFunc)(RedisModuleIO *rdb, int encver);
typedef void (*RedisModuleTypeSaveFunc)(RedisModuleIO *rdb, void *value);
typedef int (*RedisModuleTypeAuxLoadFunc)(RedisModuleIO *rdb, int encver, int when);
typedef void (*RedisModuleTypeAuxSaveFunc)(RedisModuleIO *rdb, int when);
typedef void (*RedisModuleTypeRewriteFunc)(RedisModuleIO *aof, RedisModuleString *key, void *value);
typedef size_t (*RedisModuleTypeMemUsageFunc)(const void *value);
typedef void (*RedisModuleTypeDigestFunc)(RedisModuleDigest *digest, void *value);
typedef void (*RedisModuleTypeFreeFunc)(void *value);
typedef void (*RedisModuleClusterMessageReceiver)(RedisModuleCtx *ctx, const char *sender_id, uint8_t type, const unsigned char *payload, uint32_t len);
typedef void (*RedisModuleTimerProc)(RedisModuleCtx *ctx, void *data);
typedef void (*RedisModuleCommandFilterFunc) (RedisModuleCommandFilterCtx *filter);
typedef void (*RedisModuleForkDoneHandler) (int exitcode, int bysignal, void *user_data);
typedef void (*RedisModuleInfoFunc)(RedisModuleInfoCtx *ctx, int for_crash_report);
#define REDISMODULE_TYPE_METHOD_VERSION 1
#define REDISMODULE_TYPE_METHOD_VERSION 2
typedef struct RedisModuleTypeMethods {
uint64_t version;
RedisModuleTypeLoadFunc rdb_load;
@ -172,6 +210,9 @@ typedef struct RedisModuleTypeMethods {
RedisModuleTypeMemUsageFunc mem_usage;
RedisModuleTypeDigestFunc digest;
RedisModuleTypeFreeFunc free;
RedisModuleTypeAuxLoadFunc aux_load;
RedisModuleTypeAuxSaveFunc aux_save;
int aux_save_triggers;
} RedisModuleTypeMethods;
#define REDISMODULE_GET_API(name) \
@ -217,6 +258,7 @@ int REDISMODULE_API_FUNC(RedisModule_ReplyWithSimpleString)(RedisModuleCtx *ctx,
int REDISMODULE_API_FUNC(RedisModule_ReplyWithArray)(RedisModuleCtx *ctx, long len);
void REDISMODULE_API_FUNC(RedisModule_ReplySetArrayLength)(RedisModuleCtx *ctx, long len);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithStringBuffer)(RedisModuleCtx *ctx, const char *buf, size_t len);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithCString)(RedisModuleCtx *ctx, const char *buf);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithString)(RedisModuleCtx *ctx, RedisModuleString *str);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithNull)(RedisModuleCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_ReplyWithDouble)(RedisModuleCtx *ctx, double d);
@ -259,6 +301,8 @@ RedisModuleType *REDISMODULE_API_FUNC(RedisModule_CreateDataType)(RedisModuleCtx
int REDISMODULE_API_FUNC(RedisModule_ModuleTypeSetValue)(RedisModuleKey *key, RedisModuleType *mt, void *value);
RedisModuleType *REDISMODULE_API_FUNC(RedisModule_ModuleTypeGetType)(RedisModuleKey *key);
void *REDISMODULE_API_FUNC(RedisModule_ModuleTypeGetValue)(RedisModuleKey *key);
int REDISMODULE_API_FUNC(RedisModule_IsIOError)(RedisModuleIO *io);
void REDISMODULE_API_FUNC(RedisModule_SetModuleOptions)(RedisModuleCtx *ctx, int options);
void REDISMODULE_API_FUNC(RedisModule_SaveUnsigned)(RedisModuleIO *io, uint64_t value);
uint64_t REDISMODULE_API_FUNC(RedisModule_LoadUnsigned)(RedisModuleIO *io);
void REDISMODULE_API_FUNC(RedisModule_SaveSigned)(RedisModuleIO *io, int64_t value);
@ -274,10 +318,12 @@ void REDISMODULE_API_FUNC(RedisModule_SaveFloat)(RedisModuleIO *io, float value)
float REDISMODULE_API_FUNC(RedisModule_LoadFloat)(RedisModuleIO *io);
void REDISMODULE_API_FUNC(RedisModule_Log)(RedisModuleCtx *ctx, const char *level, const char *fmt, ...);
void REDISMODULE_API_FUNC(RedisModule_LogIOError)(RedisModuleIO *io, const char *levelstr, const char *fmt, ...);
void REDISMODULE_API_FUNC(RedisModule__Assert)(const char *estr, const char *file, int line);
int REDISMODULE_API_FUNC(RedisModule_StringAppendBuffer)(RedisModuleCtx *ctx, RedisModuleString *str, const char *buf, size_t len);
void REDISMODULE_API_FUNC(RedisModule_RetainString)(RedisModuleCtx *ctx, RedisModuleString *str);
int REDISMODULE_API_FUNC(RedisModule_StringCompare)(RedisModuleString *a, RedisModuleString *b);
RedisModuleCtx *REDISMODULE_API_FUNC(RedisModule_GetContextFromIO)(RedisModuleIO *io);
const RedisModuleString *REDISMODULE_API_FUNC(RedisModule_GetKeyNameFromIO)(RedisModuleIO *io);
long long REDISMODULE_API_FUNC(RedisModule_Milliseconds)(void);
void REDISMODULE_API_FUNC(RedisModule_DigestAddStringBuffer)(RedisModuleDigest *md, unsigned char *ele, size_t len);
void REDISMODULE_API_FUNC(RedisModule_DigestAddLongLong)(RedisModuleDigest *md, long long ele);
@ -304,6 +350,15 @@ RedisModuleString *REDISMODULE_API_FUNC(RedisModule_DictNext)(RedisModuleCtx *ct
RedisModuleString *REDISMODULE_API_FUNC(RedisModule_DictPrev)(RedisModuleCtx *ctx, RedisModuleDictIter *di, void **dataptr);
int REDISMODULE_API_FUNC(RedisModule_DictCompareC)(RedisModuleDictIter *di, const char *op, void *key, size_t keylen);
int REDISMODULE_API_FUNC(RedisModule_DictCompare)(RedisModuleDictIter *di, const char *op, RedisModuleString *key);
int REDISMODULE_API_FUNC(RedisModule_RegisterInfoFunc)(RedisModuleCtx *ctx, RedisModuleInfoFunc cb);
int REDISMODULE_API_FUNC(RedisModule_InfoAddSection)(RedisModuleInfoCtx *ctx, char *name);
int REDISMODULE_API_FUNC(RedisModule_InfoBeginDictField)(RedisModuleInfoCtx *ctx, char *name);
int REDISMODULE_API_FUNC(RedisModule_InfoEndDictField)(RedisModuleInfoCtx *ctx);
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldString)(RedisModuleInfoCtx *ctx, char *field, RedisModuleString *value);
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldCString)(RedisModuleInfoCtx *ctx, char *field, char *value);
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldDouble)(RedisModuleInfoCtx *ctx, char *field, double value);
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldLongLong)(RedisModuleInfoCtx *ctx, char *field, long long value);
int REDISMODULE_API_FUNC(RedisModule_InfoAddFieldULongLong)(RedisModuleInfoCtx *ctx, char *field, unsigned long long value);
/* Experimental APIs */
#ifdef REDISMODULE_EXPERIMENTAL_API
@ -337,8 +392,20 @@ void REDISMODULE_API_FUNC(RedisModule_SetDisconnectCallback)(RedisModuleBlockedC
void REDISMODULE_API_FUNC(RedisModule_SetClusterFlags)(RedisModuleCtx *ctx, uint64_t flags);
int REDISMODULE_API_FUNC(RedisModule_ExportSharedAPI)(RedisModuleCtx *ctx, const char *apiname, void *func);
void *REDISMODULE_API_FUNC(RedisModule_GetSharedAPI)(RedisModuleCtx *ctx, const char *apiname);
RedisModuleCommandFilter *REDISMODULE_API_FUNC(RedisModule_RegisterCommandFilter)(RedisModuleCtx *ctx, RedisModuleCommandFilterFunc cb, int flags);
int REDISMODULE_API_FUNC(RedisModule_UnregisterCommandFilter)(RedisModuleCtx *ctx, RedisModuleCommandFilter *filter);
int REDISMODULE_API_FUNC(RedisModule_CommandFilterArgsCount)(RedisModuleCommandFilterCtx *fctx);
const RedisModuleString *REDISMODULE_API_FUNC(RedisModule_CommandFilterArgGet)(RedisModuleCommandFilterCtx *fctx, int pos);
int REDISMODULE_API_FUNC(RedisModule_CommandFilterArgInsert)(RedisModuleCommandFilterCtx *fctx, int pos, RedisModuleString *arg);
int REDISMODULE_API_FUNC(RedisModule_CommandFilterArgReplace)(RedisModuleCommandFilterCtx *fctx, int pos, RedisModuleString *arg);
int REDISMODULE_API_FUNC(RedisModule_CommandFilterArgDelete)(RedisModuleCommandFilterCtx *fctx, int pos);
int REDISMODULE_API_FUNC(RedisModule_Fork)(RedisModuleForkDoneHandler cb, void *user_data);
int REDISMODULE_API_FUNC(RedisModule_ExitFromChild)(int retcode);
int REDISMODULE_API_FUNC(RedisModule_KillForkChild)(int child_pid);
#endif
#define RedisModule_IsAOFClient(id) ((id) == UINT64_MAX)
/* This is included inline inside each Redis module. */
static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int apiver) __attribute__((unused));
static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int apiver) {
@ -359,6 +426,7 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(ReplyWithArray);
REDISMODULE_GET_API(ReplySetArrayLength);
REDISMODULE_GET_API(ReplyWithStringBuffer);
REDISMODULE_GET_API(ReplyWithCString);
REDISMODULE_GET_API(ReplyWithString);
REDISMODULE_GET_API(ReplyWithNull);
REDISMODULE_GET_API(ReplyWithCallReply);
@ -423,6 +491,8 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(ModuleTypeSetValue);
REDISMODULE_GET_API(ModuleTypeGetType);
REDISMODULE_GET_API(ModuleTypeGetValue);
REDISMODULE_GET_API(IsIOError);
REDISMODULE_GET_API(SetModuleOptions);
REDISMODULE_GET_API(SaveUnsigned);
REDISMODULE_GET_API(LoadUnsigned);
REDISMODULE_GET_API(SaveSigned);
@ -438,10 +508,12 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(EmitAOF);
REDISMODULE_GET_API(Log);
REDISMODULE_GET_API(LogIOError);
REDISMODULE_GET_API(_Assert);
REDISMODULE_GET_API(StringAppendBuffer);
REDISMODULE_GET_API(RetainString);
REDISMODULE_GET_API(StringCompare);
REDISMODULE_GET_API(GetContextFromIO);
REDISMODULE_GET_API(GetKeyNameFromIO);
REDISMODULE_GET_API(Milliseconds);
REDISMODULE_GET_API(DigestAddStringBuffer);
REDISMODULE_GET_API(DigestAddLongLong);
@ -468,6 +540,15 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(DictPrev);
REDISMODULE_GET_API(DictCompare);
REDISMODULE_GET_API(DictCompareC);
REDISMODULE_GET_API(RegisterInfoFunc);
REDISMODULE_GET_API(InfoAddSection);
REDISMODULE_GET_API(InfoBeginDictField);
REDISMODULE_GET_API(InfoEndDictField);
REDISMODULE_GET_API(InfoAddFieldString);
REDISMODULE_GET_API(InfoAddFieldCString);
REDISMODULE_GET_API(InfoAddFieldDouble);
REDISMODULE_GET_API(InfoAddFieldLongLong);
REDISMODULE_GET_API(InfoAddFieldULongLong);
#ifdef REDISMODULE_EXPERIMENTAL_API
REDISMODULE_GET_API(GetThreadSafeContext);
@ -499,6 +580,16 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
REDISMODULE_GET_API(SetClusterFlags);
REDISMODULE_GET_API(ExportSharedAPI);
REDISMODULE_GET_API(GetSharedAPI);
REDISMODULE_GET_API(RegisterCommandFilter);
REDISMODULE_GET_API(UnregisterCommandFilter);
REDISMODULE_GET_API(CommandFilterArgsCount);
REDISMODULE_GET_API(CommandFilterArgGet);
REDISMODULE_GET_API(CommandFilterArgInsert);
REDISMODULE_GET_API(CommandFilterArgReplace);
REDISMODULE_GET_API(CommandFilterArgDelete);
REDISMODULE_GET_API(Fork);
REDISMODULE_GET_API(ExitFromChild);
REDISMODULE_GET_API(KillForkChild);
#endif
if (RedisModule_IsModuleNameBusy && RedisModule_IsModuleNameBusy(name)) return REDISMODULE_ERR;
@ -506,6 +597,8 @@ static int RedisModule_Init(RedisModuleCtx *ctx, const char *name, int ver, int
return REDISMODULE_OK;
}
#define RedisModule_Assert(_e) ((_e)?(void)0 : (RedisModule__Assert(#_e,__FILE__,__LINE__),exit(1)))
#else
/* Things only defined for the modules core, not exported to modules

14
src/release.c
View File

@ -32,6 +32,7 @@
* files using this functions. */
#include <string.h>
#include <stdio.h>
#include "release.h"
#include "version.h"
@ -50,3 +51,16 @@ uint64_t redisBuildId(void) {
return crc64(0,(unsigned char*)buildid,strlen(buildid));
}
/* Return a cached value of the build string in order to avoid recomputing
* and converting it in hex every time: this string is shown in the INFO
* output that should be fast. */
char *redisBuildIdString(void) {
static char buf[32];
static int cached = 0;
if (!cached) {
snprintf(buf,sizeof(buf),"%llx",(unsigned long long) redisBuildId());
cached = 1;
}
return buf;
}

869
src/replication.c
View File

File diff suppressed because it is too large Load Diff

262
src/rio.c
View File

@ -92,6 +92,7 @@ static const rio rioBufferIO = {
rioBufferFlush,
NULL, /* update_checksum */
0, /* current checksum */
0, /* flags */
0, /* bytes read or written */
0, /* read/write chunk size */
{ { NULL, 0 } } /* union for io-specific vars */
@ -145,6 +146,7 @@ static const rio rioFileIO = {
rioFileFlush,
NULL, /* update_checksum */
0, /* current checksum */
0, /* flags */
0, /* bytes read or written */
0, /* read/write chunk size */
{ { NULL, 0 } } /* union for io-specific vars */
@ -157,81 +159,180 @@ void rioInitWithFile(rio *r, FILE *fp) {
r->io.file.autosync = 0;
}
/* ------------------- File descriptors set implementation ------------------- */
/* ------------------- Connection implementation -------------------
* We use this RIO implemetnation when reading an RDB file directly from
* the connection to the memory via rdbLoadRio(), thus this implementation
* only implements reading from a connection that is, normally,
* just a socket. */
static size_t rioConnWrite(rio *r, const void *buf, size_t len) {
UNUSED(r);
UNUSED(buf);
UNUSED(len);
return 0; /* Error, this target does not yet support writing. */
}
/* Returns 1 or 0 for success/failure. */
static size_t rioConnRead(rio *r, void *buf, size_t len) {
size_t avail = sdslen(r->io.conn.buf)-r->io.conn.pos;
/* If the buffer is too small for the entire request: realloc. */
if (sdslen(r->io.conn.buf) + sdsavail(r->io.conn.buf) < len)
r->io.conn.buf = sdsMakeRoomFor(r->io.conn.buf, len - sdslen(r->io.conn.buf));
/* If the remaining unused buffer is not large enough: memmove so that we
* can read the rest. */
if (len > avail && sdsavail(r->io.conn.buf) < len - avail) {
sdsrange(r->io.conn.buf, r->io.conn.pos, -1);
r->io.conn.pos = 0;
}
/* If we don't already have all the data in the sds, read more */
while (len > sdslen(r->io.conn.buf) - r->io.conn.pos) {
size_t buffered = sdslen(r->io.conn.buf) - r->io.conn.pos;
size_t toread = len - buffered;
/* Read either what's missing, or PROTO_IOBUF_LEN, the bigger of
* the two. */
if (toread < PROTO_IOBUF_LEN) toread = PROTO_IOBUF_LEN;
if (toread > sdsavail(r->io.conn.buf)) toread = sdsavail(r->io.conn.buf);
if (r->io.conn.read_limit != 0 &&
r->io.conn.read_so_far + buffered + toread > r->io.conn.read_limit)
{
if (r->io.conn.read_limit >= r->io.conn.read_so_far - buffered)
toread = r->io.conn.read_limit - r->io.conn.read_so_far - buffered;
else {
errno = EOVERFLOW;
return 0;
}
}
int retval = connRead(r->io.conn.conn,
(char*)r->io.conn.buf + sdslen(r->io.conn.buf),
toread);
if (retval <= 0) {
if (errno == EWOULDBLOCK) errno = ETIMEDOUT;
return 0;
}
sdsIncrLen(r->io.conn.buf, retval);
}
memcpy(buf, (char*)r->io.conn.buf + r->io.conn.pos, len);
r->io.conn.read_so_far += len;
r->io.conn.pos += len;
return len;
}
/* Returns read/write position in file. */
static off_t rioConnTell(rio *r) {
return r->io.conn.read_so_far;
}
/* Flushes any buffer to target device if applicable. Returns 1 on success
* and 0 on failures. */
static int rioConnFlush(rio *r) {
/* Our flush is implemented by the write method, that recognizes a
* buffer set to NULL with a count of zero as a flush request. */
return rioConnWrite(r,NULL,0);
}
static const rio rioConnIO = {
rioConnRead,
rioConnWrite,
rioConnTell,
rioConnFlush,
NULL, /* update_checksum */
0, /* current checksum */
0, /* flags */
0, /* bytes read or written */
0, /* read/write chunk size */
{ { NULL, 0 } } /* union for io-specific vars */
};
/* Create an RIO that implements a buffered read from an fd
* read_limit argument stops buffering when the reaching the limit. */
void rioInitWithConn(rio *r, connection *conn, size_t read_limit) {
*r = rioConnIO;
r->io.conn.conn = conn;
r->io.conn.pos = 0;
r->io.conn.read_limit = read_limit;
r->io.conn.read_so_far = 0;
r->io.conn.buf = sdsnewlen(NULL, PROTO_IOBUF_LEN);
sdsclear(r->io.conn.buf);
}
/* Release the RIO tream. Optionally returns the unread buffered data
* when the SDS pointer 'remaining' is passed. */
void rioFreeConn(rio *r, sds *remaining) {
if (remaining && (size_t)r->io.conn.pos < sdslen(r->io.conn.buf)) {
if (r->io.conn.pos > 0) sdsrange(r->io.conn.buf, r->io.conn.pos, -1);
*remaining = r->io.conn.buf;
} else {
sdsfree(r->io.conn.buf);
if (remaining) *remaining = NULL;
}
r->io.conn.buf = NULL;
}
/* ------------------- File descriptor implementation ------------------
* This target is used to write the RDB file to pipe, when the master just
* streams the data to the replicas without creating an RDB on-disk image
* (diskless replication option).
* It only implements writes. */
/* Returns 1 or 0 for success/failure.
* The function returns success as long as we are able to correctly write
* to at least one file descriptor.
*
* When buf is NULL and len is 0, the function performs a flush operation
* if there is some pending buffer, so this function is also used in order
* to implement rioFdsetFlush(). */
static size_t rioFdsetWrite(rio *r, const void *buf, size_t len) {
* to implement rioFdFlush(). */
static size_t rioFdWrite(rio *r, const void *buf, size_t len) {
ssize_t retval;
int j;
unsigned char *p = (unsigned char*) buf;
int doflush = (buf == NULL && len == 0);
/* To start we always append to our buffer. If it gets larger than
* a given size, we actually write to the sockets. */
if (len) {
r->io.fdset.buf = sdscatlen(r->io.fdset.buf,buf,len);
len = 0; /* Prevent entering the while below if we don't flush. */
if (sdslen(r->io.fdset.buf) > PROTO_IOBUF_LEN) doflush = 1;
}
if (doflush) {
p = (unsigned char*) r->io.fdset.buf;
len = sdslen(r->io.fdset.buf);
}
/* Write in little chunchs so that when there are big writes we
* parallelize while the kernel is sending data in background to
* the TCP socket. */
while(len) {
size_t count = len < 1024 ? len : 1024;
int broken = 0;
for (j = 0; j < r->io.fdset.numfds; j++) {
if (r->io.fdset.state[j] != 0) {
/* Skip FDs alraedy in error. */
broken++;
continue;
}
/* Make sure to write 'count' bytes to the socket regardless
* of short writes. */
size_t nwritten = 0;
while(nwritten != count) {
retval = write(r->io.fdset.fds[j],p+nwritten,count-nwritten);
if (retval <= 0) {
/* With blocking sockets, which is the sole user of this
* rio target, EWOULDBLOCK is returned only because of
* the SO_SNDTIMEO socket option, so we translate the error
* into one more recognizable by the user. */
if (retval == -1 && errno == EWOULDBLOCK) errno = ETIMEDOUT;
break;
}
nwritten += retval;
}
if (nwritten != count) {
/* Mark this FD as broken. */
r->io.fdset.state[j] = errno;
if (r->io.fdset.state[j] == 0) r->io.fdset.state[j] = EIO;
}
/* For small writes, we rather keep the data in user-space buffer, and flush
* it only when it grows. however for larger writes, we prefer to flush
* any pre-existing buffer, and write the new one directly without reallocs
* and memory copying. */
if (len > PROTO_IOBUF_LEN) {
/* First, flush any pre-existing buffered data. */
if (sdslen(r->io.fd.buf)) {
if (rioFdWrite(r, NULL, 0) == 0)
return 0;
}
if (broken == r->io.fdset.numfds) return 0; /* All the FDs in error. */
p += count;
len -= count;
r->io.fdset.pos += count;
/* Write the new data, keeping 'p' and 'len' from the input. */
} else {
if (len) {
r->io.fd.buf = sdscatlen(r->io.fd.buf,buf,len);
if (sdslen(r->io.fd.buf) > PROTO_IOBUF_LEN)
doflush = 1;
if (!doflush)
return 1;
}
/* Flusing the buffered data. set 'p' and 'len' accordintly. */
p = (unsigned char*) r->io.fd.buf;
len = sdslen(r->io.fd.buf);
}
if (doflush) sdsclear(r->io.fdset.buf);
size_t nwritten = 0;
while(nwritten != len) {
retval = write(r->io.fd.fd,p+nwritten,len-nwritten);
if (retval <= 0) {
/* With blocking io, which is the sole user of this
* rio target, EWOULDBLOCK is returned only because of
* the SO_SNDTIMEO socket option, so we translate the error
* into one more recognizable by the user. */
if (retval == -1 && errno == EWOULDBLOCK) errno = ETIMEDOUT;
return 0; /* error. */
}
nwritten += retval;
}
r->io.fd.pos += len;
sdsclear(r->io.fd.buf);
return 1;
}
/* Returns 1 or 0 for success/failure. */
static size_t rioFdsetRead(rio *r, void *buf, size_t len) {
static size_t rioFdRead(rio *r, void *buf, size_t len) {
UNUSED(r);
UNUSED(buf);
UNUSED(len);
@ -239,48 +340,41 @@ static size_t rioFdsetRead(rio *r, void *buf, size_t len) {
}
/* Returns read/write position in file. */
static off_t rioFdsetTell(rio *r) {
return r->io.fdset.pos;
static off_t rioFdTell(rio *r) {
return r->io.fd.pos;
}
/* Flushes any buffer to target device if applicable. Returns 1 on success
* and 0 on failures. */
static int rioFdsetFlush(rio *r) {
static int rioFdFlush(rio *r) {
/* Our flush is implemented by the write method, that recognizes a
* buffer set to NULL with a count of zero as a flush request. */
return rioFdsetWrite(r,NULL,0);
return rioFdWrite(r,NULL,0);
}
static const rio rioFdsetIO = {
rioFdsetRead,
rioFdsetWrite,
rioFdsetTell,
rioFdsetFlush,
static const rio rioFdIO = {
rioFdRead,
rioFdWrite,
rioFdTell,
rioFdFlush,
NULL, /* update_checksum */
0, /* current checksum */
0, /* flags */
0, /* bytes read or written */
0, /* read/write chunk size */
{ { NULL, 0 } } /* union for io-specific vars */
};
void rioInitWithFdset(rio *r, int *fds, int numfds) {
int j;
*r = rioFdsetIO;
r->io.fdset.fds = zmalloc(sizeof(int)*numfds);
r->io.fdset.state = zmalloc(sizeof(int)*numfds);
memcpy(r->io.fdset.fds,fds,sizeof(int)*numfds);
for (j = 0; j < numfds; j++) r->io.fdset.state[j] = 0;
r->io.fdset.numfds = numfds;
r->io.fdset.pos = 0;
r->io.fdset.buf = sdsempty();
void rioInitWithFd(rio *r, int fd) {
*r = rioFdIO;
r->io.fd.fd = fd;
r->io.fd.pos = 0;
r->io.fd.buf = sdsempty();
}
/* release the rio stream. */
void rioFreeFdset(rio *r) {
zfree(r->io.fdset.fds);
zfree(r->io.fdset.state);
sdsfree(r->io.fdset.buf);
void rioFreeFd(rio *r) {
sdsfree(r->io.fd.buf);
}
/* ---------------------------- Generic functions ---------------------------- */
@ -300,7 +394,7 @@ void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len) {
* disk I/O concentrated in very little time. When we fsync in an explicit
* way instead the I/O pressure is more distributed across time. */
void rioSetAutoSync(rio *r, off_t bytes) {
serverAssert(r->read == rioFileIO.read);
if(r->write != rioFileIO.write) return;
r->io.file.autosync = bytes;
}

58
src/rio.h
View File

@ -1,6 +1,6 @@
/*
* Copyright (c) 2009-2012, Pieter Noordhuis <pcnoordhuis at gmail dot com>
* Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2009-2019, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@ -35,6 +35,10 @@
#include <stdio.h>
#include <stdint.h>
#include "sds.h"
#include "connection.h"
#define RIO_FLAG_READ_ERROR (1<<0)
#define RIO_FLAG_WRITE_ERROR (1<<1)
struct _rio {
/* Backend functions.
@ -51,8 +55,8 @@ struct _rio {
* computation. */
void (*update_cksum)(struct _rio *, const void *buf, size_t len);
/* The current checksum */
uint64_t cksum;
/* The current checksum and flags (see RIO_FLAG_*) */
uint64_t cksum, flags;
/* number of bytes read or written */
size_t processed_bytes;
@ -73,14 +77,20 @@ struct _rio {
off_t buffered; /* Bytes written since last fsync. */
off_t autosync; /* fsync after 'autosync' bytes written. */
} file;
/* Multiple FDs target (used to write to N sockets). */
/* Connection object (used to read from socket) */
struct {
int *fds; /* File descriptors. */
int *state; /* Error state of each fd. 0 (if ok) or errno. */
int numfds;
connection *conn; /* Connection */
off_t pos; /* pos in buf that was returned */
sds buf; /* buffered data */
size_t read_limit; /* don't allow to buffer/read more than that */
size_t read_so_far; /* amount of data read from the rio (not buffered) */
} conn;
/* FD target (used to write to pipe). */
struct {
int fd; /* File descriptor. */
off_t pos;
sds buf;
} fdset;
} fd;
} io;
};
@ -91,11 +101,14 @@ typedef struct _rio rio;
* if needed. */
static inline size_t rioWrite(rio *r, const void *buf, size_t len) {
if (r->flags & RIO_FLAG_WRITE_ERROR) return 0;
while (len) {
size_t bytes_to_write = (r->max_processing_chunk && r->max_processing_chunk < len) ? r->max_processing_chunk : len;
if (r->update_cksum) r->update_cksum(r,buf,bytes_to_write);
if (r->write(r,buf,bytes_to_write) == 0)
if (r->write(r,buf,bytes_to_write) == 0) {
r->flags |= RIO_FLAG_WRITE_ERROR;
return 0;
}
buf = (char*)buf + bytes_to_write;
len -= bytes_to_write;
r->processed_bytes += bytes_to_write;
@ -104,10 +117,13 @@ static inline size_t rioWrite(rio *r, const void *buf, size_t len) {
}
static inline size_t rioRead(rio *r, void *buf, size_t len) {
if (r->flags & RIO_FLAG_READ_ERROR) return 0;
while (len) {
size_t bytes_to_read = (r->max_processing_chunk && r->max_processing_chunk < len) ? r->max_processing_chunk : len;
if (r->read(r,buf,bytes_to_read) == 0)
if (r->read(r,buf,bytes_to_read) == 0) {
r->flags |= RIO_FLAG_READ_ERROR;
return 0;
}
if (r->update_cksum) r->update_cksum(r,buf,bytes_to_read);
buf = (char*)buf + bytes_to_read;
len -= bytes_to_read;
@ -124,11 +140,29 @@ static inline int rioFlush(rio *r) {
return r->flush(r);
}
/* This function allows to know if there was a read error in any past
* operation, since the rio stream was created or since the last call
* to rioClearError(). */
static inline int rioGetReadError(rio *r) {
return (r->flags & RIO_FLAG_READ_ERROR) != 0;
}
/* Like rioGetReadError() but for write errors. */
static inline int rioGetWriteError(rio *r) {
return (r->flags & RIO_FLAG_WRITE_ERROR) != 0;
}
static inline void rioClearErrors(rio *r) {
r->flags &= ~(RIO_FLAG_READ_ERROR|RIO_FLAG_WRITE_ERROR);
}
void rioInitWithFile(rio *r, FILE *fp);
void rioInitWithBuffer(rio *r, sds s);
void rioInitWithFdset(rio *r, int *fds, int numfds);
void rioInitWithConn(rio *r, connection *conn, size_t read_limit);
void rioInitWithFd(rio *r, int fd);
void rioFreeFdset(rio *r);
void rioFreeFd(rio *r);
void rioFreeConn(rio *r, sds* out_remainingBufferedData);
size_t rioWriteBulkCount(rio *r, char prefix, long count);
size_t rioWriteBulkString(rio *r, const char *buf, size_t len);

297
src/scripting.c
View File

@ -42,7 +42,10 @@ char *redisProtocolToLuaType_Int(lua_State *lua, char *reply);
char *redisProtocolToLuaType_Bulk(lua_State *lua, char *reply);
char *redisProtocolToLuaType_Status(lua_State *lua, char *reply);
char *redisProtocolToLuaType_Error(lua_State *lua, char *reply);
char *redisProtocolToLuaType_MultiBulk(lua_State *lua, char *reply, int atype);
char *redisProtocolToLuaType_Aggregate(lua_State *lua, char *reply, int atype);
char *redisProtocolToLuaType_Null(lua_State *lua, char *reply);
char *redisProtocolToLuaType_Bool(lua_State *lua, char *reply, int tf);
char *redisProtocolToLuaType_Double(lua_State *lua, char *reply);
int redis_math_random (lua_State *L);
int redis_math_randomseed (lua_State *L);
void ldbInit(void);
@ -58,7 +61,7 @@ sds ldbCatStackValue(sds s, lua_State *lua, int idx);
#define LDB_BREAKPOINTS_MAX 64 /* Max number of breakpoints. */
#define LDB_MAX_LEN_DEFAULT 256 /* Default len limit for replies / var dumps. */
struct ldbState {
int fd; /* Socket of the debugging client. */
connection *conn; /* Connection of the debugging client. */
int active; /* Are we debugging EVAL right now? */
int forked; /* Is this a fork()ed debugging session? */
list *logs; /* List of messages to send to the client. */
@ -132,9 +135,12 @@ char *redisProtocolToLuaType(lua_State *lua, char* reply) {
case '$': p = redisProtocolToLuaType_Bulk(lua,reply); break;
case '+': p = redisProtocolToLuaType_Status(lua,reply); break;
case '-': p = redisProtocolToLuaType_Error(lua,reply); break;
case '*': p = redisProtocolToLuaType_MultiBulk(lua,reply,*p); break;
case '%': p = redisProtocolToLuaType_MultiBulk(lua,reply,*p); break;
case '~': p = redisProtocolToLuaType_MultiBulk(lua,reply,*p); break;
case '*': p = redisProtocolToLuaType_Aggregate(lua,reply,*p); break;
case '%': p = redisProtocolToLuaType_Aggregate(lua,reply,*p); break;
case '~': p = redisProtocolToLuaType_Aggregate(lua,reply,*p); break;
case '_': p = redisProtocolToLuaType_Null(lua,reply); break;
case '#': p = redisProtocolToLuaType_Bool(lua,reply,p[1]); break;
case ',': p = redisProtocolToLuaType_Double(lua,reply); break;
}
return p;
}
@ -182,13 +188,13 @@ char *redisProtocolToLuaType_Error(lua_State *lua, char *reply) {
return p+2;
}
char *redisProtocolToLuaType_MultiBulk(lua_State *lua, char *reply, int atype) {
char *redisProtocolToLuaType_Aggregate(lua_State *lua, char *reply, int atype) {
char *p = strchr(reply+1,'\r');
long long mbulklen;
int j = 0;
string2ll(reply+1,p-reply-1,&mbulklen);
if (server.lua_caller->resp == 2 || atype == '*') {
if (server.lua_client->resp == 2 || atype == '*') {
p += 2;
if (mbulklen == -1) {
lua_pushboolean(lua,0);
@ -200,11 +206,15 @@ char *redisProtocolToLuaType_MultiBulk(lua_State *lua, char *reply, int atype) {
p = redisProtocolToLuaType(lua,p);
lua_settable(lua,-3);
}
} else if (server.lua_caller->resp == 3) {
} else if (server.lua_client->resp == 3) {
/* Here we handle only Set and Map replies in RESP3 mode, since arrays
* follow the above RESP2 code path. */
* follow the above RESP2 code path. Note that those are represented
* as a table with the "map" or "set" field populated with the actual
* table representing the set or the map type. */
p += 2;
lua_newtable(lua);
lua_pushstring(lua,atype == '%' ? "map" : "set");
lua_newtable(lua);
for (j = 0; j < mbulklen; j++) {
p = redisProtocolToLuaType(lua,p);
if (atype == '%') {
@ -214,10 +224,44 @@ char *redisProtocolToLuaType_MultiBulk(lua_State *lua, char *reply, int atype) {
}
lua_settable(lua,-3);
}
lua_settable(lua,-3);
}
return p;
}
char *redisProtocolToLuaType_Null(lua_State *lua, char *reply) {
char *p = strchr(reply+1,'\r');
lua_pushnil(lua);
return p+2;
}
char *redisProtocolToLuaType_Bool(lua_State *lua, char *reply, int tf) {
char *p = strchr(reply+1,'\r');
lua_pushboolean(lua,tf == 't');
return p+2;
}
char *redisProtocolToLuaType_Double(lua_State *lua, char *reply) {
char *p = strchr(reply+1,'\r');
char buf[MAX_LONG_DOUBLE_CHARS+1];
size_t len = p-reply-1;
double d;
if (len <= MAX_LONG_DOUBLE_CHARS) {
memcpy(buf,reply+1,len);
buf[len] = '\0';
d = strtod(buf,NULL); /* We expect a valid representation. */
} else {
d = 0;
}
lua_newtable(lua);
lua_pushstring(lua,"double");
lua_pushnumber(lua,d);
lua_settable(lua,-3);
return p+2;
}
/* This function is used in order to push an error on the Lua stack in the
* format used by redis.pcall to return errors, which is a lua table
* with a single "err" field set to the error string. Note that this
@ -292,6 +336,8 @@ void luaSortArray(lua_State *lua) {
* Lua reply to Redis reply conversion functions.
* ------------------------------------------------------------------------- */
/* Reply to client 'c' converting the top element in the Lua stack to a
* Redis reply. As a side effect the element is consumed from the stack. */
void luaReplyToRedisReply(client *c, lua_State *lua) {
int t = lua_type(lua,-1);
@ -300,7 +346,11 @@ void luaReplyToRedisReply(client *c, lua_State *lua) {
addReplyBulkCBuffer(c,(char*)lua_tostring(lua,-1),lua_strlen(lua,-1));
break;
case LUA_TBOOLEAN:
addReply(c,lua_toboolean(lua,-1) ? shared.cone : shared.null[c->resp]);
if (server.lua_client->resp == 2)
addReply(c,lua_toboolean(lua,-1) ? shared.cone :
shared.null[c->resp]);
else
addReplyBool(c,lua_toboolean(lua,-1));
break;
case LUA_TNUMBER:
addReplyLongLong(c,(long long)lua_tonumber(lua,-1));
@ -310,6 +360,8 @@ void luaReplyToRedisReply(client *c, lua_State *lua) {
* Error are returned as a single element table with 'err' field.
* Status replies are returned as single element table with 'ok'
* field. */
/* Handle error reply. */
lua_pushstring(lua,"err");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
@ -321,8 +373,9 @@ void luaReplyToRedisReply(client *c, lua_State *lua) {
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
lua_pop(lua,1);
/* Handle status reply. */
lua_pushstring(lua,"ok");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
@ -331,25 +384,81 @@ void luaReplyToRedisReply(client *c, lua_State *lua) {
sdsmapchars(ok,"\r\n"," ",2);
addReplySds(c,sdscatprintf(sdsempty(),"+%s\r\n",ok));
sdsfree(ok);
lua_pop(lua,1);
} else {
void *replylen = addReplyDeferredLen(c);
int j = 1, mbulklen = 0;
lua_pop(lua,1); /* Discard the 'ok' field value we popped */
while(1) {
lua_pushnumber(lua,j++);
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TNIL) {
lua_pop(lua,1);
break;
}
luaReplyToRedisReply(c, lua);
mbulklen++;
}
setDeferredArrayLen(c,replylen,mbulklen);
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle double reply. */
lua_pushstring(lua,"double");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TNUMBER) {
addReplyDouble(c,lua_tonumber(lua,-1));
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle map reply. */
lua_pushstring(lua,"map");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TTABLE) {
int maplen = 0;
void *replylen = addReplyDeferredLen(c);
lua_pushnil(lua); /* Use nil to start iteration. */
while (lua_next(lua,-2)) {
/* Stack now: table, key, value */
luaReplyToRedisReply(c, lua); /* Return value. */
lua_pushvalue(lua,-1); /* Dup key before consuming. */
luaReplyToRedisReply(c, lua); /* Return key. */
/* Stack now: table, key. */
maplen++;
}
setDeferredMapLen(c,replylen,maplen);
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle set reply. */
lua_pushstring(lua,"set");
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TTABLE) {
int setlen = 0;
void *replylen = addReplyDeferredLen(c);
lua_pushnil(lua); /* Use nil to start iteration. */
while (lua_next(lua,-2)) {
/* Stack now: table, key, true */
lua_pop(lua,1); /* Discard the boolean value. */
lua_pushvalue(lua,-1); /* Dup key before consuming. */
luaReplyToRedisReply(c, lua); /* Return key. */
/* Stack now: table, key. */
setlen++;
}
setDeferredSetLen(c,replylen,setlen);
lua_pop(lua,2);
return;
}
lua_pop(lua,1); /* Discard field name pushed before. */
/* Handle the array reply. */
void *replylen = addReplyDeferredLen(c);
int j = 1, mbulklen = 0;
while(1) {
lua_pushnumber(lua,j++);
lua_gettable(lua,-2);
t = lua_type(lua,-1);
if (t == LUA_TNIL) {
lua_pop(lua,1);
break;
}
luaReplyToRedisReply(c, lua);
mbulklen++;
}
setDeferredArrayLen(c,replylen,mbulklen);
break;
default:
addReplyNull(c);
@ -462,6 +571,11 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
c->argc = argc;
c->user = server.lua_caller->user;
/* Process module hooks */
moduleCallCommandFilters(c);
argv = c->argv;
argc = c->argc;
/* Log the command if debugging is active. */
if (ldb.active && ldb.step) {
sds cmdlog = sdsnew("<redis>");
@ -854,6 +968,25 @@ int luaLogCommand(lua_State *lua) {
return 0;
}
/* redis.setresp() */
int luaSetResp(lua_State *lua) {
int argc = lua_gettop(lua);
if (argc != 1) {
lua_pushstring(lua, "redis.setresp() requires one argument.");
return lua_error(lua);
}
int resp = lua_tonumber(lua,-argc);
if (resp != 2 && resp != 3) {
lua_pushstring(lua, "RESP version must be 2 or 3.");
return lua_error(lua);
}
server.lua_client->resp = resp;
return 0;
}
/* ---------------------------------------------------------------------------
* Lua engine initialization and reset.
* ------------------------------------------------------------------------- */
@ -950,6 +1083,7 @@ void scriptingInit(int setup) {
if (setup) {
server.lua_client = NULL;
server.lua_caller = NULL;
server.lua_cur_script = NULL;
server.lua_timedout = 0;
ldbInit();
}
@ -981,6 +1115,11 @@ void scriptingInit(int setup) {
lua_pushcfunction(lua,luaLogCommand);
lua_settable(lua,-3);
/* redis.setresp */
lua_pushstring(lua,"setresp");
lua_pushcfunction(lua,luaSetResp);
lua_settable(lua,-3);
lua_pushstring(lua,"LOG_DEBUG");
lua_pushnumber(lua,LL_DEBUG);
lua_settable(lua,-3);
@ -1104,7 +1243,7 @@ void scriptingInit(int setup) {
* Note: there is no need to create it again when this function is called
* by scriptingReset(). */
if (server.lua_client == NULL) {
server.lua_client = createClient(-1);
server.lua_client = createClient(NULL);
server.lua_client->flags |= CLIENT_LUA;
}
@ -1269,7 +1408,11 @@ void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
/* Set the timeout condition if not already set and the maximum
* execution time was reached. */
if (elapsed >= server.lua_time_limit && server.lua_timedout == 0) {
serverLog(LL_WARNING,"Lua slow script detected: still in execution after %lld milliseconds. You can try killing the script using the SCRIPT KILL command.",elapsed);
serverLog(LL_WARNING,
"Lua slow script detected: still in execution after %lld milliseconds. "
"You can try killing the script using the SCRIPT KILL command. "
"Script SHA1 is: %s",
elapsed, server.lua_cur_script);
server.lua_timedout = 1;
/* Once the script timeouts we reenter the event loop to permit others
* to call SCRIPT KILL or SHUTDOWN NOSAVE if needed. For this reason
@ -1374,8 +1517,9 @@ void evalGenericCommand(client *c, int evalsha) {
luaSetGlobalArray(lua,"KEYS",c->argv+3,numkeys);
luaSetGlobalArray(lua,"ARGV",c->argv+3+numkeys,c->argc-3-numkeys);
/* Select the right DB in the context of the Lua client */
/* Set the Lua client database and protocol. */
selectDb(server.lua_client,c->db->id);
server.lua_client->resp = 2; /* Default is RESP2, scripts can change it. */
/* Set a hook in order to be able to stop the script execution if it
* is running for too much time.
@ -1385,6 +1529,7 @@ void evalGenericCommand(client *c, int evalsha) {
* If we are debugging, we set instead a "line" hook so that the
* debugger is call-back at every line executed by the script. */
server.lua_caller = c;
server.lua_cur_script = funcname + 2;
server.lua_time_start = mstime();
server.lua_kill = 0;
if (server.lua_time_limit > 0 && ldb.active == 0) {
@ -1411,6 +1556,7 @@ void evalGenericCommand(client *c, int evalsha) {
queueClientForReprocessing(server.master);
}
server.lua_caller = NULL;
server.lua_cur_script = NULL;
/* Call the Lua garbage collector from time to time to avoid a
* full cycle performed by Lua, which adds too latency.
@ -1588,7 +1734,7 @@ NULL
/* Initialize Lua debugger data structures. */
void ldbInit(void) {
ldb.fd = -1;
ldb.conn = NULL;
ldb.active = 0;
ldb.logs = listCreate();
listSetFreeMethod(ldb.logs,(void (*)(void*))sdsfree);
@ -1610,7 +1756,7 @@ void ldbFlushLog(list *log) {
void ldbEnable(client *c) {
c->flags |= CLIENT_LUA_DEBUG;
ldbFlushLog(ldb.logs);
ldb.fd = c->fd;
ldb.conn = c->conn;
ldb.step = 1;
ldb.bpcount = 0;
ldb.luabp = 0;
@ -1665,7 +1811,7 @@ void ldbSendLogs(void) {
proto = sdscatlen(proto,"\r\n",2);
listDelNode(ldb.logs,ln);
}
if (write(ldb.fd,proto,sdslen(proto)) == -1) {
if (connWrite(ldb.conn,proto,sdslen(proto)) == -1) {
/* Avoid warning. We don't check the return value of write()
* since the next read() will catch the I/O error and will
* close the debugging session. */
@ -1688,7 +1834,7 @@ void ldbSendLogs(void) {
int ldbStartSession(client *c) {
ldb.forked = (c->flags & CLIENT_LUA_DEBUG_SYNC) == 0;
if (ldb.forked) {
pid_t cp = fork();
pid_t cp = redisFork();
if (cp == -1) {
addReplyError(c,"Fork() failed: can't run EVAL in debugging mode.");
return 0;
@ -1705,7 +1851,6 @@ int ldbStartSession(client *c) {
* socket to make sure if the parent crashes a reset is sent
* to the clients. */
serverLog(LL_WARNING,"Redis forked for debugging eval");
closeListeningSockets(0);
} else {
/* Parent */
listAddNodeTail(ldb.children,(void*)(unsigned long)cp);
@ -1718,8 +1863,8 @@ int ldbStartSession(client *c) {
}
/* Setup our debugging session. */
anetBlock(NULL,ldb.fd);
anetSendTimeout(NULL,ldb.fd,5000);
connBlock(ldb.conn);
connSendTimeout(ldb.conn,5000);
ldb.active = 1;
/* First argument of EVAL is the script itself. We split it into different
@ -1746,7 +1891,7 @@ void ldbEndSession(client *c) {
/* If it's a fork()ed session, we just exit. */
if (ldb.forked) {
writeToClient(c->fd, c, 0);
writeToClient(c,0);
serverLog(LL_WARNING,"Lua debugging session child exiting");
exitFromChild(0);
} else {
@ -1755,8 +1900,8 @@ void ldbEndSession(client *c) {
}
/* Otherwise let's restore client's state. */
anetNonBlock(NULL,ldb.fd);
anetSendTimeout(NULL,ldb.fd,0);
connNonBlock(ldb.conn);
connSendTimeout(ldb.conn,0);
/* Close the client connectin after sending the final EVAL reply
* in order to signal the end of the debugging session. */
@ -2048,6 +2193,11 @@ char *ldbRedisProtocolToHuman_Int(sds *o, char *reply);
char *ldbRedisProtocolToHuman_Bulk(sds *o, char *reply);
char *ldbRedisProtocolToHuman_Status(sds *o, char *reply);
char *ldbRedisProtocolToHuman_MultiBulk(sds *o, char *reply);
char *ldbRedisProtocolToHuman_Set(sds *o, char *reply);
char *ldbRedisProtocolToHuman_Map(sds *o, char *reply);
char *ldbRedisProtocolToHuman_Null(sds *o, char *reply);
char *ldbRedisProtocolToHuman_Bool(sds *o, char *reply);
char *ldbRedisProtocolToHuman_Double(sds *o, char *reply);
/* Get Redis protocol from 'reply' and appends it in human readable form to
* the passed SDS string 'o'.
@ -2062,6 +2212,11 @@ char *ldbRedisProtocolToHuman(sds *o, char *reply) {
case '+': p = ldbRedisProtocolToHuman_Status(o,reply); break;
case '-': p = ldbRedisProtocolToHuman_Status(o,reply); break;
case '*': p = ldbRedisProtocolToHuman_MultiBulk(o,reply); break;
case '~': p = ldbRedisProtocolToHuman_Set(o,reply); break;
case '%': p = ldbRedisProtocolToHuman_Map(o,reply); break;
case '_': p = ldbRedisProtocolToHuman_Null(o,reply); break;
case '#': p = ldbRedisProtocolToHuman_Bool(o,reply); break;
case ',': p = ldbRedisProtocolToHuman_Double(o,reply); break;
}
return p;
}
@ -2116,6 +2271,62 @@ char *ldbRedisProtocolToHuman_MultiBulk(sds *o, char *reply) {
return p;
}
char *ldbRedisProtocolToHuman_Set(sds *o, char *reply) {
char *p = strchr(reply+1,'\r');
long long mbulklen;
int j = 0;
string2ll(reply+1,p-reply-1,&mbulklen);
p += 2;
*o = sdscatlen(*o,"~(",2);
for (j = 0; j < mbulklen; j++) {
p = ldbRedisProtocolToHuman(o,p);
if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
}
*o = sdscatlen(*o,")",1);
return p;
}
char *ldbRedisProtocolToHuman_Map(sds *o, char *reply) {
char *p = strchr(reply+1,'\r');
long long mbulklen;
int j = 0;
string2ll(reply+1,p-reply-1,&mbulklen);
p += 2;
*o = sdscatlen(*o,"{",1);
for (j = 0; j < mbulklen; j++) {
p = ldbRedisProtocolToHuman(o,p);
*o = sdscatlen(*o," => ",4);
p = ldbRedisProtocolToHuman(o,p);
if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
}
*o = sdscatlen(*o,"}",1);
return p;
}
char *ldbRedisProtocolToHuman_Null(sds *o, char *reply) {
char *p = strchr(reply+1,'\r');
*o = sdscatlen(*o,"(null)",6);
return p+2;
}
char *ldbRedisProtocolToHuman_Bool(sds *o, char *reply) {
char *p = strchr(reply+1,'\r');
if (reply[1] == 't')
*o = sdscatlen(*o,"#true",5);
else
*o = sdscatlen(*o,"#false",6);
return p+2;
}
char *ldbRedisProtocolToHuman_Double(sds *o, char *reply) {
char *p = strchr(reply+1,'\r');
*o = sdscatlen(*o,"(double) ",9);
*o = sdscatlen(*o,reply+1,p-reply-1);
return p+2;
}
/* Log a Redis reply as debugger output, in an human readable format.
* If the resulting string is longer than 'len' plus a few more chars
* used as prefix, it gets truncated. */
@ -2327,7 +2538,7 @@ int ldbRepl(lua_State *lua) {
while(1) {
while((argv = ldbReplParseCommand(&argc)) == NULL) {
char buf[1024];
int nread = read(ldb.fd,buf,sizeof(buf));
int nread = connRead(ldb.conn,buf,sizeof(buf));
if (nread <= 0) {
/* Make sure the script runs without user input since the
* client is no longer connected. */

4
src/sds.c
View File

@ -603,6 +603,10 @@ sds sdscatfmt(sds s, char const *fmt, ...) {
long i;
va_list ap;
/* To avoid continuous reallocations, let's start with a buffer that
* can hold at least two times the format string itself. It's not the
* best heuristic but seems to work in practice. */
s = sdsMakeRoomFor(s, initlen + strlen(fmt)*2);
va_start(ap,fmt);
f = fmt; /* Next format specifier byte to process. */
i = initlen; /* Position of the next byte to write to dest str. */

37
src/sentinel.c
View File

@ -30,6 +30,10 @@
#include "server.h"
#include "hiredis.h"
#ifdef USE_OPENSSL
#include "openssl/ssl.h"
#include "hiredis_ssl.h"
#endif
#include "async.h"
#include <ctype.h>
@ -40,6 +44,10 @@
extern char **environ;
#ifdef USE_OPENSSL
extern SSL_CTX *redis_tls_ctx;
#endif
#define REDIS_SENTINEL_PORT 26379
/* ======================== Sentinel global state =========================== */
@ -1995,6 +2003,19 @@ void sentinelSetClientName(sentinelRedisInstance *ri, redisAsyncContext *c, char
}
}
static int instanceLinkNegotiateTLS(redisAsyncContext *context) {
#ifndef USE_OPENSSL
(void) context;
#else
if (!redis_tls_ctx) return C_ERR;
SSL *ssl = SSL_new(redis_tls_ctx);
if (!ssl) return C_ERR;
if (redisInitiateSSL(&context->c, ssl) == REDIS_ERR) return C_ERR;
#endif
return C_OK;
}
/* Create the async connections for the instance link if the link
* is disconnected. Note that link->disconnected is true even if just
* one of the two links (commands and pub/sub) is missing. */
@ -2010,7 +2031,11 @@ void sentinelReconnectInstance(sentinelRedisInstance *ri) {
/* Commands connection. */
if (link->cc == NULL) {
link->cc = redisAsyncConnectBind(ri->addr->ip,ri->addr->port,NET_FIRST_BIND_ADDR);
if (link->cc->err) {
if (!link->cc->err && server.tls_replication &&
(instanceLinkNegotiateTLS(link->cc) == C_ERR)) {
sentinelEvent(LL_DEBUG,"-cmd-link-reconnection",ri,"%@ #Failed to initialize TLS");
instanceLinkCloseConnection(link,link->cc);
} else if (link->cc->err) {
sentinelEvent(LL_DEBUG,"-cmd-link-reconnection",ri,"%@ #%s",
link->cc->errstr);
instanceLinkCloseConnection(link,link->cc);
@ -2033,7 +2058,10 @@ void sentinelReconnectInstance(sentinelRedisInstance *ri) {
/* Pub / Sub */
if ((ri->flags & (SRI_MASTER|SRI_SLAVE)) && link->pc == NULL) {
link->pc = redisAsyncConnectBind(ri->addr->ip,ri->addr->port,NET_FIRST_BIND_ADDR);
if (link->pc->err) {
if (!link->pc->err && server.tls_replication &&
(instanceLinkNegotiateTLS(link->pc) == C_ERR)) {
sentinelEvent(LL_DEBUG,"-pubsub-link-reconnection",ri,"%@ #Failed to initialize TLS");
} else if (link->pc->err) {
sentinelEvent(LL_DEBUG,"-pubsub-link-reconnection",ri,"%@ #%s",
link->pc->errstr);
instanceLinkCloseConnection(link,link->pc);
@ -2584,8 +2612,9 @@ int sentinelSendHello(sentinelRedisInstance *ri) {
return C_ERR;
announce_ip = ip;
}
announce_port = sentinel.announce_port ?
sentinel.announce_port : server.port;
if (sentinel.announce_port) announce_port = sentinel.announce_port;
else if (server.tls_replication && server.tls_port) announce_port = server.tls_port;
else announce_port = server.port;
/* Format and send the Hello message. */
snprintf(payload,sizeof(payload),

457
src/server.c
View File

@ -146,6 +146,8 @@ volatile unsigned long lru_clock; /* Server global current LRU time. */
* in this condition but just a few.
*
* no-monitor: Do not automatically propagate the command on MONITOR.
*
* no-slowlog: Do not automatically propagate the command to the slowlog.
*
* cluster-asking: Perform an implicit ASKING for this command, so the
* command will be accepted in cluster mode if the slot is marked
@ -627,7 +629,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"auth",authCommand,-2,
"no-script ok-loading ok-stale fast @connection",
"no-script ok-loading ok-stale fast no-monitor no-slowlog @connection",
0,NULL,0,0,0,0,0,0},
/* We don't allow PING during loading since in Redis PING is used as
@ -670,7 +672,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"exec",execCommand,1,
"no-script no-monitor @transaction",
"no-script no-monitor no-slowlog @transaction",
0,NULL,0,0,0,0,0,0},
{"discard",discardCommand,1,
@ -715,7 +717,7 @@ struct redisCommand redisCommandTable[] = {
{"touch",touchCommand,-2,
"read-only fast @keyspace",
0,NULL,1,1,1,0,0,0},
0,NULL,1,-1,1,0,0,0},
{"pttl",pttlCommand,2,
"read-only fast random @keyspace",
@ -822,7 +824,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"hello",helloCommand,-2,
"no-script fast @connection",
"no-script fast no-monitor no-slowlog @connection",
0,NULL,0,0,0,0,0,0},
/* EVAL can modify the dataset, however it is not flagged as a write
@ -863,7 +865,7 @@ struct redisCommand redisCommandTable[] = {
"no-script @keyspace",
0,NULL,0,0,0,0,0,0},
{"command",commandCommand,0,
{"command",commandCommand,-1,
"ok-loading ok-stale random @connection",
0,NULL,0,0,0,0,0,0},
@ -1447,12 +1449,18 @@ int incrementallyRehash(int dbid) {
* for dict.c to resize the hash tables accordingly to the fact we have o not
* running childs. */
void updateDictResizePolicy(void) {
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1)
if (!hasActiveChildProcess())
dictEnableResize();
else
dictDisableResize();
}
int hasActiveChildProcess() {
return server.rdb_child_pid != -1 ||
server.aof_child_pid != -1 ||
server.module_child_pid != -1;
}
/* ======================= Cron: called every 100 ms ======================== */
/* Add a sample to the operations per second array of samples. */
@ -1674,10 +1682,12 @@ void clientsCron(void) {
void databasesCron(void) {
/* Expire keys by random sampling. Not required for slaves
* as master will synthesize DELs for us. */
if (server.active_expire_enabled && server.masterhost == NULL) {
activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);
} else if (server.masterhost != NULL) {
expireSlaveKeys();
if (server.active_expire_enabled) {
if (server.masterhost == NULL) {
activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);
} else {
expireSlaveKeys();
}
}
/* Defrag keys gradually. */
@ -1687,7 +1697,7 @@ void databasesCron(void) {
/* Perform hash tables rehashing if needed, but only if there are no
* other processes saving the DB on disk. Otherwise rehashing is bad
* as will cause a lot of copy-on-write of memory pages. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1) {
if (!hasActiveChildProcess()) {
/* We use global counters so if we stop the computation at a given
* DB we'll be able to start from the successive in the next
* cron loop iteration. */
@ -1728,19 +1738,76 @@ void databasesCron(void) {
* every object access, and accuracy is not needed. To access a global var is
* a lot faster than calling time(NULL) */
void updateCachedTime(void) {
time_t unixtime = time(NULL);
atomicSet(server.unixtime,unixtime);
server.unixtime = time(NULL);
server.mstime = mstime();
/* To get information about daylight saving time, we need to call localtime_r
* and cache the result. However calling localtime_r in this context is safe
* since we will never fork() while here, in the main thread. The logging
* function will call a thread safe version of localtime that has no locks. */
/* To get information about daylight saving time, we need to call
* localtime_r and cache the result. However calling localtime_r in this
* context is safe since we will never fork() while here, in the main
* thread. The logging function will call a thread safe version of
* localtime that has no locks. */
struct tm tm;
localtime_r(&server.unixtime,&tm);
time_t ut = server.unixtime;
localtime_r(&ut,&tm);
server.daylight_active = tm.tm_isdst;
}
void checkChildrenDone(void) {
int statloc;
pid_t pid;
/* If we have a diskless rdb child (note that we support only one concurrent
* child), we want to avoid collecting it's exit status and acting on it
* as long as we didn't finish to drain the pipe, since then we're at risk
* of starting a new fork and a new pipe before we're done with the previous
* one. */
if (server.rdb_child_pid != -1 && server.rdb_pipe_conns)
return;
if ((pid = wait3(&statloc,WNOHANG,NULL)) != 0) {
int exitcode = WEXITSTATUS(statloc);
int bysignal = 0;
if (WIFSIGNALED(statloc)) bysignal = WTERMSIG(statloc);
/* sigKillChildHandler catches the signal and calls exit(), but we
* must make sure not to flag lastbgsave_status, etc incorrectly.
* We could directly terminate the child process via SIGUSR1
* without handling it, but in this case Valgrind will log an
* annoying error. */
if (exitcode == SERVER_CHILD_NOERROR_RETVAL) {
bysignal = SIGUSR1;
exitcode = 1;
}
if (pid == -1) {
serverLog(LL_WARNING,"wait3() returned an error: %s. "
"rdb_child_pid = %d, aof_child_pid = %d, module_child_pid = %d",
strerror(errno),
(int) server.rdb_child_pid,
(int) server.aof_child_pid,
(int) server.module_child_pid);
} else if (pid == server.rdb_child_pid) {
backgroundSaveDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else if (pid == server.aof_child_pid) {
backgroundRewriteDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else if (pid == server.module_child_pid) {
ModuleForkDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else {
if (!ldbRemoveChild(pid)) {
serverLog(LL_WARNING,
"Warning, detected child with unmatched pid: %ld",
(long)pid);
}
}
updateDictResizePolicy();
closeChildInfoPipe();
}
}
/* This is our timer interrupt, called server.hz times per second.
* Here is where we do a number of things that need to be done asynchronously.
* For instance:
@ -1807,8 +1874,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
*
* Note that you can change the resolution altering the
* LRU_CLOCK_RESOLUTION define. */
unsigned long lruclock = getLRUClock();
atomicSet(server.lruclock,lruclock);
server.lruclock = getLRUClock();
/* Record the max memory used since the server was started. */
if (zmalloc_used_memory() > server.stat_peak_memory)
@ -1884,47 +1950,16 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
/* Start a scheduled AOF rewrite if this was requested by the user while
* a BGSAVE was in progress. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1 &&
if (!hasActiveChildProcess() &&
server.aof_rewrite_scheduled)
{
rewriteAppendOnlyFileBackground();
}
/* Check if a background saving or AOF rewrite in progress terminated. */
if (server.rdb_child_pid != -1 || server.aof_child_pid != -1 ||
ldbPendingChildren())
if (hasActiveChildProcess() || ldbPendingChildren())
{
int statloc;
pid_t pid;
if ((pid = wait3(&statloc,WNOHANG,NULL)) != 0) {
int exitcode = WEXITSTATUS(statloc);
int bysignal = 0;
if (WIFSIGNALED(statloc)) bysignal = WTERMSIG(statloc);
if (pid == -1) {
serverLog(LL_WARNING,"wait3() returned an error: %s. "
"rdb_child_pid = %d, aof_child_pid = %d",
strerror(errno),
(int) server.rdb_child_pid,
(int) server.aof_child_pid);
} else if (pid == server.rdb_child_pid) {
backgroundSaveDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else if (pid == server.aof_child_pid) {
backgroundRewriteDoneHandler(exitcode,bysignal);
if (!bysignal && exitcode == 0) receiveChildInfo();
} else {
if (!ldbRemoveChild(pid)) {
serverLog(LL_WARNING,
"Warning, detected child with unmatched pid: %ld",
(long)pid);
}
}
updateDictResizePolicy();
closeChildInfoPipe();
}
checkChildrenDone();
} else {
/* If there is not a background saving/rewrite in progress check if
* we have to save/rewrite now. */
@ -1952,8 +1987,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
/* Trigger an AOF rewrite if needed. */
if (server.aof_state == AOF_ON &&
server.rdb_child_pid == -1 &&
server.aof_child_pid == -1 &&
!hasActiveChildProcess() &&
server.aof_rewrite_perc &&
server.aof_current_size > server.aof_rewrite_min_size)
{
@ -1981,9 +2015,6 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
flushAppendOnlyFile(0);
}
/* Close clients that need to be closed asynchronous */
freeClientsInAsyncFreeQueue();
/* Clear the paused clients flag if needed. */
clientsArePaused(); /* Don't check return value, just use the side effect.*/
@ -2004,6 +2035,9 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
migrateCloseTimedoutSockets();
}
/* Stop the I/O threads if we don't have enough pending work. */
stopThreadedIOIfNeeded();
/* Start a scheduled BGSAVE if the corresponding flag is set. This is
* useful when we are forced to postpone a BGSAVE because an AOF
* rewrite is in progress.
@ -2011,7 +2045,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
* Note: this code must be after the replicationCron() call above so
* make sure when refactoring this file to keep this order. This is useful
* because we want to give priority to RDB savings for replication. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1 &&
if (!hasActiveChildProcess() &&
server.rdb_bgsave_scheduled &&
(server.unixtime-server.lastbgsave_try > CONFIG_BGSAVE_RETRY_DELAY ||
server.lastbgsave_status == C_OK))
@ -2032,6 +2066,11 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
void beforeSleep(struct aeEventLoop *eventLoop) {
UNUSED(eventLoop);
/* Handle TLS pending data. (must be done before flushAppendOnlyFile) */
tlsProcessPendingData();
/* If tls still has pending unread data don't sleep at all. */
aeSetDontWait(server.el, tlsHasPendingData());
/* Call the Redis Cluster before sleep function. Note that this function
* may change the state of Redis Cluster (from ok to fail or vice versa),
* so it's a good idea to call it before serving the unblocked clients
@ -2075,7 +2114,10 @@ void beforeSleep(struct aeEventLoop *eventLoop) {
flushAppendOnlyFile(0);
/* Handle writes with pending output buffers. */
handleClientsWithPendingWrites();
handleClientsWithPendingWritesUsingThreads();
/* Close clients that need to be closed asynchronous */
freeClientsInAsyncFreeQueue();
/* Before we are going to sleep, let the threads access the dataset by
* releasing the GIL. Redis main thread will not touch anything at this
@ -2089,6 +2131,7 @@ void beforeSleep(struct aeEventLoop *eventLoop) {
void afterSleep(struct aeEventLoop *eventLoop) {
UNUSED(eventLoop);
if (moduleCount()) moduleAcquireGIL();
handleClientsWithPendingReadsUsingThreads();
}
/* =========================== Server initialization ======================== */
@ -2153,6 +2196,16 @@ void createSharedObjects(void) {
shared.nullarray[2] = createObject(OBJ_STRING,sdsnew("*-1\r\n"));
shared.nullarray[3] = createObject(OBJ_STRING,sdsnew("_\r\n"));
shared.emptymap[0] = NULL;
shared.emptymap[1] = NULL;
shared.emptymap[2] = createObject(OBJ_STRING,sdsnew("*0\r\n"));
shared.emptymap[3] = createObject(OBJ_STRING,sdsnew("%0\r\n"));
shared.emptyset[0] = NULL;
shared.emptyset[1] = NULL;
shared.emptyset[2] = createObject(OBJ_STRING,sdsnew("*0\r\n"));
shared.emptyset[3] = createObject(OBJ_STRING,sdsnew("~0\r\n"));
for (j = 0; j < PROTO_SHARED_SELECT_CMDS; j++) {
char dictid_str[64];
int dictid_len;
@ -2199,10 +2252,6 @@ void createSharedObjects(void) {
void initServerConfig(void) {
int j;
pthread_mutex_init(&server.next_client_id_mutex,NULL);
pthread_mutex_init(&server.lruclock_mutex,NULL);
pthread_mutex_init(&server.unixtime_mutex,NULL);
updateCachedTime();
getRandomHexChars(server.runid,CONFIG_RUN_ID_SIZE);
server.runid[CONFIG_RUN_ID_SIZE] = '\0';
@ -2215,11 +2264,13 @@ void initServerConfig(void) {
server.dynamic_hz = CONFIG_DEFAULT_DYNAMIC_HZ;
server.arch_bits = (sizeof(long) == 8) ? 64 : 32;
server.port = CONFIG_DEFAULT_SERVER_PORT;
server.tls_port = CONFIG_DEFAULT_SERVER_TLS_PORT;
server.tcp_backlog = CONFIG_DEFAULT_TCP_BACKLOG;
server.bindaddr_count = 0;
server.unixsocket = NULL;
server.unixsocketperm = CONFIG_DEFAULT_UNIX_SOCKET_PERM;
server.ipfd_count = 0;
server.tlsfd_count = 0;
server.sofd = -1;
server.protected_mode = CONFIG_DEFAULT_PROTECTED_MODE;
server.gopher_enabled = CONFIG_DEFAULT_GOPHER_ENABLED;
@ -2228,6 +2279,7 @@ void initServerConfig(void) {
server.maxidletime = CONFIG_DEFAULT_CLIENT_TIMEOUT;
server.tcpkeepalive = CONFIG_DEFAULT_TCP_KEEPALIVE;
server.active_expire_enabled = 1;
server.jemalloc_bg_thread = 1;
server.active_defrag_enabled = CONFIG_DEFAULT_ACTIVE_DEFRAG;
server.active_defrag_ignore_bytes = CONFIG_DEFAULT_DEFRAG_IGNORE_BYTES;
server.active_defrag_threshold_lower = CONFIG_DEFAULT_DEFRAG_THRESHOLD_LOWER;
@ -2253,6 +2305,7 @@ void initServerConfig(void) {
server.aof_rewrite_min_size = AOF_REWRITE_MIN_SIZE;
server.aof_rewrite_base_size = 0;
server.aof_rewrite_scheduled = 0;
server.aof_flush_sleep = 0;
server.aof_last_fsync = time(NULL);
server.aof_rewrite_time_last = -1;
server.aof_rewrite_time_start = -1;
@ -2263,6 +2316,8 @@ void initServerConfig(void) {
server.aof_flush_postponed_start = 0;
server.aof_rewrite_incremental_fsync = CONFIG_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC;
server.rdb_save_incremental_fsync = CONFIG_DEFAULT_RDB_SAVE_INCREMENTAL_FSYNC;
server.rdb_key_save_delay = CONFIG_DEFAULT_RDB_KEY_SAVE_DELAY;
server.key_load_delay = CONFIG_DEFAULT_KEY_LOAD_DELAY;
server.aof_load_truncated = CONFIG_DEFAULT_AOF_LOAD_TRUNCATED;
server.aof_use_rdb_preamble = CONFIG_DEFAULT_AOF_USE_RDB_PREAMBLE;
server.pidfile = NULL;
@ -2314,9 +2369,10 @@ void initServerConfig(void) {
server.lazyfree_lazy_server_del = CONFIG_DEFAULT_LAZYFREE_LAZY_SERVER_DEL;
server.always_show_logo = CONFIG_DEFAULT_ALWAYS_SHOW_LOGO;
server.lua_time_limit = LUA_SCRIPT_TIME_LIMIT;
server.io_threads_num = CONFIG_DEFAULT_IO_THREADS_NUM;
server.io_threads_do_reads = CONFIG_DEFAULT_IO_THREADS_DO_READS;
unsigned int lruclock = getLRUClock();
atomicSet(server.lruclock,lruclock);
server.lruclock = getLRUClock();
resetServerSaveParams();
appendServerSaveParams(60*60,1); /* save after 1 hour and 1 change */
@ -2331,6 +2387,9 @@ void initServerConfig(void) {
server.cached_master = NULL;
server.master_initial_offset = -1;
server.repl_state = REPL_STATE_NONE;
server.repl_transfer_tmpfile = NULL;
server.repl_transfer_fd = -1;
server.repl_transfer_s = NULL;
server.repl_syncio_timeout = CONFIG_REPL_SYNCIO_TIMEOUT;
server.repl_serve_stale_data = CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA;
server.repl_slave_ro = CONFIG_DEFAULT_SLAVE_READ_ONLY;
@ -2339,6 +2398,7 @@ void initServerConfig(void) {
server.repl_down_since = 0; /* Never connected, repl is down since EVER. */
server.repl_disable_tcp_nodelay = CONFIG_DEFAULT_REPL_DISABLE_TCP_NODELAY;
server.repl_diskless_sync = CONFIG_DEFAULT_REPL_DISKLESS_SYNC;
server.repl_diskless_load = CONFIG_DEFAULT_REPL_DISKLESS_LOAD;
server.repl_diskless_sync_delay = CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY;
server.repl_ping_slave_period = CONFIG_DEFAULT_REPL_PING_SLAVE_PERIOD;
server.repl_timeout = CONFIG_DEFAULT_REPL_TIMEOUT;
@ -2395,6 +2455,9 @@ void initServerConfig(void) {
/* Latency monitor */
server.latency_monitor_threshold = CONFIG_DEFAULT_LATENCY_MONITOR_THRESHOLD;
/* Tracking. */
server.tracking_table_max_fill = CONFIG_DEFAULT_TRACKING_TABLE_MAX_FILL;
/* Debugging */
server.assert_failed = "<no assertion failed>";
server.assert_file = "<no file>";
@ -2714,6 +2777,7 @@ void initServer(void) {
server.slaves = listCreate();
server.monitors = listCreate();
server.clients_pending_write = listCreate();
server.clients_pending_read = listCreate();
server.slaveseldb = -1; /* Force to emit the first SELECT command. */
server.unblocked_clients = listCreate();
server.ready_keys = listCreate();
@ -2722,6 +2786,11 @@ void initServer(void) {
server.clients_paused = 0;
server.system_memory_size = zmalloc_get_memory_size();
if (server.tls_port && tlsConfigure(&server.tls_ctx_config) == C_ERR) {
serverLog(LL_WARNING, "Failed to configure TLS. Check logs for more info.");
exit(1);
}
createSharedObjects();
adjustOpenFilesLimit();
server.el = aeCreateEventLoop(server.maxclients+CONFIG_FDSET_INCR);
@ -2737,6 +2806,9 @@ void initServer(void) {
if (server.port != 0 &&
listenToPort(server.port,server.ipfd,&server.ipfd_count) == C_ERR)
exit(1);
if (server.tls_port != 0 &&
listenToPort(server.tls_port,server.tlsfd,&server.tlsfd_count) == C_ERR)
exit(1);
/* Open the listening Unix domain socket. */
if (server.unixsocket != NULL) {
@ -2751,7 +2823,7 @@ void initServer(void) {
}
/* Abort if there are no listening sockets at all. */
if (server.ipfd_count == 0 && server.sofd < 0) {
if (server.ipfd_count == 0 && server.tlsfd_count == 0 && server.sofd < 0) {
serverLog(LL_WARNING, "Configured to not listen anywhere, exiting.");
exit(1);
}
@ -2775,7 +2847,13 @@ void initServer(void) {
server.cronloops = 0;
server.rdb_child_pid = -1;
server.aof_child_pid = -1;
server.module_child_pid = -1;
server.rdb_child_type = RDB_CHILD_TYPE_NONE;
server.rdb_pipe_conns = NULL;
server.rdb_pipe_numconns = 0;
server.rdb_pipe_numconns_writing = 0;
server.rdb_pipe_buff = NULL;
server.rdb_pipe_bufflen = 0;
server.rdb_bgsave_scheduled = 0;
server.child_info_pipe[0] = -1;
server.child_info_pipe[1] = -1;
@ -2793,6 +2871,7 @@ void initServer(void) {
server.stat_peak_memory = 0;
server.stat_rdb_cow_bytes = 0;
server.stat_aof_cow_bytes = 0;
server.stat_module_cow_bytes = 0;
server.cron_malloc_stats.zmalloc_used = 0;
server.cron_malloc_stats.process_rss = 0;
server.cron_malloc_stats.allocator_allocated = 0;
@ -2821,6 +2900,14 @@ void initServer(void) {
"Unrecoverable error creating server.ipfd file event.");
}
}
for (j = 0; j < server.tlsfd_count; j++) {
if (aeCreateFileEvent(server.el, server.tlsfd[j], AE_READABLE,
acceptTLSHandler,NULL) == AE_ERR)
{
serverPanic(
"Unrecoverable error creating server.tlsfd file event.");
}
}
if (server.sofd > 0 && aeCreateFileEvent(server.el,server.sofd,AE_READABLE,
acceptUnixHandler,NULL) == AE_ERR) serverPanic("Unrecoverable error creating server.sofd file event.");
@ -2860,7 +2947,17 @@ void initServer(void) {
scriptingInit(1);
slowlogInit();
latencyMonitorInit();
}
/* Some steps in server initialization need to be done last (after modules
* are loaded).
* Specifically, creation of threads due to a race bug in ld.so, in which
* Thread Local Storage initialization collides with dlopen call.
* see: https://sourceware.org/bugzilla/show_bug.cgi?id=19329 */
void InitServerLast() {
bioInit();
initThreadedIO();
set_jemalloc_bg_thread(server.jemalloc_bg_thread);
server.initial_memory_usage = zmalloc_used_memory();
}
@ -2899,6 +2996,8 @@ int populateCommandTableParseFlags(struct redisCommand *c, char *strflags) {
c->flags |= CMD_STALE;
} else if (!strcasecmp(flag,"no-monitor")) {
c->flags |= CMD_SKIP_MONITOR;
} else if (!strcasecmp(flag,"no-slowlog")) {
c->flags |= CMD_SKIP_SLOWLOG;
} else if (!strcasecmp(flag,"cluster-asking")) {
c->flags |= CMD_ASKING;
} else if (!strcasecmp(flag,"fast")) {
@ -3183,12 +3282,13 @@ void call(client *c, int flags) {
/* Log the command into the Slow log if needed, and populate the
* per-command statistics that we show in INFO commandstats. */
if (flags & CMD_CALL_SLOWLOG && c->cmd->proc != execCommand) {
if (flags & CMD_CALL_SLOWLOG && !(c->cmd->flags & CMD_SKIP_SLOWLOG)) {
char *latency_event = (c->cmd->flags & CMD_FAST) ?
"fast-command" : "command";
latencyAddSampleIfNeeded(latency_event,duration/1000);
slowlogPushEntryIfNeeded(c,c->argv,c->argc,duration);
}
if (flags & CMD_CALL_STATS) {
/* use the real command that was executed (cmd and lastamc) may be
* different, in case of MULTI-EXEC or re-written commands such as
@ -3256,6 +3356,16 @@ void call(client *c, int flags) {
redisOpArrayFree(&server.also_propagate);
}
server.also_propagate = prev_also_propagate;
/* If the client has keys tracking enabled for client side caching,
* make sure to remember the keys it fetched via this command. */
if (c->cmd->flags & CMD_READONLY) {
client *caller = (c->flags & CLIENT_LUA && server.lua_caller) ?
server.lua_caller : c;
if (caller->flags & CLIENT_TRACKING)
trackingRememberKeys(caller);
}
server.stat_numcommands++;
}
@ -3268,6 +3378,8 @@ void call(client *c, int flags) {
* other operations can be performed by the caller. Otherwise
* if C_ERR is returned the client was destroyed (i.e. after QUIT). */
int processCommand(client *c) {
moduleCallCommandFilters(c);
/* The QUIT command is handled separately. Normal command procs will
* go through checking for replication and QUIT will cause trouble
* when FORCE_REPLICATION is enabled and would be implemented in
@ -3301,11 +3413,12 @@ int processCommand(client *c) {
/* Check if the user is authenticated. This check is skipped in case
* the default user is flagged as "nopass" and is active. */
int auth_required = !(DefaultUser->flags & USER_FLAG_NOPASS) &&
int auth_required = (!(DefaultUser->flags & USER_FLAG_NOPASS) ||
DefaultUser->flags & USER_FLAG_DISABLED) &&
!c->authenticated;
if (auth_required || DefaultUser->flags & USER_FLAG_DISABLED) {
if (auth_required) {
/* AUTH and HELLO are valid even in non authenticated state. */
if (c->cmd->proc != authCommand || c->cmd->proc == helloCommand) {
if (c->cmd->proc != authCommand && c->cmd->proc != helloCommand) {
flagTransaction(c);
addReply(c,shared.noautherr);
return C_OK;
@ -3320,7 +3433,7 @@ int processCommand(client *c) {
if (acl_retval == ACL_DENIED_CMD)
addReplyErrorFormat(c,
"-NOPERM this user has no permissions to run "
"the '%s' command or its subcommnad", c->cmd->name);
"the '%s' command or its subcommand", c->cmd->name);
else
addReplyErrorFormat(c,
"-NOPERM this user has no permissions to access "
@ -3371,13 +3484,20 @@ int processCommand(client *c) {
* is in MULTI/EXEC context? Error. */
if (out_of_memory &&
(c->cmd->flags & CMD_DENYOOM ||
(c->flags & CLIENT_MULTI && c->cmd->proc != execCommand))) {
(c->flags & CLIENT_MULTI &&
c->cmd->proc != execCommand &&
c->cmd->proc != discardCommand)))
{
flagTransaction(c);
addReply(c, shared.oomerr);
return C_OK;
}
}
/* Make sure to use a reasonable amount of memory for client side
* caching metadata. */
if (server.tracking_clients) trackingLimitUsedSlots();
/* Don't accept write commands if there are problems persisting on disk
* and if this is a master instance. */
int deny_write_type = writeCommandsDeniedByDiskError();
@ -3492,6 +3612,7 @@ void closeListeningSockets(int unlink_unix_socket) {
int j;
for (j = 0; j < server.ipfd_count; j++) close(server.ipfd[j]);
for (j = 0; j < server.tlsfd_count; j++) close(server.tlsfd[j]);
if (server.sofd != -1) close(server.sofd);
if (server.cluster_enabled)
for (j = 0; j < server.cfd_count; j++) close(server.cfd[j]);
@ -3518,6 +3639,12 @@ int prepareForShutdown(int flags) {
killRDBChild();
}
/* Kill module child if there is one. */
if (server.module_child_pid != -1) {
serverLog(LL_WARNING,"There is a module fork child. Killing it!");
TerminateModuleForkChild(server.module_child_pid,0);
}
if (server.aof_state != AOF_OFF) {
/* Kill the AOF saving child as the AOF we already have may be longer
* but contains the full dataset anyway. */
@ -3672,6 +3799,7 @@ void addReplyCommand(client *c, struct redisCommand *cmd) {
flagcount += addReplyCommandFlag(c,cmd,CMD_LOADING, "loading");
flagcount += addReplyCommandFlag(c,cmd,CMD_STALE, "stale");
flagcount += addReplyCommandFlag(c,cmd,CMD_SKIP_MONITOR, "skip_monitor");
flagcount += addReplyCommandFlag(c,cmd,CMD_SKIP_SLOWLOG, "skip_slowlog");
flagcount += addReplyCommandFlag(c,cmd,CMD_ASKING, "asking");
flagcount += addReplyCommandFlag(c,cmd,CMD_FAST, "fast");
if ((cmd->getkeys_proc && !(cmd->flags & CMD_MODULE)) ||
@ -3786,12 +3914,15 @@ sds genRedisInfoString(char *section) {
time_t uptime = server.unixtime-server.stat_starttime;
int j;
struct rusage self_ru, c_ru;
int allsections = 0, defsections = 0;
int allsections = 0, defsections = 0, everything = 0, modules = 0;
int sections = 0;
if (section == NULL) section = "default";
allsections = strcasecmp(section,"all") == 0;
defsections = strcasecmp(section,"default") == 0;
everything = strcasecmp(section,"everything") == 0;
modules = strcasecmp(section,"modules") == 0;
if (everything) allsections = 1;
getrusage(RUSAGE_SELF, &self_ru);
getrusage(RUSAGE_CHILDREN, &c_ru);
@ -3814,34 +3945,32 @@ sds genRedisInfoString(char *section) {
call_uname = 0;
}
unsigned int lruclock;
atomicGet(server.lruclock,lruclock);
info = sdscatprintf(info,
info = sdscatfmt(info,
"# Server\r\n"
"redis_version:%s\r\n"
"redis_git_sha1:%s\r\n"
"redis_git_dirty:%d\r\n"
"redis_build_id:%llx\r\n"
"redis_git_dirty:%i\r\n"
"redis_build_id:%s\r\n"
"redis_mode:%s\r\n"
"os:%s %s %s\r\n"
"arch_bits:%d\r\n"
"arch_bits:%i\r\n"
"multiplexing_api:%s\r\n"
"atomicvar_api:%s\r\n"
"gcc_version:%d.%d.%d\r\n"
"process_id:%ld\r\n"
"gcc_version:%i.%i.%i\r\n"
"process_id:%I\r\n"
"run_id:%s\r\n"
"tcp_port:%d\r\n"
"uptime_in_seconds:%jd\r\n"
"uptime_in_days:%jd\r\n"
"hz:%d\r\n"
"configured_hz:%d\r\n"
"lru_clock:%ld\r\n"
"tcp_port:%i\r\n"
"uptime_in_seconds:%I\r\n"
"uptime_in_days:%I\r\n"
"hz:%i\r\n"
"configured_hz:%i\r\n"
"lru_clock:%u\r\n"
"executable:%s\r\n"
"config_file:%s\r\n",
REDIS_VERSION,
redisGitSHA1(),
strtol(redisGitDirty(),NULL,10) > 0,
(unsigned long long) redisBuildId(),
redisBuildIdString(),
mode,
name.sysname, name.release, name.machine,
server.arch_bits,
@ -3852,14 +3981,14 @@ sds genRedisInfoString(char *section) {
#else
0,0,0,
#endif
(long) getpid(),
(int64_t) getpid(),
server.runid,
server.port,
(intmax_t)uptime,
(intmax_t)(uptime/(3600*24)),
server.port ? server.port : server.tls_port,
(int64_t)uptime,
(int64_t)(uptime/(3600*24)),
server.hz,
server.config_hz,
(unsigned long) lruclock,
server.lruclock,
server.executable ? server.executable : "",
server.configfile ? server.configfile : "");
}
@ -3874,10 +4003,12 @@ sds genRedisInfoString(char *section) {
"connected_clients:%lu\r\n"
"client_recent_max_input_buffer:%zu\r\n"
"client_recent_max_output_buffer:%zu\r\n"
"blocked_clients:%d\r\n",
"blocked_clients:%d\r\n"
"tracking_clients:%d\r\n",
listLength(server.clients)-listLength(server.slaves),
maxin, maxout,
server.blocked_clients);
server.blocked_clients,
server.tracking_clients);
}
/* Memory */
@ -3983,8 +4114,11 @@ sds genRedisInfoString(char *section) {
mh->allocator_rss_bytes,
mh->rss_extra,
mh->rss_extra_bytes,
mh->total_frag, /* this is the total RSS overhead, including fragmentation, */
mh->total_frag_bytes, /* named so for backwards compatibility */
mh->total_frag, /* This is the total RSS overhead, including
fragmentation, but not just it. This field
(and the next one) is named like that just
for backward compatibility. */
mh->total_frag_bytes,
freeMemoryGetNotCountedMemory(),
mh->repl_backlog,
mh->clients_slaves,
@ -4017,7 +4151,9 @@ sds genRedisInfoString(char *section) {
"aof_current_rewrite_time_sec:%jd\r\n"
"aof_last_bgrewrite_status:%s\r\n"
"aof_last_write_status:%s\r\n"
"aof_last_cow_size:%zu\r\n",
"aof_last_cow_size:%zu\r\n"
"module_fork_in_progress:%d\r\n"
"module_fork_last_cow_size:%zu\r\n",
server.loading,
server.dirty,
server.rdb_child_pid != -1,
@ -4035,9 +4171,11 @@ sds genRedisInfoString(char *section) {
-1 : time(NULL)-server.aof_rewrite_time_start),
(server.aof_lastbgrewrite_status == C_OK) ? "ok" : "err",
(server.aof_last_write_status == C_OK) ? "ok" : "err",
server.stat_aof_cow_bytes);
server.stat_aof_cow_bytes,
server.module_child_pid != -1,
server.stat_module_cow_bytes);
if (server.aof_state != AOF_OFF) {
if (server.aof_enabled) {
info = sdscatprintf(info,
"aof_current_size:%lld\r\n"
"aof_base_size:%lld\r\n"
@ -4117,7 +4255,8 @@ sds genRedisInfoString(char *section) {
"active_defrag_hits:%lld\r\n"
"active_defrag_misses:%lld\r\n"
"active_defrag_key_hits:%lld\r\n"
"active_defrag_key_misses:%lld\r\n",
"active_defrag_key_misses:%lld\r\n"
"tracking_used_slots:%lld\r\n",
server.stat_numconnections,
server.stat_numcommands,
getInstantaneousMetric(STATS_METRIC_COMMAND),
@ -4143,7 +4282,8 @@ sds genRedisInfoString(char *section) {
server.stat_active_defrag_hits,
server.stat_active_defrag_misses,
server.stat_active_defrag_key_hits,
server.stat_active_defrag_key_misses);
server.stat_active_defrag_key_misses,
trackingGetUsedSlots());
}
/* Replication */
@ -4191,7 +4331,7 @@ sds genRedisInfoString(char *section) {
if (server.repl_state != REPL_STATE_CONNECTED) {
info = sdscatprintf(info,
"master_link_down_since_seconds:%jd\r\n",
(intmax_t)server.unixtime-server.repl_down_since);
(intmax_t)(server.unixtime-server.repl_down_since));
}
info = sdscatprintf(info,
"slave_priority:%d\r\n"
@ -4227,7 +4367,7 @@ sds genRedisInfoString(char *section) {
long lag = 0;
if (slaveip[0] == '\0') {
if (anetPeerToString(slave->fd,ip,sizeof(ip),&port) == -1)
if (connPeerToString(slave->conn,ip,sizeof(ip),&port) == -1)
continue;
slaveip = ip;
}
@ -4289,6 +4429,13 @@ sds genRedisInfoString(char *section) {
(long)c_ru.ru_utime.tv_sec, (long)c_ru.ru_utime.tv_usec);
}
/* Modules */
if (allsections || defsections || !strcasecmp(section,"modules")) {
if (sections++) info = sdscat(info,"\r\n");
info = sdscatprintf(info,"# Modules\r\n");
info = genModulesInfoString(info);
}
/* Command statistics */
if (allsections || !strcasecmp(section,"commandstats")) {
if (sections++) info = sdscat(info,"\r\n");
@ -4334,6 +4481,17 @@ sds genRedisInfoString(char *section) {
}
}
}
/* Get info from modules.
* if user asked for "everything" or "modules", or a specific section
* that's not found yet. */
if (everything || modules ||
(!allsections && !defsections && sections==0)) {
info = modulesCollectInfo(info,
everything || modules ? NULL: section,
0, /* not a crash report */
sections);
}
return info;
}
@ -4344,7 +4502,9 @@ void infoCommand(client *c) {
addReply(c,shared.syntaxerr);
return;
}
addReplyBulkSds(c, genRedisInfoString(section));
sds info = genRedisInfoString(section);
addReplyVerbatim(c,info,sdslen(info),"txt");
sdsfree(info);
}
void monitorCommand(client *c) {
@ -4461,7 +4621,7 @@ void redisAsciiArt(void) {
if (!show_logo) {
serverLog(LL_NOTICE,
"Running mode=%s, port=%d.",
mode, server.port
mode, server.port ? server.port : server.tls_port
);
} else {
snprintf(buf,1024*16,ascii_logo,
@ -4469,7 +4629,7 @@ void redisAsciiArt(void) {
redisGitSHA1(),
strtol(redisGitDirty(),NULL,10) > 0,
(sizeof(long) == 8) ? "64" : "32",
mode, server.port,
mode, server.port ? server.port : server.tls_port,
(long) getpid()
);
serverLogRaw(LL_NOTICE|LL_RAW,buf);
@ -4500,6 +4660,7 @@ static void sigShutdownHandler(int sig) {
rdbRemoveTempFile(getpid());
exit(1); /* Exit with an error since this was not a clean shutdown. */
} else if (server.loading) {
serverLogFromHandler(LL_WARNING, "Received shutdown signal during loading, exiting now.");
exit(0);
}
@ -4530,6 +4691,61 @@ void setupSignalHandlers(void) {
return;
}
/* This is the signal handler for children process. It is currently useful
* in order to track the SIGUSR1, that we send to a child in order to terminate
* it in a clean way, without the parent detecting an error and stop
* accepting writes because of a write error condition. */
static void sigKillChildHandler(int sig) {
UNUSED(sig);
serverLogFromHandler(LL_WARNING, "Received SIGUSR1 in child, exiting now.");
exitFromChild(SERVER_CHILD_NOERROR_RETVAL);
}
void setupChildSignalHandlers(void) {
struct sigaction act;
/* When the SA_SIGINFO flag is set in sa_flags then sa_sigaction is used.
* Otherwise, sa_handler is used. */
sigemptyset(&act.sa_mask);
act.sa_flags = 0;
act.sa_handler = sigKillChildHandler;
sigaction(SIGUSR1, &act, NULL);
return;
}
int redisFork() {
int childpid;
long long start = ustime();
if ((childpid = fork()) == 0) {
/* Child */
closeListeningSockets(0);
setupChildSignalHandlers();
} else {
/* Parent */
server.stat_fork_time = ustime()-start;
server.stat_fork_rate = (double) zmalloc_used_memory() * 1000000 / server.stat_fork_time / (1024*1024*1024); /* GB per second. */
latencyAddSampleIfNeeded("fork",server.stat_fork_time/1000);
if (childpid == -1) {
return -1;
}
updateDictResizePolicy();
}
return childpid;
}
void sendChildCOWInfo(int ptype, char *pname) {
size_t private_dirty = zmalloc_get_private_dirty(-1);
if (private_dirty) {
serverLog(LL_NOTICE,
"%s: %zu MB of memory used by copy-on-write",
pname, private_dirty/(1024*1024));
}
server.child_info_data.cow_size = private_dirty;
sendChildInfo(ptype);
}
void memtest(size_t megabytes, int passes);
/* Returns 1 if there is --sentinel among the arguments or if
@ -4556,12 +4772,14 @@ void loadDataFromDisk(void) {
(float)(ustime()-start)/1000000);
/* Restore the replication ID / offset from the RDB file. */
if ((server.masterhost || (server.cluster_enabled && nodeIsSlave(server.cluster->myself)))&&
if ((server.masterhost ||
(server.cluster_enabled &&
nodeIsSlave(server.cluster->myself))) &&
rsi.repl_id_is_set &&
rsi.repl_offset != -1 &&
/* Note that older implementations may save a repl_stream_db
* of -1 inside the RDB file in a wrong way, see more information
* in function rdbPopulateSaveInfo. */
* of -1 inside the RDB file in a wrong way, see more
* information in function rdbPopulateSaveInfo. */
rsi.repl_stream_db != -1)
{
memcpy(server.replid,rsi.repl_id,sizeof(server.replid));
@ -4594,7 +4812,7 @@ void redisSetProcTitle(char *title) {
setproctitle("%s %s:%d%s",
title,
server.bindaddr_count ? server.bindaddr[0] : "*",
server.port,
server.port ? server.port : server.tls_port,
server_mode);
#else
UNUSED(title);
@ -4715,8 +4933,6 @@ int main(int argc, char **argv) {
return sha1Test(argc, argv);
} else if (!strcasecmp(argv[2], "util")) {
return utilTest(argc, argv);
} else if (!strcasecmp(argv[2], "sds")) {
return sdsTest(argc, argv);
} else if (!strcasecmp(argv[2], "endianconv")) {
return endianconvTest(argc, argv);
} else if (!strcasecmp(argv[2], "crc64")) {
@ -4739,14 +4955,15 @@ int main(int argc, char **argv) {
srand(time(NULL)^getpid());
gettimeofday(&tv,NULL);
char hashseed[16];
getRandomHexChars(hashseed,sizeof(hashseed));
dictSetHashFunctionSeed((uint8_t*)hashseed);
uint8_t hashseed[16];
getRandomBytes(hashseed,sizeof(hashseed));
dictSetHashFunctionSeed(hashseed);
server.sentinel_mode = checkForSentinelMode(argc,argv);
initServerConfig();
ACLInit(); /* The ACL subsystem must be initialized ASAP because the
basic networking code and client creation depends on it. */
moduleInitModulesSystem();
tlsInit();
/* Store the executable path and arguments in a safe place in order
* to be able to restart the server later. */
@ -4870,6 +5087,7 @@ int main(int argc, char **argv) {
#endif
moduleLoadFromQueue();
ACLLoadUsersAtStartup();
InitServerLast();
loadDataFromDisk();
if (server.cluster_enabled) {
if (verifyClusterConfigWithData() == C_ERR) {
@ -4879,11 +5097,12 @@ int main(int argc, char **argv) {
exit(1);
}
}
if (server.ipfd_count > 0)
if (server.ipfd_count > 0 || server.tlsfd_count > 0)
serverLog(LL_NOTICE,"Ready to accept connections");
if (server.sofd > 0)
serverLog(LL_NOTICE,"The server is now ready to accept connections at %s", server.unixsocket);
} else {
InitServerLast();
sentinelIsRunning();
}

260
src/server.h
View File

@ -66,6 +66,7 @@ typedef long long mstime_t; /* millisecond time type. */
#include "quicklist.h" /* Lists are encoded as linked lists of
N-elements flat arrays */
#include "rax.h" /* Radix tree */
#include "connection.h" /* Connection abstraction */
/* Following includes allow test functions to be called from Redis main() */
#include "zipmap.h"
@ -84,9 +85,12 @@ typedef long long mstime_t; /* millisecond time type. */
#define CONFIG_MAX_HZ 500
#define MAX_CLIENTS_PER_CLOCK_TICK 200 /* HZ is adapted based on that. */
#define CONFIG_DEFAULT_SERVER_PORT 6379 /* TCP port. */
#define CONFIG_DEFAULT_SERVER_TLS_PORT 0 /* TCP port. */
#define CONFIG_DEFAULT_TCP_BACKLOG 511 /* TCP listen backlog. */
#define CONFIG_DEFAULT_CLIENT_TIMEOUT 0 /* Default client timeout: infinite */
#define CONFIG_DEFAULT_DBNUM 16
#define CONFIG_DEFAULT_IO_THREADS_NUM 1 /* Single threaded by default */
#define CONFIG_DEFAULT_IO_THREADS_DO_READS 0 /* Read + parse from threads? */
#define CONFIG_MAX_LINE 1024
#define CRON_DBS_PER_CALL 16
#define NET_MAX_WRITES_PER_EVENT (1024*64)
@ -130,6 +134,8 @@ typedef long long mstime_t; /* millisecond time type. */
#define CONFIG_DEFAULT_RDB_FILENAME "dump.rdb"
#define CONFIG_DEFAULT_REPL_DISKLESS_SYNC 0
#define CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY 5
#define CONFIG_DEFAULT_RDB_KEY_SAVE_DELAY 0
#define CONFIG_DEFAULT_KEY_LOAD_DELAY 0
#define CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA 1
#define CONFIG_DEFAULT_SLAVE_READ_ONLY 1
#define CONFIG_DEFAULT_SLAVE_IGNORE_MAXMEMORY 1
@ -168,6 +174,7 @@ typedef long long mstime_t; /* millisecond time type. */
#define CONFIG_DEFAULT_DEFRAG_CYCLE_MAX 75 /* 75% CPU max (at upper threshold) */
#define CONFIG_DEFAULT_DEFRAG_MAX_SCAN_FIELDS 1000 /* keys with more than 1000 fields will be processed separately */
#define CONFIG_DEFAULT_PROTO_MAX_BULK_LEN (512ll*1024*1024) /* Bulk request max size */
#define CONFIG_DEFAULT_TRACKING_TABLE_MAX_FILL 10 /* 10% tracking table max fill. */
#define ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP 20 /* Loopkups per loop. */
#define ACTIVE_EXPIRE_CYCLE_FAST_DURATION 1000 /* Microseconds */
@ -175,6 +182,14 @@ typedef long long mstime_t; /* millisecond time type. */
#define ACTIVE_EXPIRE_CYCLE_SLOW 0
#define ACTIVE_EXPIRE_CYCLE_FAST 1
/* Children process will exit with this status code to signal that the
* process terminated without an error: this is useful in order to kill
* a saving child (RDB or AOF one), without triggering in the parent the
* write protection that is normally turned on on write errors.
* Usually children that are terminated with SIGUSR1 will exit with this
* special code. */
#define SERVER_CHILD_NOERROR_RETVAL 255
/* Instantaneous metrics tracking. */
#define STATS_METRIC_SAMPLES 16 /* Number of samples per metric. */
#define STATS_METRIC_COMMAND 0 /* Number of commands executed. */
@ -216,35 +231,36 @@ typedef long long mstime_t; /* millisecond time type. */
#define CMD_LOADING (1ULL<<9) /* "ok-loading" flag */
#define CMD_STALE (1ULL<<10) /* "ok-stale" flag */
#define CMD_SKIP_MONITOR (1ULL<<11) /* "no-monitor" flag */
#define CMD_ASKING (1ULL<<12) /* "cluster-asking" flag */
#define CMD_FAST (1ULL<<13) /* "fast" flag */
#define CMD_SKIP_SLOWLOG (1ULL<<12) /* "no-slowlog" flag */
#define CMD_ASKING (1ULL<<13) /* "cluster-asking" flag */
#define CMD_FAST (1ULL<<14) /* "fast" flag */
/* Command flags used by the module system. */
#define CMD_MODULE_GETKEYS (1ULL<<14) /* Use the modules getkeys interface. */
#define CMD_MODULE_NO_CLUSTER (1ULL<<15) /* Deny on Redis Cluster. */
#define CMD_MODULE_GETKEYS (1ULL<<15) /* Use the modules getkeys interface. */
#define CMD_MODULE_NO_CLUSTER (1ULL<<16) /* Deny on Redis Cluster. */
/* Command flags that describe ACLs categories. */
#define CMD_CATEGORY_KEYSPACE (1ULL<<16)
#define CMD_CATEGORY_READ (1ULL<<17)
#define CMD_CATEGORY_WRITE (1ULL<<18)
#define CMD_CATEGORY_SET (1ULL<<19)
#define CMD_CATEGORY_SORTEDSET (1ULL<<20)
#define CMD_CATEGORY_LIST (1ULL<<21)
#define CMD_CATEGORY_HASH (1ULL<<22)
#define CMD_CATEGORY_STRING (1ULL<<23)
#define CMD_CATEGORY_BITMAP (1ULL<<24)
#define CMD_CATEGORY_HYPERLOGLOG (1ULL<<25)
#define CMD_CATEGORY_GEO (1ULL<<26)
#define CMD_CATEGORY_STREAM (1ULL<<27)
#define CMD_CATEGORY_PUBSUB (1ULL<<28)
#define CMD_CATEGORY_ADMIN (1ULL<<29)
#define CMD_CATEGORY_FAST (1ULL<<30)
#define CMD_CATEGORY_SLOW (1ULL<<31)
#define CMD_CATEGORY_BLOCKING (1ULL<<32)
#define CMD_CATEGORY_DANGEROUS (1ULL<<33)
#define CMD_CATEGORY_CONNECTION (1ULL<<34)
#define CMD_CATEGORY_TRANSACTION (1ULL<<35)
#define CMD_CATEGORY_SCRIPTING (1ULL<<36)
#define CMD_CATEGORY_KEYSPACE (1ULL<<17)
#define CMD_CATEGORY_READ (1ULL<<18)
#define CMD_CATEGORY_WRITE (1ULL<<19)
#define CMD_CATEGORY_SET (1ULL<<20)
#define CMD_CATEGORY_SORTEDSET (1ULL<<21)
#define CMD_CATEGORY_LIST (1ULL<<22)
#define CMD_CATEGORY_HASH (1ULL<<23)
#define CMD_CATEGORY_STRING (1ULL<<24)
#define CMD_CATEGORY_BITMAP (1ULL<<25)
#define CMD_CATEGORY_HYPERLOGLOG (1ULL<<26)
#define CMD_CATEGORY_GEO (1ULL<<27)
#define CMD_CATEGORY_STREAM (1ULL<<28)
#define CMD_CATEGORY_PUBSUB (1ULL<<29)
#define CMD_CATEGORY_ADMIN (1ULL<<30)
#define CMD_CATEGORY_FAST (1ULL<<31)
#define CMD_CATEGORY_SLOW (1ULL<<32)
#define CMD_CATEGORY_BLOCKING (1ULL<<33)
#define CMD_CATEGORY_DANGEROUS (1ULL<<34)
#define CMD_CATEGORY_CONNECTION (1ULL<<35)
#define CMD_CATEGORY_TRANSACTION (1ULL<<36)
#define CMD_CATEGORY_SCRIPTING (1ULL<<37)
/* AOF states */
#define AOF_OFF 0 /* AOF is off */
@ -252,8 +268,8 @@ typedef long long mstime_t; /* millisecond time type. */
#define AOF_WAIT_REWRITE 2 /* AOF waits rewrite to start appending */
/* Client flags */
#define CLIENT_SLAVE (1<<0) /* This client is a slave server */
#define CLIENT_MASTER (1<<1) /* This client is a master server */
#define CLIENT_SLAVE (1<<0) /* This client is a repliaca */
#define CLIENT_MASTER (1<<1) /* This client is a master */
#define CLIENT_MONITOR (1<<2) /* This client is a slave monitor, see MONITOR */
#define CLIENT_MULTI (1<<3) /* This client is in a MULTI context */
#define CLIENT_BLOCKED (1<<4) /* The client is waiting in a blocking operation */
@ -284,6 +300,16 @@ typedef long long mstime_t; /* millisecond time type. */
#define CLIENT_LUA_DEBUG_SYNC (1<<26) /* EVAL debugging without fork() */
#define CLIENT_MODULE (1<<27) /* Non connected client used by some module. */
#define CLIENT_PROTECTED (1<<28) /* Client should not be freed for now. */
#define CLIENT_PENDING_READ (1<<29) /* The client has pending reads and was put
in the list of clients we can read
from. */
#define CLIENT_PENDING_COMMAND (1<<30) /* Used in threaded I/O to signal after
we return single threaded that the
client has already pending commands
to be executed. */
#define CLIENT_TRACKING (1<<31) /* Client enabled keys tracking in order to
perform client side caching. */
#define CLIENT_TRACKING_BROKEN_REDIR (1ULL<<32) /* Target client is invalid. */
/* Client block type (btype field in client structure)
* if CLIENT_BLOCKED flag is set. */
@ -382,6 +408,12 @@ typedef long long mstime_t; /* millisecond time type. */
#define AOF_FSYNC_EVERYSEC 2
#define CONFIG_DEFAULT_AOF_FSYNC AOF_FSYNC_EVERYSEC
/* Replication diskless load defines */
#define REPL_DISKLESS_LOAD_DISABLED 0
#define REPL_DISKLESS_LOAD_WHEN_DB_EMPTY 1
#define REPL_DISKLESS_LOAD_SWAPDB 2
#define CONFIG_DEFAULT_REPL_DISKLESS_LOAD REPL_DISKLESS_LOAD_DISABLED
/* Zipped structures related defaults */
#define OBJ_HASH_MAX_ZIPLIST_ENTRIES 512
#define OBJ_HASH_MAX_ZIPLIST_VALUE 64
@ -468,7 +500,8 @@ typedef long long mstime_t; /* millisecond time type. */
#define NOTIFY_EXPIRED (1<<8) /* x */
#define NOTIFY_EVICTED (1<<9) /* e */
#define NOTIFY_STREAM (1<<10) /* t */
#define NOTIFY_ALL (NOTIFY_GENERIC | NOTIFY_STRING | NOTIFY_LIST | NOTIFY_SET | NOTIFY_HASH | NOTIFY_ZSET | NOTIFY_EXPIRED | NOTIFY_EVICTED | NOTIFY_STREAM) /* A flag */
#define NOTIFY_KEY_MISS (1<<11) /* m */
#define NOTIFY_ALL (NOTIFY_GENERIC | NOTIFY_STRING | NOTIFY_LIST | NOTIFY_SET | NOTIFY_HASH | NOTIFY_ZSET | NOTIFY_EXPIRED | NOTIFY_EVICTED | NOTIFY_STREAM | NOTIFY_KEY_MISS) /* A flag */
/* Get the first bind addr or NULL */
#define NET_FIRST_BIND_ADDR (server.bindaddr_count ? server.bindaddr[0] : NULL)
@ -516,6 +549,10 @@ typedef long long mstime_t; /* millisecond time type. */
#define REDISMODULE_TYPE_ENCVER(id) (id & REDISMODULE_TYPE_ENCVER_MASK)
#define REDISMODULE_TYPE_SIGN(id) ((id & ~((uint64_t)REDISMODULE_TYPE_ENCVER_MASK)) >>REDISMODULE_TYPE_ENCVER_BITS)
/* Bit flags for moduleTypeAuxSaveFunc */
#define REDISMODULE_AUX_BEFORE_RDB (1<<0)
#define REDISMODULE_AUX_AFTER_RDB (1<<1)
struct RedisModule;
struct RedisModuleIO;
struct RedisModuleDigest;
@ -528,6 +565,8 @@ struct redisObject;
* is deleted. */
typedef void *(*moduleTypeLoadFunc)(struct RedisModuleIO *io, int encver);
typedef void (*moduleTypeSaveFunc)(struct RedisModuleIO *io, void *value);
typedef int (*moduleTypeAuxLoadFunc)(struct RedisModuleIO *rdb, int encver, int when);
typedef void (*moduleTypeAuxSaveFunc)(struct RedisModuleIO *rdb, int when);
typedef void (*moduleTypeRewriteFunc)(struct RedisModuleIO *io, struct redisObject *key, void *value);
typedef void (*moduleTypeDigestFunc)(struct RedisModuleDigest *digest, void *value);
typedef size_t (*moduleTypeMemUsageFunc)(const void *value);
@ -544,6 +583,9 @@ typedef struct RedisModuleType {
moduleTypeMemUsageFunc mem_usage;
moduleTypeDigestFunc digest;
moduleTypeFreeFunc free;
moduleTypeAuxLoadFunc aux_load;
moduleTypeAuxSaveFunc aux_save;
int aux_save_triggers;
char name[10]; /* 9 bytes name + null term. Charset: A-Z a-z 0-9 _- */
} moduleType;
@ -578,16 +620,18 @@ typedef struct RedisModuleIO {
int ver; /* Module serialization version: 1 (old),
* 2 (current version with opcodes annotation). */
struct RedisModuleCtx *ctx; /* Optional context, see RM_GetContextFromIO()*/
struct redisObject *key; /* Optional name of key processed */
} RedisModuleIO;
/* Macro to initialize an IO context. Note that the 'ver' field is populated
* inside rdb.c according to the version of the value to load. */
#define moduleInitIOContext(iovar,mtype,rioptr) do { \
#define moduleInitIOContext(iovar,mtype,rioptr,keyptr) do { \
iovar.rio = rioptr; \
iovar.type = mtype; \
iovar.bytes = 0; \
iovar.error = 0; \
iovar.ver = 0; \
iovar.key = keyptr; \
iovar.ctx = NULL; \
} while(0);
@ -637,6 +681,11 @@ typedef struct redisObject {
void *ptr;
} robj;
/* The a string name for an object's type as listed above
* Native types are checked against the OBJ_STRING, OBJ_LIST, OBJ_* defines,
* and Module types have their registered name returned. */
char *getObjectTypeName(robj*);
/* Macro used to initialize a Redis object allocated on the stack.
* Note that this macro is taken near the structure definition to make sure
* we'll update it when the structure is changed, to avoid bugs like
@ -778,9 +827,14 @@ typedef struct user {
/* With multiplexing we need to take per-client state.
* Clients are taken in a linked list. */
#define CLIENT_ID_AOF (UINT64_MAX) /* Reserved ID for the AOF client. If you
need more reserved IDs use UINT64_MAX-1,
-2, ... and so forth. */
typedef struct client {
uint64_t id; /* Client incremental unique ID. */
int fd; /* Client socket. */
connection *conn;
int resp; /* RESP protocol version. Can be 2 or 3. */
redisDb *db; /* Pointer to currently SELECTed DB. */
robj *name; /* As set by CLIENT SETNAME. */
@ -807,10 +861,10 @@ typedef struct client {
time_t ctime; /* Client creation time. */
time_t lastinteraction; /* Time of the last interaction, used for timeout */
time_t obuf_soft_limit_reached_time;
int flags; /* Client flags: CLIENT_* macros. */
uint64_t flags; /* Client flags: CLIENT_* macros. */
int authenticated; /* Needed when the default user requires auth. */
int replstate; /* Replication state if this is a slave. */
int repl_put_online_on_ack; /* Install slave write handler on ACK. */
int repl_put_online_on_ack; /* Install slave write handler on first ACK. */
int repldbfd; /* Replication DB file descriptor. */
off_t repldboff; /* Replication DB file offset. */
off_t repldbsize; /* Replication DB file size. */
@ -836,6 +890,11 @@ typedef struct client {
sds peerid; /* Cached peer ID. */
listNode *client_list_node; /* list node in client list */
/* If this client is in tracking mode and this field is non zero,
* invalidation messages for keys fetched by this client will be send to
* the specified client ID. */
uint64_t client_tracking_redirection;
/* Response buffer */
int bufpos;
char buf[PROTO_REPLY_CHUNK_BYTES];
@ -854,7 +913,7 @@ struct moduleLoadQueueEntry {
struct sharedObjectsStruct {
robj *crlf, *ok, *err, *emptybulk, *czero, *cone, *pong, *space,
*colon, *queued, *null[4], *nullarray[4],
*colon, *queued, *null[4], *nullarray[4], *emptymap[4], *emptyset[4],
*emptyarray, *wrongtypeerr, *nokeyerr, *syntaxerr, *sameobjecterr,
*outofrangeerr, *noscripterr, *loadingerr, *slowscripterr, *bgsaveerr,
*masterdownerr, *roslaveerr, *execaborterr, *noautherr, *noreplicaserr,
@ -983,6 +1042,22 @@ struct malloc_stats {
size_t allocator_resident;
};
/*-----------------------------------------------------------------------------
* TLS Context Configuration
*----------------------------------------------------------------------------*/
typedef struct redisTLSContextConfig {
char *cert_file;
char *key_file;
char *dh_params_file;
char *ca_cert_file;
char *ca_cert_dir;
char *protocols;
char *ciphers;
char *ciphersuites;
int prefer_server_ciphers;
} redisTLSContextConfig;
/*-----------------------------------------------------------------------------
* Global server state
*----------------------------------------------------------------------------*/
@ -998,6 +1073,7 @@ struct clusterState;
#define CHILD_INFO_MAGIC 0xC17DDA7A12345678LL
#define CHILD_INFO_TYPE_RDB 0
#define CHILD_INFO_TYPE_AOF 1
#define CHILD_INFO_TYPE_MODULE 3
struct redisServer {
/* General */
@ -1014,7 +1090,7 @@ struct redisServer {
dict *commands; /* Command table */
dict *orig_commands; /* Command table before command renaming. */
aeEventLoop *el;
unsigned int lruclock; /* Clock for LRU eviction */
_Atomic unsigned int lruclock; /* Clock for LRU eviction */
int shutdown_asap; /* SHUTDOWN needed ASAP */
int activerehashing; /* Incremental rehash in serverCron() */
int active_defrag_running; /* Active defragmentation running (holds current scan aggressiveness) */
@ -1033,8 +1109,10 @@ struct redisServer {
int module_blocked_pipe[2]; /* Pipe used to awake the event loop if a
client blocked on a module command needs
to be processed. */
pid_t module_child_pid; /* PID of module child */
/* Networking */
int port; /* TCP listening port */
int tls_port; /* TLS listening port */
int tcp_backlog; /* TCP listen() backlog */
char *bindaddr[CONFIG_BINDADDR_MAX]; /* Addresses we should bind to */
int bindaddr_count; /* Number of addresses in server.bindaddr[] */
@ -1042,12 +1120,15 @@ struct redisServer {
mode_t unixsocketperm; /* UNIX socket permission */
int ipfd[CONFIG_BINDADDR_MAX]; /* TCP socket file descriptors */
int ipfd_count; /* Used slots in ipfd[] */
int tlsfd[CONFIG_BINDADDR_MAX]; /* TLS socket file descriptors */
int tlsfd_count; /* Used slots in tlsfd[] */
int sofd; /* Unix socket file descriptor */
int cfd[CONFIG_BINDADDR_MAX];/* Cluster bus listening socket */
int cfd_count; /* Used slots in cfd[] */
list *clients; /* List of active clients */
list *clients_to_close; /* Clients to close asynchronously */
list *clients_pending_write; /* There is to write or install handler. */
list *clients_pending_read; /* Client has pending read socket buffers. */
list *slaves, *monitors; /* List of slaves and MONITORs */
client *current_client; /* Current client, only used on crash report */
rax *clients_index; /* Active clients dictionary by client ID. */
@ -1055,10 +1136,13 @@ struct redisServer {
mstime_t clients_pause_end_time; /* Time when we undo clients_paused */
char neterr[ANET_ERR_LEN]; /* Error buffer for anet.c */
dict *migrate_cached_sockets;/* MIGRATE cached sockets */
uint64_t next_client_id; /* Next client unique ID. Incremental. */
_Atomic uint64_t next_client_id; /* Next client unique ID. Incremental. */
int protected_mode; /* Don't accept external connections. */
int gopher_enabled; /* If true the server will reply to gopher
queries. Will still serve RESP2 queries. */
int io_threads_num; /* Number of IO threads to use. */
int io_threads_do_reads; /* Read and parse from IO threads? */
/* RDB / AOF loading information */
int loading; /* We are loading data from disk if true */
off_t loading_total_bytes;
@ -1098,10 +1182,11 @@ struct redisServer {
long long slowlog_log_slower_than; /* SLOWLOG time limit (to get logged) */
unsigned long slowlog_max_len; /* SLOWLOG max number of items logged */
struct malloc_stats cron_malloc_stats; /* sampled in serverCron(). */
long long stat_net_input_bytes; /* Bytes read from network. */
long long stat_net_output_bytes; /* Bytes written to network. */
_Atomic long long stat_net_input_bytes; /* Bytes read from network. */
_Atomic long long stat_net_output_bytes; /* Bytes written to network. */
size_t stat_rdb_cow_bytes; /* Copy on write bytes during RDB saving. */
size_t stat_aof_cow_bytes; /* Copy on write bytes during AOF rewrite. */
size_t stat_module_cow_bytes; /* Copy on write bytes during module fork. */
/* The following two are used to track instantaneous metrics, like
* number of operations per second, network traffic. */
struct {
@ -1116,19 +1201,21 @@ struct redisServer {
int tcpkeepalive; /* Set SO_KEEPALIVE if non-zero. */
int active_expire_enabled; /* Can be disabled for testing purposes. */
int active_defrag_enabled;
int jemalloc_bg_thread; /* Enable jemalloc background thread */
size_t active_defrag_ignore_bytes; /* minimum amount of fragmentation waste to start active defrag */
int active_defrag_threshold_lower; /* minimum percentage of fragmentation to start active defrag */
int active_defrag_threshold_upper; /* maximum percentage of fragmentation at which we use maximum effort */
int active_defrag_cycle_min; /* minimal effort for defrag in CPU percentage */
int active_defrag_cycle_max; /* maximal effort for defrag in CPU percentage */
unsigned long active_defrag_max_scan_fields; /* maximum number of fields of set/hash/zset/list to process from within the main dict scan */
size_t client_max_querybuf_len; /* Limit for client query buffer length */
_Atomic size_t client_max_querybuf_len; /* Limit for client query buffer length */
int dbnum; /* Total number of configured DBs */
int supervised; /* 1 if supervised, 0 otherwise. */
int supervised_mode; /* See SUPERVISED_* */
int daemonize; /* True if running as a daemon */
clientBufferLimitsConfig client_obuf_limits[CLIENT_TYPE_OBUF_COUNT];
/* AOF persistence */
int aof_enabled; /* AOF configuration */
int aof_state; /* AOF_(ON|OFF|WAIT_REWRITE) */
int aof_fsync; /* Kind of fsync() policy */
char *aof_filename; /* Name of the AOF file */
@ -1137,6 +1224,8 @@ struct redisServer {
off_t aof_rewrite_min_size; /* the AOF file is at least N bytes. */
off_t aof_rewrite_base_size; /* AOF size on latest startup or rewrite. */
off_t aof_current_size; /* AOF current size. */
off_t aof_fsync_offset; /* AOF offset which is already synced to disk. */
int aof_flush_sleep; /* Micros to sleep before flush. (used by tests) */
int aof_rewrite_scheduled; /* Rewrite once BGSAVE terminates. */
pid_t aof_child_pid; /* PID if rewriting process */
list *aof_rewrite_buf_blocks; /* Hold changes during an AOF rewrite. */
@ -1182,8 +1271,17 @@ struct redisServer {
int rdb_child_type; /* Type of save by active child. */
int lastbgsave_status; /* C_OK or C_ERR */
int stop_writes_on_bgsave_err; /* Don't allow writes if can't BGSAVE */
int rdb_pipe_write_result_to_parent; /* RDB pipes used to return the state */
int rdb_pipe_read_result_from_child; /* of each slave in diskless SYNC. */
int rdb_pipe_write; /* RDB pipes used to transfer the rdb */
int rdb_pipe_read; /* data to the parent process in diskless repl. */
connection **rdb_pipe_conns; /* Connections which are currently the */
int rdb_pipe_numconns; /* target of diskless rdb fork child. */
int rdb_pipe_numconns_writing; /* Number of rdb conns with pending writes. */
char *rdb_pipe_buff; /* In diskless replication, this buffer holds data */
int rdb_pipe_bufflen; /* that was read from the the rdb pipe. */
int rdb_key_save_delay; /* Delay in microseconds between keys while
* writing the RDB. (for testings) */
int key_load_delay; /* Delay in microseconds between keys while
* loading aof or rdb. (for testings) */
/* Pipe and data structures for child -> parent info sharing. */
int child_info_pipe[2]; /* Pipe used to write the child_info_data. */
struct {
@ -1219,7 +1317,9 @@ struct redisServer {
int repl_min_slaves_to_write; /* Min number of slaves to write. */
int repl_min_slaves_max_lag; /* Max lag of <count> slaves to write. */
int repl_good_slaves_count; /* Number of slaves with lag <= max_lag. */
int repl_diskless_sync; /* Send RDB to slaves sockets directly. */
int repl_diskless_sync; /* Master send RDB to slaves sockets directly. */
int repl_diskless_load; /* Slave parse RDB directly from the socket.
* see REPL_DISKLESS_LOAD_* enum */
int repl_diskless_sync_delay; /* Delay to start a diskless repl BGSAVE. */
/* Replication (slave) */
char *masteruser; /* AUTH with this user and masterauth with master */
@ -1234,7 +1334,7 @@ struct redisServer {
off_t repl_transfer_size; /* Size of RDB to read from master during sync. */
off_t repl_transfer_read; /* Amount of RDB read from master during sync. */
off_t repl_transfer_last_fsync_off; /* Offset when we fsync-ed last time. */
int repl_transfer_s; /* Slave -> Master SYNC socket */
connection *repl_transfer_s; /* Slave -> Master SYNC connection */
int repl_transfer_fd; /* Slave -> Master SYNC temp file descriptor */
char *repl_transfer_tmpfile; /* Slave-> master SYNC temp file name */
time_t repl_transfer_lastio; /* Unix time of the latest read, for timeout */
@ -1272,6 +1372,9 @@ struct redisServer {
unsigned int blocked_clients_by_type[BLOCKED_NUM];
list *unblocked_clients; /* list of clients to unblock before next loop */
list *ready_keys; /* List of readyList structures for BLPOP & co */
/* Client side caching. */
unsigned int tracking_clients; /* # of clients with tracking enabled.*/
int tracking_table_max_fill; /* Max fill percentage. */
/* Sort parameters - qsort_r() is only available under BSD so we
* have to take this state global, in order to pass it to sortCompare() */
int sort_desc;
@ -1291,10 +1394,10 @@ struct redisServer {
int list_max_ziplist_size;
int list_compress_depth;
/* time cache */
time_t unixtime; /* Unix time sampled every cron cycle. */
time_t timezone; /* Cached timezone. As set by tzset(). */
int daylight_active; /* Currently in daylight saving time. */
long long mstime; /* Like 'unixtime' but with milliseconds resolution. */
_Atomic time_t unixtime; /* Unix time sampled every cron cycle. */
time_t timezone; /* Cached timezone. As set by tzset(). */
int daylight_active; /* Currently in daylight saving time. */
long long mstime; /* 'unixtime' with milliseconds resolution. */
/* Pubsub */
dict *pubsub_channels; /* Map channels to list of subscribed clients */
list *pubsub_patterns; /* A list of pubsub_patterns */
@ -1322,6 +1425,7 @@ struct redisServer {
lua_State *lua; /* The Lua interpreter. We use just one for all clients */
client *lua_client; /* The "fake client" to query Redis from Lua */
client *lua_caller; /* The client running EVAL right now, or NULL */
char* lua_cur_script; /* SHA1 of the script currently running, or NULL */
dict *lua_scripts; /* A dictionary of SHA1 -> Lua scripts */
unsigned long long lua_scripts_mem; /* Cached scripts' memory + oh */
mstime_t lua_time_limit; /* Script timeout in milliseconds */
@ -1354,12 +1458,11 @@ struct redisServer {
int watchdog_period; /* Software watchdog period in ms. 0 = off */
/* System hardware info */
size_t system_memory_size; /* Total memory in system as reported by OS */
/* Mutexes used to protect atomic variables when atomic builtins are
* not available. */
pthread_mutex_t lruclock_mutex;
pthread_mutex_t next_client_id_mutex;
pthread_mutex_t unixtime_mutex;
/* TLS Configuration */
int tls_cluster;
int tls_replication;
int tls_auth_clients;
redisTLSContextConfig tls_ctx_config;
};
typedef struct pubsubPattern {
@ -1489,7 +1592,12 @@ size_t moduleCount(void);
void moduleAcquireGIL(void);
void moduleReleaseGIL(void);
void moduleNotifyKeyspaceEvent(int type, const char *event, robj *key, int dbid);
void moduleCallCommandFilters(client *c);
void ModuleForkDoneHandler(int exitcode, int bysignal);
int TerminateModuleForkChild(int child_pid, int wait);
ssize_t rdbSaveModulesAux(rio *rdb, int when);
int moduleAllDatatypesHandleErrors();
sds modulesCollectInfo(sds info, sds section, int for_crash_report, int sections);
/* Utils */
long long ustime(void);
@ -1502,12 +1610,12 @@ size_t redisPopcount(void *s, long count);
void redisSetProcTitle(char *title);
/* networking.c -- Networking and Client related operations */
client *createClient(int fd);
client *createClient(connection *conn);
void closeTimedoutClients(void);
void freeClient(client *c);
void freeClientAsync(client *c);
void resetClient(client *c);
void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask);
void sendReplyToClient(connection *conn);
void *addReplyDeferredLen(client *c);
void setDeferredArrayLen(client *c, void *node, long length);
void setDeferredMapLen(client *c, void *node, long length);
@ -1519,13 +1627,15 @@ void processInputBufferAndReplicate(client *c);
void processGopherRequest(client *c);
void acceptHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void acceptTcpHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void acceptTLSHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void acceptUnixHandler(aeEventLoop *el, int fd, void *privdata, int mask);
void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask);
void readQueryFromClient(connection *conn);
void addReplyNull(client *c);
void addReplyNullArray(client *c);
void addReplyBool(client *c, int b);
void addReplyVerbatim(client *c, const char *s, size_t len, const char *ext);
void addReplyProto(client *c, const char *s, size_t len);
void AddReplyFromClient(client *c, client *src);
void addReplyBulk(client *c, robj *obj);
void addReplyBulkCString(client *c, const char *s);
void addReplyBulkCBuffer(client *c, const void *p, size_t len);
@ -1572,12 +1682,17 @@ void pauseClients(mstime_t duration);
int clientsArePaused(void);
int processEventsWhileBlocked(void);
int handleClientsWithPendingWrites(void);
int handleClientsWithPendingWritesUsingThreads(void);
int handleClientsWithPendingReadsUsingThreads(void);
int stopThreadedIOIfNeeded(void);
int clientHasPendingReplies(client *c);
void unlinkClient(client *c);
int writeToClient(int fd, client *c, int handler_installed);
int writeToClient(client *c, int handler_installed);
void linkClient(client *c);
void protectClient(client *c);
void unprotectClient(client *c);
void initThreadedIO(void);
client *lookupClientByID(uint64_t id);
#ifdef __GNUC__
void addReplyErrorFormat(client *c, const char *fmt, ...)
@ -1589,6 +1704,15 @@ void addReplyErrorFormat(client *c, const char *fmt, ...);
void addReplyStatusFormat(client *c, const char *fmt, ...);
#endif
/* Client side caching (tracking mode) */
void enableTracking(client *c, uint64_t redirect_to);
void disableTracking(client *c);
void trackingRememberKeys(client *c);
void trackingInvalidateKey(robj *keyobj);
void trackingInvalidateKeysOnFlush(int dbid);
void trackingLimitUsedSlots(void);
unsigned long long trackingGetUsedSlots(void);
/* List data type */
void listTypeTryConversion(robj *subject, robj *value);
void listTypePush(robj *subject, robj *value, int where);
@ -1699,9 +1823,12 @@ void clearReplicationId2(void);
void chopReplicationBacklog(void);
void replicationCacheMasterUsingMyself(void);
void feedReplicationBacklog(void *ptr, size_t len);
void rdbPipeReadHandler(struct aeEventLoop *eventLoop, int fd, void *clientData, int mask);
void rdbPipeWriteHandlerConnRemoved(struct connection *conn);
/* Generic persistence functions */
void startLoading(FILE *fp);
void startLoadingFile(FILE* fp, char* filename);
void startLoading(size_t size);
void loadingProgress(off_t pos);
void stopLoading(void);
@ -1735,6 +1862,11 @@ void closeChildInfoPipe(void);
void sendChildInfo(int process_type);
void receiveChildInfo(void);
/* Fork helpers */
int redisFork();
int hasActiveChildProcess();
void sendChildCOWInfo(int ptype, char *pname);
/* acl.c -- Authentication related prototypes. */
extern rax *Users;
extern user *DefaultUser;
@ -1835,6 +1967,8 @@ struct redisCommand *lookupCommandOrOriginal(sds name);
void call(client *c, int flags);
void propagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int flags);
void alsoPropagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int target);
void redisOpArrayInit(redisOpArray *oa);
void redisOpArrayFree(redisOpArray *oa);
void forceCommandPropagation(client *c, int flags);
void preventCommandPropagation(client *c);
void preventCommandAOF(client *c);
@ -1863,6 +1997,7 @@ unsigned int LRU_CLOCK(void);
const char *evictPolicyToString(void);
struct redisMemOverhead *getMemoryOverheadData(void);
void freeMemoryOverheadData(struct redisMemOverhead *mh);
void checkChildrenDone(void);
#define RESTART_SERVER_NONE 0
#define RESTART_SERVER_GRACEFULLY (1<<0) /* Do proper shutdown. */
@ -1913,6 +2048,7 @@ int pubsubUnsubscribeAllPatterns(client *c, int notify);
void freePubsubPattern(void *p);
int listMatchPubsubPattern(void *a, void *b);
int pubsubPublishMessage(robj *channel, robj *message);
void addReplyPubsubMessage(client *c, robj *channel, robj *msg);
/* Keyspace events notification */
void notifyKeyspaceEvent(int type, char *event, robj *key, int dbid);
@ -1957,6 +2093,8 @@ robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o);
#define EMPTYDB_NO_FLAGS 0 /* No flags. */
#define EMPTYDB_ASYNC (1<<0) /* Reclaim memory in another thread. */
long long emptyDb(int dbnum, int flags, void(callback)(void*));
long long emptyDbGeneric(redisDb *dbarray, int dbnum, int flags, void(callback)(void*));
long long dbTotalServerKeyCount();
int selectDb(client *c, int id);
void signalModifiedKey(redisDb *db, robj *key);
@ -2050,6 +2188,7 @@ void dictSdsDestructor(void *privdata, void *val);
char *redisGitSHA1(void);
char *redisGitDirty(void);
uint64_t redisBuildId(void);
char *redisBuildIdString(void);
/* Commands prototypes */
void authCommand(client *c);
@ -2264,6 +2403,7 @@ void bugReportStart(void);
void serverLogObjectDebugInfo(const robj *o);
void sigsegvHandler(int sig, siginfo_t *info, void *secret);
sds genRedisInfoString(char *section);
sds genModulesInfoString(sds info);
void enableWatchdog(int period);
void disableWatchdog(void);
void watchdogScheduleSignal(int period);
@ -2273,6 +2413,10 @@ void mixDigest(unsigned char *digest, void *ptr, size_t len);
void xorDigest(unsigned char *digest, void *ptr, size_t len);
int populateCommandTableParseFlags(struct redisCommand *c, char *strflags);
/* TLS stuff */
void tlsInit(void);
int tlsConfigure(redisTLSContextConfig *ctx_config);
#define redisDebug(fmt, ...) \
printf("DEBUG %s:%d > " fmt "\n", __FILE__, __LINE__, __VA_ARGS__)
#define redisDebugMark() \

158
src/sha256.c Normal file
View File

@ -0,0 +1,158 @@
/*********************************************************************
* Filename: sha256.c
* Author: Brad Conte (brad AT bradconte.com)
* Copyright:
* Disclaimer: This code is presented "as is" without any guarantees.
* Details: Implementation of the SHA-256 hashing algorithm.
SHA-256 is one of the three algorithms in the SHA2
specification. The others, SHA-384 and SHA-512, are not
offered in this implementation.
Algorithm specification can be found here:
* http://csrc.nist.gov/publications/fips/fips180-2/fips180-2withchangenotice.pdf
This implementation uses little endian byte order.
*********************************************************************/
/*************************** HEADER FILES ***************************/
#include <stdlib.h>
#include <string.h>
#include "sha256.h"
/****************************** MACROS ******************************/
#define ROTLEFT(a,b) (((a) << (b)) | ((a) >> (32-(b))))
#define ROTRIGHT(a,b) (((a) >> (b)) | ((a) << (32-(b))))
#define CH(x,y,z) (((x) & (y)) ^ (~(x) & (z)))
#define MAJ(x,y,z) (((x) & (y)) ^ ((x) & (z)) ^ ((y) & (z)))
#define EP0(x) (ROTRIGHT(x,2) ^ ROTRIGHT(x,13) ^ ROTRIGHT(x,22))
#define EP1(x) (ROTRIGHT(x,6) ^ ROTRIGHT(x,11) ^ ROTRIGHT(x,25))
#define SIG0(x) (ROTRIGHT(x,7) ^ ROTRIGHT(x,18) ^ ((x) >> 3))
#define SIG1(x) (ROTRIGHT(x,17) ^ ROTRIGHT(x,19) ^ ((x) >> 10))
/**************************** VARIABLES *****************************/
static const WORD k[64] = {
0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5,0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5,
0xd807aa98,0x12835b01,0x243185be,0x550c7dc3,0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174,
0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc,0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da,
0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7,0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967,
0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13,0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85,
0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3,0xd192e819,0xd6990624,0xf40e3585,0x106aa070,
0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5,0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3,
0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208,0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2
};
/*********************** FUNCTION DEFINITIONS ***********************/
void sha256_transform(SHA256_CTX *ctx, const BYTE data[])
{
WORD a, b, c, d, e, f, g, h, i, j, t1, t2, m[64];
for (i = 0, j = 0; i < 16; ++i, j += 4)
m[i] = (data[j] << 24) | (data[j + 1] << 16) | (data[j + 2] << 8) | (data[j + 3]);
for ( ; i < 64; ++i)
m[i] = SIG1(m[i - 2]) + m[i - 7] + SIG0(m[i - 15]) + m[i - 16];
a = ctx->state[0];
b = ctx->state[1];
c = ctx->state[2];
d = ctx->state[3];
e = ctx->state[4];
f = ctx->state[5];
g = ctx->state[6];
h = ctx->state[7];
for (i = 0; i < 64; ++i) {
t1 = h + EP1(e) + CH(e,f,g) + k[i] + m[i];
t2 = EP0(a) + MAJ(a,b,c);
h = g;
g = f;
f = e;
e = d + t1;
d = c;
c = b;
b = a;
a = t1 + t2;
}
ctx->state[0] += a;
ctx->state[1] += b;
ctx->state[2] += c;
ctx->state[3] += d;
ctx->state[4] += e;
ctx->state[5] += f;
ctx->state[6] += g;
ctx->state[7] += h;
}
void sha256_init(SHA256_CTX *ctx)
{
ctx->datalen = 0;
ctx->bitlen = 0;
ctx->state[0] = 0x6a09e667;
ctx->state[1] = 0xbb67ae85;
ctx->state[2] = 0x3c6ef372;
ctx->state[3] = 0xa54ff53a;
ctx->state[4] = 0x510e527f;
ctx->state[5] = 0x9b05688c;
ctx->state[6] = 0x1f83d9ab;
ctx->state[7] = 0x5be0cd19;
}
void sha256_update(SHA256_CTX *ctx, const BYTE data[], size_t len)
{
WORD i;
for (i = 0; i < len; ++i) {
ctx->data[ctx->datalen] = data[i];
ctx->datalen++;
if (ctx->datalen == 64) {
sha256_transform(ctx, ctx->data);
ctx->bitlen += 512;
ctx->datalen = 0;
}
}
}
void sha256_final(SHA256_CTX *ctx, BYTE hash[])
{
WORD i;
i = ctx->datalen;
// Pad whatever data is left in the buffer.
if (ctx->datalen < 56) {
ctx->data[i++] = 0x80;
while (i < 56)
ctx->data[i++] = 0x00;
}
else {
ctx->data[i++] = 0x80;
while (i < 64)
ctx->data[i++] = 0x00;
sha256_transform(ctx, ctx->data);
memset(ctx->data, 0, 56);
}
// Append to the padding the total message's length in bits and transform.
ctx->bitlen += ctx->datalen * 8;
ctx->data[63] = ctx->bitlen;
ctx->data[62] = ctx->bitlen >> 8;
ctx->data[61] = ctx->bitlen >> 16;
ctx->data[60] = ctx->bitlen >> 24;
ctx->data[59] = ctx->bitlen >> 32;
ctx->data[58] = ctx->bitlen >> 40;
ctx->data[57] = ctx->bitlen >> 48;
ctx->data[56] = ctx->bitlen >> 56;
sha256_transform(ctx, ctx->data);
// Since this implementation uses little endian byte ordering and SHA uses big endian,
// reverse all the bytes when copying the final state to the output hash.
for (i = 0; i < 4; ++i) {
hash[i] = (ctx->state[0] >> (24 - i * 8)) & 0x000000ff;
hash[i + 4] = (ctx->state[1] >> (24 - i * 8)) & 0x000000ff;
hash[i + 8] = (ctx->state[2] >> (24 - i * 8)) & 0x000000ff;
hash[i + 12] = (ctx->state[3] >> (24 - i * 8)) & 0x000000ff;
hash[i + 16] = (ctx->state[4] >> (24 - i * 8)) & 0x000000ff;
hash[i + 20] = (ctx->state[5] >> (24 - i * 8)) & 0x000000ff;
hash[i + 24] = (ctx->state[6] >> (24 - i * 8)) & 0x000000ff;
hash[i + 28] = (ctx->state[7] >> (24 - i * 8)) & 0x000000ff;
}
}

35
src/sha256.h Normal file
View File

@ -0,0 +1,35 @@
/*********************************************************************
* Filename: sha256.h
* Author: Brad Conte (brad AT bradconte.com)
* Copyright:
* Disclaimer: This code is presented "as is" without any guarantees.
* Details: Defines the API for the corresponding SHA1 implementation.
*********************************************************************/
#ifndef SHA256_H
#define SHA256_H
/*************************** HEADER FILES ***************************/
#include <stddef.h>
#include <stdint.h>
/****************************** MACROS ******************************/
#define SHA256_BLOCK_SIZE 32 // SHA256 outputs a 32 byte digest
/**************************** DATA TYPES ****************************/
typedef uint8_t BYTE; // 8-bit byte
typedef uint32_t WORD; // 32-bit word
typedef struct {
BYTE data[64];
WORD datalen;
unsigned long long bitlen;
WORD state[8];
} SHA256_CTX;
/*********************** FUNCTION DECLARATIONS **********************/
void sha256_init(SHA256_CTX *ctx);
void sha256_update(SHA256_CTX *ctx, const BYTE data[], size_t len);
void sha256_final(SHA256_CTX *ctx, BYTE hash[]);
#endif // SHA256_H

3
src/siphash.c
View File

@ -58,7 +58,8 @@ int siptlw(int c) {
/* Test of the CPU is Little Endian and supports not aligned accesses.
* Two interesting conditions to speedup the function that happen to be
* in most of x86 servers. */
#if defined(__X86_64__) || defined(__x86_64__) || defined (__i386__)
#if defined(__X86_64__) || defined(__x86_64__) || defined (__i386__) \
|| defined (__aarch64__) || defined (__arm64__)
#define UNALIGNED_LE_CPU
#endif

55
src/sort.c
View File

@ -58,7 +58,7 @@ redisSortOperation *createSortOperation(int type, robj *pattern) {
*
* The returned object will always have its refcount increased by 1
* when it is non-NULL. */
robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst, int writeflag) {
char *p, *f, *k;
sds spat, ssub;
robj *keyobj, *fieldobj = NULL, *o;
@ -106,7 +106,10 @@ robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
decrRefCount(subst); /* Incremented by decodeObject() */
/* Lookup substituted key */
o = lookupKeyRead(db,keyobj);
if (!writeflag)
o = lookupKeyRead(db,keyobj);
else
o = lookupKeyWrite(db,keyobj);
if (o == NULL) goto noobj;
if (fieldobj) {
@ -198,30 +201,12 @@ void sortCommand(client *c) {
robj *sortval, *sortby = NULL, *storekey = NULL;
redisSortObject *vector; /* Resulting vector to sort */
/* Lookup the key to sort. It must be of the right types */
sortval = lookupKeyRead(c->db,c->argv[1]);
if (sortval && sortval->type != OBJ_SET &&
sortval->type != OBJ_LIST &&
sortval->type != OBJ_ZSET)
{
addReply(c,shared.wrongtypeerr);
return;
}
/* Create a list of operations to perform for every sorted element.
* Operations can be GET */
operations = listCreate();
listSetFreeMethod(operations,zfree);
j = 2; /* options start at argv[2] */
/* Now we need to protect sortval incrementing its count, in the future
* SORT may have options able to overwrite/delete keys during the sorting
* and the sorted key itself may get destroyed */
if (sortval)
incrRefCount(sortval);
else
sortval = createQuicklistObject();
/* The SORT command has an SQL-alike syntax, parse it */
while(j < c->argc) {
int leftargs = c->argc-j-1;
@ -280,11 +265,33 @@ void sortCommand(client *c) {
/* Handle syntax errors set during options parsing. */
if (syntax_error) {
decrRefCount(sortval);
listRelease(operations);
return;
}
/* Lookup the key to sort. It must be of the right types */
if (storekey)
sortval = lookupKeyRead(c->db,c->argv[1]);
else
sortval = lookupKeyWrite(c->db,c->argv[1]);
if (sortval && sortval->type != OBJ_SET &&
sortval->type != OBJ_LIST &&
sortval->type != OBJ_ZSET)
{
listRelease(operations);
addReply(c,shared.wrongtypeerr);
return;
}
/* Now we need to protect sortval incrementing its count, in the future
* SORT may have options able to overwrite/delete keys during the sorting
* and the sorted key itself may get destroyed */
if (sortval)
incrRefCount(sortval);
else
sortval = createQuicklistObject();
/* When sorting a set with no sort specified, we must sort the output
* so the result is consistent across scripting and replication.
*
@ -452,7 +459,7 @@ void sortCommand(client *c) {
robj *byval;
if (sortby) {
/* lookup value to sort by */
byval = lookupKeyByPattern(c->db,sortby,vector[j].obj);
byval = lookupKeyByPattern(c->db,sortby,vector[j].obj,storekey!=NULL);
if (!byval) continue;
} else {
/* use object itself to sort by */
@ -515,7 +522,7 @@ void sortCommand(client *c) {
while((ln = listNext(&li))) {
redisSortOperation *sop = ln->value;
robj *val = lookupKeyByPattern(c->db,sop->pattern,
vector[j].obj);
vector[j].obj,storekey!=NULL);
if (sop->type == SORT_OP_GET) {
if (!val) {
@ -545,7 +552,7 @@ void sortCommand(client *c) {
while((ln = listNext(&li))) {
redisSortOperation *sop = ln->value;
robj *val = lookupKeyByPattern(c->db,sop->pattern,
vector[j].obj);
vector[j].obj,storekey!=NULL);
if (sop->type == SORT_OP_GET) {
if (!val) val = createStringObject("",0);

3
src/stream.h
View File

@ -88,7 +88,7 @@ typedef struct streamNACK {
/* Stream propagation informations, passed to functions in order to propagate
* XCLAIM commands to AOF and slaves. */
typedef struct sreamPropInfo {
typedef struct streamPropInfo {
robj *keyname;
robj *groupname;
} streamPropInfo;
@ -109,5 +109,6 @@ streamCG *streamCreateCG(stream *s, char *name, size_t namelen, streamID *id);
streamNACK *streamCreateNACK(streamConsumer *consumer);
void streamDecodeID(void *buf, streamID *id);
int streamCompareID(streamID *a, streamID *b);
void streamFreeNACK(streamNACK *na);
#endif

4
src/t_hash.c
View File

@ -772,8 +772,8 @@ void genericHgetallCommand(client *c, int flags) {
hashTypeIterator *hi;
int length, count = 0;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.null[c->resp])) == NULL
|| checkType(c,o,OBJ_HASH)) return;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.emptymap[c->resp]))
== NULL || checkType(c,o,OBJ_HASH)) return;
/* We return a map if the user requested keys and values, like in the
* HGETALL case. Otherwise to use a flat array makes more sense. */

Some files were not shown because too many files have changed in this diff Show More