Using Redis as a queue

Using Redis as a queue

Redis is a great key / value in memory database.

It can be used as an efficient queue as well. One or multiple producers can push events to queue and one or multiple consumers can read data from Redis.

Producer side

The producer can push data to consumers using RPUSH Redis command. This command pushes data to the end of the list. It list is unknown, it is automatically created.

For example RPUSH key1 “string1”.

Another alternative is to push data to the beginning on the list using the LPUSH.
For more information check the following article: https://redis.io/commands/rpush

One or more producers can push data to the same list using the same key, like key1 above.

Consumer side

Redis supports block read on the list, so, the client application will block on socket read until data is available or timeout expired. If multiple consumers wait for data on the same list, only one consumer will receive the data. The other consumers will block until more data is available. According to the Redis docs, the one that waited longer will receive the data.

For this, Redis has the BRPOP and BLPOP operation. For example:
BRPOP key1 timeout

The consumer can block on one or multiple keys. For example:
BRPOP key1 key2 key3 timeout

Timeout value

The read operation can be blocked based on a timeout. A timeout of zero can be used to block indefinitely.

Looking forward hearing your comments and suggestions!

About the author

Yuli Stremovsky
Yuli StremovskyParanoid Security Guy
For the past 15 years I’ve been leading the evolution of startups and enterprises to achieve the highest level of security and compliance. Throughout my career I’ve been a Cyber Security expert and advanced solutions architect with many years of hands on experience both on offensive and defensive side. Knowledgeable at the highest level in application development, networking, data and databases, web applications, large scale Software as a Service solutions, cloud security and blockchain technologies.

I’ve been working with CISO’s of international enterprises, helping them set Information Security strategy, and overseeing the implementation of these recommendations. As part of these projects, I’ve been assisting companies to achieve compliance in GDPR, PCI, HIPAA and SOX.

Among my credits, I was a founder of a database security company GreenSQL/Hexatier which was acquired by Huawei and I’ve co-founded Kesem.io, Secure multi-signature Crypto wallet.

Specialties: Software and cloud architecture, Compliance (GDPR, HIPAA, PCI, SOX), blockchain technologies, software development, secure architectures, project management and low level research.