Redis: delete and write data

We want to use a DSL supporting EL to delete (DEL, SREM) and write data (SADD) to Redis, similar to a Redis feeder. The motivation to use the Redis feeder this way is that we often use Redis to write, update and read OAuth tokens in scripts. Would this violate the feeders concept? It is planned to commit this? And will you be ready to accept this implementation if we do it by adding redis command based on gatling-redis library?

Example DSL:

class RedisScenario {
 val redisPool = new RedisClientPool("localhost", 6379)
 def myRedisSADDFeeder(value: String, values: String*) = redisFeeder(redisPool, "key", value, values).SADD
 def myRedisDELFeeder(key: String, keys: String*) = redisFeeder(redisPool, key, keys).DEL
 def myRedisSREMFeeder(value: String, values: String*) = redisFeeder(redisPool, "key", value, values).SREM
  
 val getParameter: HttpRequestBuilder = http("GET PARAMETER")
  .get("/")
  .check(regex("""SPDX-License-Identifier: (\w.+)""").saveAs("value_1"))
  .check(regex("""Copyright (\w.+)""").saveAs("value_2"))

 val redisGetTokenScenario: ScenarioBuilder = scenario("Redis scenario")
  .exec(getParameter)
  .feed(myRedisSADDFeeder("${value_1}", "${value_2}"))
  .feed(myRedisDELFeeder("key", "key1"))
  .feed(myRedisSREMFeeder("${value_1}", "${value_2}"))

}

[slandelle]

Several things.

Using Feeder/feed for writing

That’s a big no.

First, Feeder and feed are only meant for pulling records and copy them into Sessions. They are not meant to remove or modify data, all the more from an external system, and don’t take any parameter. So something like requested doesn’t belong there:

  .feed(myRedisSADDFeeder("${value_1}", "${value_2}"))
  .feed(myRedisDELFeeder("key", "key1"))
  .feed(myRedisSREMFeeder("${value_1}", "${value_2}"))

So now quick win by just add some more operations on RedisFeederBuilder.

Feeder currently only supports a blocking API

Feeder was designed to work with in-memory data: CSV and JSON files fully loaded in memory on boot and in-memory random data generator. Those are 100% fine with the blocking API.

Later, I also introduced the lazy feeders able to deal with humongous files. The files are on the local filesystem and the CSV parser is pretty fast, so it’s mostly fine for most use cases.

Now, there’s the case of the JDBC and REDIS feeders that perform a blocking network call on each Iterator.next(). I wouldn’t use those for high throughput use cases. In order for them to be efficient, we’d most likely need new implementations based on an async API, probably respectively on R2DBC and Lettuce. We’d probably have to move those into optional modules because of the extra dependencies.

exec(function) is a blocking API

It was designed to let users edit Session.

• It’s sometimes diverted to push or pull data from a global shared in-memory store, which is fine.
• It’s also sometimes diverted to write some data on disk, which can be an issue if your disk is slow or subject to hiccups.
• It’s also sometimes diverted to perform blocking network IO and this is plain wrong. You would have to implement a proper Action.

[shoaib42]

@slandelle Yikes, I didn’t think about that. I think what @Dmitriy-Smol has requested, is similar to the RPOPLPUSH that I did. Wouldn’t it be a similar issue of blocking call with the feeder, if DEL SADD SREM were to be added? (Adding these would be pretty straight forward)