Possible memory leak in ajv.compile

What version of Ajv are you using?

8.1.0, 8.6.2

Does the issue happen if you use the latest version? Yes

Ajv options object:

{allErrors: true}

Description

As per the issue reference https://github.com/ajv-validator/ajv/issues/1221, we have to use ajv.compile to validate meta schema for a custom keyword, because ajv.validateSchema is not performing it.

While doing so, its observed a possible memory leak in ajv.compile method. On frequent calling of ajv.compile on the same schema object keep increasing the memory usage. Even ajv.removeSchema does not clear up the cache.

Code

const Ajv = require('ajv');

const ajv = new Ajv({allErrors: true});

ajv.addKeyword({
  keyword: "range",
  type: "number",
  compile([min, max], parentSchema) {
    return (data) => data > min && data < max
  },
  errors: false,
  metaSchema: {
    // schema to validate keyword value
    type: "array",
    items: [{type: "number"}, {type: "number"}],
    minItems: 2,
    additionalItems: false,
  },
});

const schema = {
  $id: '/our/schema',
  type: "object",
  properties: {
    foo: {
        type: "number", 
        range: [6, 1] 
    },
    bar: {
        type: "number", 
        range: [1, 5],
    }
  },
  required: ["foo"],
  additionalProperties: false
}

const run = () => {
    try {
       ajv.compile(schema)
    } catch (error) {
       console.error("Ajv schema is invalid", error)
    } finally {
        ajv.removeSchema(schema.$id)
    }
    
    ajv.validate(schema, { foo: 7, bar: 2 })
    ajv.validate(schema, { foo: 1, bar: 4 })
    console.log(Math.ceil(process.memoryUsage().heapUsed / 1024) + "KB")
}

for(let i=0; i < 10; i++) {
  run();
}

Its observe that memory usage is increased linearly, tested it for longer time and it end up filling GBs of memory.

Possible causes

1. It seesm ajv.compile is not recognizing the same schema object
2. And ajv.removeSchema is not clearing up the cache properly

A working example of the code can be tested here.

https://runkit.com/nazarhussain/60fad50131574c001bf0d784

[epoberezkin]

#1221 is not an issue to be resolved, it’s a limitation that is very unlikely to be removed.

You don’t want to just validate the schema - you want to confirm that it will compile, and the only way to do it is to compile it - it’s the same as compilers work - 1) analyse and validate the syntax (that’s what validateSchema does), 2) process semantics and convert it into the executable code. The first cannot do what the second does, nor it can guarantee that the second would succeed.

So, you need to do two things:

1. confirm that the schema is valid against the meta schema - it requires the ajv instance with meta schema added to it (and this meta schema would be compiled). This instance is best reused because meta-schema compilation is not too cheap.
2. compile the schema - that does not require meta-schema and can be done with another ajv instance, that could be disposed of completely after each compilation (which is ok, because without meta-schema added to it, it is quite cheap to construct).

This way you can get what you need ensuring that no memory leaks happen, because the whole ajv instance (the second one) would be garbage collected. Does it make sense?

You just need to make sure that the second instance is created with meta: false and validateSchema: false options (see the docs). Or if performance is not a concern, you could just use one ajv instance and dispose of it every time.

[nazarhussain]

@epoberezkin For now we fixed that behaviour to make sure we don’t compile (validate meta schema) more than once for any schema in the application. But if that would get difficult in future (considering extensibility of application) then will try usage that approach you suggested. To create new Ajv instance for this task so it could be garbage collected.