Add `!`-prefix parsing to glob util

It’d be potentially useful to have ! as a prefix added to our glob.ts utils. This could function similarly to how we already handle .gitignore files in the webworker search here:

1. if a file matches a normal line and doesn’t match an !-line, exclude it (we do this today)
2. If it matches both a normal line and a !-line, include it
3. Otherwise, include it (we do this today)

This isn’t quite as powerful as a real.gitignore as there are only two “layers” of include/exclude, whereas in gitignore each line introduces a new “layer”, but I believe it would handle the 99% of cases well enough, for instance every example given to-date in https://github.com/microsoft/vscode/issues/869.

To handle the many-layer case requires either:

• migrating to an array to configure files.exclude and search.exclude, which is undesirable from a backwards/forwards compatibility perspective
• taking care to ensure the existing *.exclude objects are handled in a key-order sensitive way, which is potentially undesirable from a JSON/JS object semantics compatibility perspective

cc @bpasero for any and all input on this

[JacksonKearl]

If I understand correctly, users would configure files.exclude with the new support for negated patterns?

Yes exactly.

I guess, but it would probably be worth looking at ripgrep or git source before we get bit by edge cases

We mostly implement their logic here: https://github.com/microsoft/vscode/blob/2358283d76a0afdf57e60ec2fc76a57b1b0146c7/src/vs/workbench/services/search/common/ignoreFile.ts#L90 but it’d require some significant changes to get that working for excludes, for instance handling filtering specifically directories vs files and following parent pointers differently from how we merge the settings today. Technically we may need to handle ordering as well, but the code above doesn’t, and that wouldn’t really work with our object based model anyways.

Ideally, the following would work:

"files.exclude": {
    "node_modules/*": true,
    "!node_modules/@types": true,
    "node_modules/@types/*": true,
    "!node_modules/@types/*.js": true,
}

However that requires some understanding of the ordering that we may not want to rely on, especially given things like settings scope merging.

Absent that, it’d be helpful to have this:

"files.exclude": {
    "node_modules/*": true,
    "!node_modules/@types": true,
}

where if a file matches some bare expression and no negated expression, it matches the glob and is included. This way ordering isn’t important, a simple 2 pass approach works. This is actually what we do already in the ignore file parser.

[bpasero]

It is somewhat easy to add global negate support for glob, see https://github.com/microsoft/vscode/commit/efa5122689eea38e5c8ca4480f58ae58e97230e6 for a first stab at it.

However, there are a few questions:

• splitGlobAware probably needs to change to account for the leading ! (e.g. change the return type to { segments: string[]; negated: boolean; }) [1]
• how to properly handle glob expressions that are not just a simple pattern [2]

[1]

Not sure the ripples or adoption cost where it is used, @roblourens in search. Does RipGrep even support these patterns?

[2]

Take this example:

let globExpression = {
	'!*.js': true,
	'!*.ts': true
};

Since glob expressions are basically a OR combination of all patterns, the above pattern will match on everything, even though it may look as if this will match every file that is not .js or .ts. Not sure how realistic that scenario is though, but wanted to bring it up anyway.