feat: allow searching for similar images by providing a path or URL to an image

Sometimes it’s more convenient to search for “a picture similar to the one I have in mind”, instead of trying to describe it with words. Especially considering the extremely short phrases CLIP allows.

I’d like to be able to run:

 rclip https://i.redd.it/a7jf5rlghsz51.jpg

to return pictures of similar buildings like these.

It would also be nice to be able to search for pictures similar to a local file, like rclip /tmp/turtle.jpg and/or rclip file:///tmp/turtle.jpg.

This branch has a working work-in-progress for both of those features; but probably isn’t ready for a pull request yet because it has a python3.9 dependency (str.removeprefix is a 3.9 feature). If you’re interested, I’ll remove that dependency and open a pull request.

[yurijmikhalevich]

Hi @ramayer,

I like the idea of having a simple parser that treats everything that starts with https://, http://, file://, /, X:\, or ./ as a file path.

I would love to keep the number of parameters to a minimum unless we really need to add them. I can’t imagine a situation when you’ll want to include these prefixes in your regular text query. What do you think?

Let’s start by implementing a simple parser. If we will encounter any issues in the future, we will adjust accordingly 😃

[yurijmikhalevich]

I like the option to search for similar images.

This will affect optimizing querying by splitting the model into text and vision transformers, but we may only load the vision transformer when the user needs to search for similar images.

I also like an option to search for rclip /image.png -a night 😃

Let’s merge the feature for adding and subtracting queries first.

I left a few comments here: https://github.com/ramayer/rclip/commit/ea09b083cf80197d027426f6ee7177c93a6bd19a