Question about changing model in system and real-time issue

Hi @juanmc2005 ! it’s me again, sorry for bothering you… I have several questions

1. I have already fine-tuned a segmentation model with other dataset (not AIM, VoxConverse) and store in “.ckpt” model. I sure that it can be loaded by using ckpt_path = "path/where/store/ckpt/epoch=9_best.ckpt" Model.from_pretrained(ckpt_path, strict=False) i want to put replace this model in your system for some testing. But when i change here by segmentation: PipelineModel = Model.from_pretrained('path/where/store/ckpt/epoch=9_best.ckpt'). When using “python -m diart.stream”, the system is still using “pyannote/segmentation”. could you give me some hints?
2. Is there any useful augmentation method for training or fine-tuning? I know that can add back noise or something. Where should i add them?
3. this system can use for real-time right (streaming)? If yes, how to confirm “real-time”? like real-time factor or something else?

Thanks for your awesome project, it helps me a lot!! Looking forward to your reply~~ Best Regards

[juanmc2005]

In version 0.3 it defaults to GPU yes, both in the python API and diart.stream/diart.inference. Adding --cpu when you run the script forces CPU usage.

[juanmc2005]

I’m glad it’s clear now 😃 At this point you should probably be able to understand quite a bit of diart’s code, so if you ever think of some interesting feature or you find a bug, any contribution/pull request is more than welcome!

But why not said that the system can beat the official offline baseline when setting the latency as 4s or 5s? Are there any considerations?

That is also shown, although not specifically written. I guess the takeaway from that sentence is that we succeed in obtaining the same performance with much lower requirements: low system latency, very low real-time latency (165ms on CPU), online processing, and with continual improvement.