Neat. I was just looking for something like this for auto gating etc, for mixing podcast audio.
Also phoneme level accuracy from something like this would be useful for realtime lipsync animation, and tools support pronunciation learning.
I also recently looked into stuff like this for studying rap music across wildly different languages and styles and time periods to see what kinds of patterns emerge. Niche, but maybe one day we will get there.
How does this compare to using a model like facebook/wav2vec2-base960h for scoring at a 20ms per frame, and then something like torchaudio.functional.forced_align to map those frames onto the transcript words?
I get pretty accurate results this way, along with some nudging heuristics based on the expected leading or trailing word sibilance.
For multilingual work, facebook/mms-1b-all (trained on ~1100 languages) works extremely well, but is significantly slower on my CPU.
At a certain point, blaming tools like AI for fake news and misleading information is like blaming electricity for it. Anybody left on Earth today who takes every video to be an authentic and faithful representation of a reality that actually happened should really get out from under the rock they’ve been under for the last four years.
Yes, it is easier than ever to make fake content. No, no matter what the mainstream does, bad actors will attempt to fool people with this tech. The cat has been out of the bag for a long time.
I get pretty accurate results this way, along with some nudging heuristics based on the expected leading or trailing word sibilance.
For multilingual work, facebook/mms-1b-all (trained on ~1100 languages) works extremely well, but is significantly slower on my CPU.
How's the performance across different languages?
Yes, it is easier than ever to make fake content. No, no matter what the mainstream does, bad actors will attempt to fool people with this tech. The cat has been out of the bag for a long time.