Using AI to Sync Song Lyrics

In this video, we explore using various Python AI models to sync song lyrics. We cover Spleeter by Deezer, Whisper by OpenAI and text similarity techniques, to create a synced lyric file (LRC file) which can be read by a variety of medium programs to display synced lyrics. The end result is sometimes better than what's available on Musixmatch today, which ends up on Spotify.
Try it out for yourself:
github.com/rye...
Update on Music:
Unfortunately, despite the fact that the music in the video was designated royalty free at the time of upload, RYYZN has started claiming the song. As such, I have unfortunately muted the demo portion. If you want to try it out for yourself, please download the library from GitHub.

[@RahulSingh-fl6bf]

This. Is. So. Cool. I would love to learn more and see more videos about this and an explanation! Any links for further info and learning?

[@KOSEIUS]

I got this
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.

[@tvrrp]

And what about karaoke-style synchronization? I think Apple did something similar. Previously they used musixmatch data synced by the community, but word by word sync in musixmatch is impossible to create (because of the awful editor interface)

[@syntaxbyte]

The difficulty there is getting word boundaries. But yes, if you had word boundaries as an output of the speech to text library, it could be done.

[@monsieur2D]

this video doesn't show how to use this tool. Care to explain?

[@ClaireCJS]

you should turn on the karaoke option in minilyrics ;)

[@samijohn207]

hey Ryan, does this also work for songs in other languages?

[@samijohn207]

and how did you use spleeter by deezer please?

[@travelmindanao5755]

haha is this a talk show?

[@emiliomola6479]

Hello, I write beats and full on songs (no vocals or lyrics). I've heard some songs that are completely generated by AI. Is it possible for the AI to sing to my songs? Where would I find the application(s) to do this? It looks like that's what you are doing here? Thank you for your assistance.

[@syntaxbyte]

Hi, that’s not what’s being done here. This is just generating time stamps on lyrics in existing songs. I’m not sure of AI audio generators for vocals, it’s something you’d have to Google and do more research into.

[@lildracer]

maybe try instrumental first then getting the rest.....

[@waktuluang98]

Hi Ryan :)
how to write the code if song and lyrics have name "sample" ?
where should i put the file?
python pysync [-h] [--output_file OUTPUT_FILE] music lyrics

[@syntaxbyte]

I assume they have file extensions, they must be two different files. You need a text file with the lyrics and an audio file with the song. Then you would do python pysync sample.mp3 sample.txt

[@zekiz774]

Enable file extensions in the file explorer settings

[@kanz_alzikrayat]

Is that Tom Holland?!