Can't sync midi file notes with audio file

Hi, first of all, good job for this library and the support you always give to the users.

I’m new to unity and I’m having fun trying to make a guitar hero style game where there are notes spawned in lines and you have to press the correct button at the correct moment. To do it, I’m using midi files to have the map of the notes and an audio file to play the real song above the notes. The audio file is a simple mp3 file so it doesn’t contains any information of the notes.

Following the issue #79 I managed to spawn and visualize in sync notes in my game, but when I tried to add the real audio file to the game, I notice that after a few seconds, the visualized notes and the audio get out of sync and I can’t understand why.

With my little knowledge, I would expect that the midi file notes would map perfectly the audio file notes, but this isn’t the case… So I tried to fix this issue by matching the midi file position to the audio source file position but I still can’t get my game in sync.

This is the piece of code I wrote:

    # To delay the start of the music by X seconds
    [SerializeField] private float startAudioSourceDelayInSeconds = 4;

    # To delay the start of the playback by X seconds (it can be different from the audio delay)
    [SerializeField] private float startPlaybackDelayInSeconds = 5;

    private float dspSongTime;

    # I'm basing the sync with the unity audio system to be more precise
    void Awake() {
        dspSongTime = (float) AudioSettings.dspTime + startAudioSourceDelayInSeconds;
        audioSource.PlayScheduled(dspSongTime);
    }

    # I'm using to "MoveToTime" function to move the midi file to the audio source position each frame
    private void Update() {
        long positionInMicroseconds = (long)((AudioSettings.dspTime - dspSongTime + startPlaybackDelayInSeconds) * 1000000); // Convert seconds to microseconds
        var positionInTicks = new MetricTimeSpan(positionInMicroseconds);
        _playback.MoveToTime(positionInTicks);
    }

I would also share how I’m transposing the notes from top to bottom:

    # The initialPositionY is where the note is spawned
    private void Update() {
        float yPosition = initialPositionY - _playback.GetCurrentTime<MetricTimeSpan>().TotalMicroseconds / 100000.0f;
        transform.position = new Vector3(transform.position.x, yPosition, transform.position.z);
    }

Have you any idea on how I could reach my goal?

[DanieleCali]

Just to update this question. I’ve downloaded waveform free as DAW and manually changed the tempo of both midi and audio file for each bar in order to get them in sync. Now it is also working on my game. Thanks again for the support, have a good day 😄

[DanieleCali]

So, you suggest to work on a DAW to sync midi and song file ok. I just have to study how to do it 😄 Thanks again with the support!