SSML bookmarks broken when using the <lang> tag with multilingual voice

Describe the bug When synthesizing SSML with the voice en-US-JennyMultilingualNeural using the .NET SDK, the bookmark events are fired with unusable data or even not at all.

To Reproduce

Synthesizing sample 1…

<speak xmlns="http://www.w3.org/2001/10/synthesis" version="1.0" xml:lang="en-US">
    <voice name='en-US-JennyMultilingualNeural' xml:lang="en-US">
        <bookmark mark="One"/>
        <lang xml:lang="en-US"> A </lang>
        <bookmark mark="Two"/>
        <lang xml:lang="en-US"> B </lang>
        <bookmark mark="Three"/>
        <lang xml:lang="en-US"> C </lang>
        <bookmark mark="Four"/>
    </voice>
</speak>

…returns…

Synthesis started: SynthesizingAudioStarted
Bookmark reached at 0: 児ࡓ网
Bookmark reached at 0: 樐࡜网
Bookmark reached at 0: ࡕ网
Bookmark reached at 0: Four
Synthesis completed

…with the first 3 bookmark names being different characters each time you try it.

Synthesizing sample 2…

<speak xmlns="http://www.w3.org/2001/10/synthesis" version="1.0" xml:lang="en-US">
    <voice name='en-US-JennyMultilingualNeural' xml:lang="en-US">
        <lang xml:lang="en-US">
            <bookmark mark="One"/>
            A
            <bookmark mark="Two"/>
            B
            <bookmark mark="Three"/>
            C
            <bookmark mark="Four"/>
        </lang>
    </voice>
</speak>

…returns no bookmarks at all:

Synthesis started: SynthesizingAudioStarted
Synthesis completed

Expected behavior

Synthesizing the working sample…

<speak xmlns="http://www.w3.org/2001/10/synthesis" version="1.0" xml:lang="en-US">
    <voice name='en-US-JennyMultilingualNeural' xml:lang="en-US">
        <prosody rate="+15%">
            <bookmark mark="One"/>
            A
            <bookmark mark="Two"/>
            B
            <bookmark mark="Three"/>
            C
            <bookmark mark="Four"/>
        </prosody>
    </voice>
</speak>

…returns…

Bookmark reached at 0: One
Bookmark reached at 4250000: Two
Bookmark reached at 10500000: Three
Bookmark reached at 17500000: Four

…as expected. Adding prosody or phoneme tags does work as expected, as well.

Version of the Cognitive Services Speech SDK 1.24.2

Platform, Operating System, and Programming Language

• OS: Windows 11
• Hardware: x64
• Programming language: C#

Please let me know if providing the demo application we used for testing would help. Thanks!

[pankopon]

Presuming the same internal work item ref. 4681629 as in another case - if different then @jiajzhan please update.

[jiajzhan]

Thanks for reporting this issue. I will investigate this issue.