WebRTC to .NetCore API, GoogleSpeechApi return nothing

Hi,

I send my webrtc audio to my .netcore api.

Angular TypeScript

record() {
    this.isRecording = true;

    navigator.mediaDevices.getUserMedia({
      audio: true
    })
      .then((stream) => {
        var options = {
          audioBitsPerSecond: 128000,
          mimeType: 'audio/webm\;codecs=opus'
        }

        if (!MediaRecorder.isTypeSupported(options.mimeType)) options.mimeType = "audio/ogg;codecs=opus";

        this.stream = stream;
        this.mediaRecorder = new MediaRecorder(stream, options);
        this.mediaRecorder.onstop = e => {
          console.log("stop recording => OK");
          const blob = new Blob(this.chunks); 
          this.chunks.length = 0;

          console.log("blob => OK");

          var reader = new FileReader();
          reader.readAsDataURL(blob)
          reader.onloadend = () => {
            this.sampleService.sendFile(<FileDto>{
              name: 'record.webm',
              content: reader.result.split(',')[1], //because first is data:;base64 and i dont want this in my content
              contentLength: 0,
              contentType: options.mimeType
            })
              .subscribe(result => {
                console.log("sendFile=> OK");
              }, error => {
                console.error(error);
              });
          }
        }
        this.mediaRecorder.ondataavailable = e => {
          this.chunks.push(e.data);
        }
        this.mediaRecorder.start();
      });
  }

  stop() {
    this.isRecording = false;

    this.stream.getAudioTracks().forEach(track => track.stop());
    this.stream.getVideoTracks().forEach(track => track.stop());

    this.mediaRecorder.stop();
  }

C#

        [HttpPost]
        [ProducesResponseType(200)]
        public async Task<IActionResult> SendFile([FromBody] FileDto file)
        {
                       var byteContent = Convert.FromBase64String(file.Content);

            var speech = await SpeechClient.CreateAsync();
            var response = await speech.RecognizeAsync(new RecognitionConfig()
            {
                Encoding = RecognitionConfig.Types.AudioEncoding.OggOpus,
                SampleRateHertz = 48000,
                LanguageCode = "en-CA"
            }, RecognitionAudio.FromBytes(byteContent));
            foreach (var result in response.Results)
            {
                foreach (var alternative in result.Alternatives)
                {
                    Console.WriteLine(alternative.Transcript);
                }
            }

            return Ok();
       }

How did I know about the good SampleRateHertz? I copy the byteContent to a localFile:

            var path = "C:\\Users\\Cedric\\Downloads\\Temp\\" + Guid.NewGuid().ToString();
            if (file.ContentType.Contains("webm"))
            {
                path += ".webm";
            }
            else
            {
                path += ".ogg";
            }
            System.IO.File.WriteAllBytes(path, byteContent);

First I played it to verify that I can listen my voice, and It works! Then I open the details tab property in the file properties and I saw 48 000 kHz

How did I know that Encoding is OggOpus? When I created the mediaRecorder I specified in the option : mimeType: ‘audio/webm;codecs=opus’

It was pain in the ass to understand that because I had no knowledge about Audio in general so for those who dont know that too, It could be helpfull.

Of course, The Google Environnment Variable is set in the LaunchSettings, I added in the environmentVariables section: “GOOGLE_APPLICATION_CREDENTIALS”: “C:\MYPATH\XXX-202418-645374f15cb1.json”

About the issue I have. on the audio recorded from WebRtc, I can cleary hear “This is a test”. But I always have no answer from Google Speech, the Api doesnt failed, it just returns nothing.

Did I miss something? How can I make it work?

[jskeet]

Hooray! Very glad to hear that 😃

[jskeet]

As per your screenshot, that’s now Ogg Vorbis, not Ogg Opus. Ogg isn’t the audio format - that’s the container format. Opus is the audio format, and Google Cloud Speech supports Opus within an Ogg container.