Make audio stream available when muted

What are you trying to do?

When an attendee starts talking while muted I want to show a message saying “You are muted, please unmute yourself”. This means I want to “watch” the audiostream of the microphone when the attendee is muted and when it reaches a certain threshold (so it detects ‘speaking’) show the message.

What documentation have you looked at so far?

createAnalyserNodeForAudioInput() I have tried using the meetingSession.audioVideo.createAnalyserNodeForAudioInput(). The data is fine when the attendee is not muted, but from the moment the attendee is muted the data is always 0. Is there a different way to get the captured sound?

Any other documentation I couldn’t find anything in the documentation that is related to getting the audio (stream) other than the createAnalyserNodeForAudioInput().

[simmkyu]

When you mute the microphone using realtimeMuteLocalAudio(), SDK simply disables the media stream of the audio input device. createAnalyserNodeForAudioInput() uses this disabled stream, so it won’t help.

---

As a workaround, you might get the microphone’s stream yourself and create an AnalyserNode on it.

While in the meeting, try running this snippet in the browser console. Make sure to replace __your_device_id__ with the device ID of your muted microphone. You’ll see the non-zero percentage in the console while you speak.

(async function() {
  const stream = await navigator.mediaDevices.getUserMedia({
    audio: { deviceId: '__your_device_id__' }
  });

  // Some browsers have an implementation-defined maximum number of allowed AudioContext.
  // Try reusing the single AudioContext object in production.
  const audioContext = new (window.AudioContext || window.webkitAudioContext)();
  const analyserNode = audioContext.createAnalyser();
  audioContext.createMediaStreamSource(stream).connect(analyserNode);

  const data = new Uint8Array(analyserNode.fftSize);
  let frameIndex = 0;
  const analyserNodeCallback = () => {
    if (frameIndex === 0) {
      analyserNode.getByteTimeDomainData(data);
      const lowest = 0.01;
      let max = lowest;
      for (const f of data) {
        max = Math.max(max, (f - 128) / 128);
      }
      let normalized = (Math.log(lowest) - Math.log(max)) / Math.log(lowest);
      let percent = Math.min(Math.max(normalized * 100, 0), 100);
      console.log(percent); // <-- Use this value to detect "speaking"
    }
    frameIndex = (frameIndex + 1) % 2;
    requestAnimationFrame(analyserNodeCallback);
  };
  requestAnimationFrame(analyserNodeCallback);
})();

I used the startAudioPreview() in the SDK demo. I haven’t tested this code thoroughly, but hope you can get some ideas.

[blundercode]

Workaround works great still great job.