question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Play audio on gaze

See original GitHub issue

I’m looking to have some my object AudioSource components start playing when they first gazed at (I actually need to figure out something better, because I need to place them in the spatial mesh first, then, later on, have the audio start, but first I need to fix this issue!).

How do I do a raycast from the camera (head position) to the various objects in the scene (that already have colliders available, as they use TapToPlace)

I have tried the following, but the audio starts right away (audio source property “play on awake” is off)

public class PlaySoundOnGaze : MonoBehaviour
{
    protected SpatialMappingManager spatialMappingManager;

    private bool isAudioPlaying = false;

    void Start()
    {
        spatialMappingManager = SpatialMappingManager.Instance;
    }

    void Update()
    {
        if (!isAudioPlaying)
        {
            Vector3 headPosition = Camera.main.transform.position;
            Vector3 gazeDirection = Camera.main.transform.forward;

            RaycastHit hitInfo;
            if (Physics.Raycast(headPosition, gazeDirection, out hitInfo, 30.0f, spatialMappingManager.LayerMask))
            {
                Debug.Log("Playing audio...");
                GameObject child = gameObject.transform.GetChild(0).gameObject;
                child.GetComponent<AudioSource>().Play();
                Debug.Log("is playing? " + child.GetComponent<AudioSource>().isPlaying);
                isAudioPlaying = true;
            }
        }
    }
}

I think the problem is that I’m raycasting into the spatial mesh? The raycast code was lifted out of the TapToPlace script.

How do I make it so the cursor has to be on the object for the raycast to be true?

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
jessemccullochcommented, Apr 29, 2017

I second @killerantz suggestion to use the input manager, and implement the IFocusable interface and use that for your events. This is the recommended way to do all input management at this point (Click, Focus, Manipulation, Navigation, etc)

2reactions
killerantzcommented, Apr 28, 2017

Will he input system and gaze manager work? Create a behavior that implements the IFocusable interface and add the functions OnFocusEnter() and OnFocusExit(). Inside the OnFocusEnter() is where the audio should start playing. The gaze manager will handle the raycasting and hit detection, thought it will stop at the parent collider.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Play audio on gaze - microsoft/MixedRealityToolkit-Unity
I'm looking to have some my object AudioSource components start playing when they first gazed at (I actually need to figure out something ......
Read more >
Eye Gaze Technology and Music
Much like selecting phrases for speech devices, users can select instruments and the notes they play to then build their own song from...
Read more >
Gaze controlled keyboard with Python and Opencv p.9
Play sounds – Gaze controlled keyboard with Python and Opencv p.9 ... We start doing this by installing a new library to play...
Read more >
ii-Music for Eye Gaze
ii-Music software for eye gaze operates using cursor position and “dwell to click” meaning that you can use it with almost any eye...
Read more >
Audio-augmented museum experiences with gaze tracking
In this work, we enrich landscape and genre paintings by spatializing sounds for the drawn objects and scenes, which expands visitors' ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found