Touch input API proposal

Inspired by:

https://developer.mozilla.org/en-US/docs/Web/API/Touch_events https://developer.mozilla.org/en-US/docs/Web/API/Touch https://developer.mozilla.org/en-US/docs/Web/API/Touch_events/Using_Touch_Events http://who-t.blogspot.com/2012/01/multitouch-in-x-touch-grab-handling.html https://docs.microsoft.com/en-us/xamarin/xamarin-forms/app-fundamentals/gestures/

Touch events

struct Touch
{
    long Id;
    Point Position;
}


class TouchEventArgs : RoutedEventArgs
{
   int SequenceId;
   TouchEventType Type;
   ulong Timestamp;
   List<Touches> Touches;
   List<Touches> AddedTouches;
   List<Touches> ChangedTouches;
   List<Touches> RemovedTouches;

   IAcceptedTouchSequence AcceptTouches(IControl control);
}

TouchCancelledEventArgs : TouchEventArgs
{
   IControl AcceptorControl;
}

TouchesBegan(TouchEventArgs args) event

Triggered when the first touch point is activated

TouchesChanged(TouchEventArgs args) event

Triggered when a touch point is moved or when new touch point is added or when a touch point is removed but there is at least one left.

TouchesEnded(TouchEventArgs args) event

Triggered when the last touch point is removed. Both Touches and ChangedTouches are Empty.

TouchesCancelled(TouchCancelledEventArgs args) event

Triggered when touch sequence is cancelled due to platform-specific reasons or when touch sequence is accepted by another control.

Event flow

Touch events are triggered in a sequences. A sequence begins when first touch point is activated and bounds itself to the control determined by the hittest of the first touch point’s position. TouchesChanged and TouchesEnded events are sent with that control being event target. It’s guaranteed that there is only one active touch sequence

Events follow the normal tunnel/bubble flow until some control accepts the sequence. After a control has accepted the sequence touch events from that sequence will only be sent to that control. Target control will be sent a TouchesCancelled event sent through the tunnel/bubble flow, ignoring the acceptor control.

This way we can have touch gesture recognition hierarchies when at the start every control in the tree can react to touch events, but once gesture is recognized events will only flow to the recognizer.

Gesture recognition

Each control will have AvaloniaList<IGestureRecognizer> GestureRecognizers property, default implementations of touch event handlers will delegate their work to gesture recognizers associated with the control. The end user API is similar to Xamarin.Forms.

Gesture recognizer can accept or reject the current touch sequence. When gesture recognizer rejects the sequence it won’t be sent any further events from that sequence. If recognizer accepts the sequence, the control will accept the sequence and send events to acceptor recognizer. Recognizers can register themselves to run on tunnel, bubble or both stages of the event flow.

Example gesture recognizer:

class TapGestureRecognizer : IGestureRecognizer
{
    ulong _started;
    Point _startPoint;
    const double Distance = 20;
    const ulong MaxTapDuration = 500;
    GestureRecognizerResult.IGestureRecognizer Handle(TouchEventArgs args)
    {
       // Multi-touch sequence
       if(args.Touches.Count > 1)
           return GestureRecognizerResult.Reject;
       // Sequence started, save the start time
       if(args.Type == TouchEventType.TouchesBegan)
       {
           _started = args.Timestamp;
           _startPoint = ev.Touches[0].Position;
           return GestureRecognizerResult.Continue;
       }
       if(args.Type == TouchEventType.TouchesEnded)
       {
           var endPoint = args.RemovedTouches[0];
           if(Math.Abs(endPoint.X - _startPoint.X) < Distance
               && Math.Abs(endPoint.Y - _startPoint.Y) < Distance
               && (args.Timestamp - _started) < MaxTapDuration)
           {
               args.Source.RaiseEvent(new RoutedEventArgs(TappedEvent));
               return GestureRecognizerResult.Accept;
           }
           else
               return GestureRecognizerResult.Reject;
       }
       return GestureRecognizerResult.Continue;
    }
}

Touch event emulation

InputManager.EnableTouchEmulationViaMouse will enable touch emulation, so at least one-finger touch support could be developed and debugged on touch-incapable machines. Mouse pointer would behave like a single touch point when pressed.

Cancelling an accepted touch session

(this might not be required, we need to try other things before implementing)

Gesture recognizer (or control) can reject already accepted grab. In this case touch events that happened after the grab would be replayed.

interface IAcceptedTouchSequence 
{
     void Cancel();
}

“Common” events

We are currently converting unhandled pointer events to Tapped and DoubleTapped here: https://github.com/AvaloniaUI/Avalonia/blob/master/src/Avalonia.Input/Gestures.cs#L29

Same would happen in touch gesture recognizer. We would place TapGestureRecognizer to TopLevel’s GestureRecognizers collection by default.

[mstr2]

If I understand you correctly, there can only be exactly one active touch sequence. This may be very limiting, since it will not allow for more advanced multi-touch interactions. For example, on the home screen in iOS, you can grab an app icon with one finger (assuming that you’re in wiggle mode), and then use a second finger to scroll to another home screen page while you are holding onto the app icon.

Another example is the UIDatePicker control in iOS, which allows you to scroll the individual spinners simultaneously. There are several more examples of such interactions, so it’s not a fringe feature (at least in iOS).

[kekekeks]

We’ve adopted UWP’s model. See https://github.com/AvaloniaUI/Avalonia/blob/c00a1d6331eca132ffa851fd0a521162d4d91d7f/src/Avalonia.Input/GestureRecognizers/ScrollGestureRecognizer.cs for example recognizer implementation.