UX Improvements for Keyboard Control (Kino)

[zachallaun]

I've spent the last few days playing around with a little custom Livebook integration and have been using Kino.Control.keyboard/1 with it. It's very easy to use and generally works well, but there are a few things that keep coming up:

1. You have to click a button to enable the control, which means you can't really use it to create a keyboard-driven interface. You will always have to use the mouse to enable/disable the control.
2. You can enable multiple keyboard controls simultaneously and keyboard events will go to all enabled controls. While I can imagine a use for this, I only ever enabled multiple controls by accident.
3. Since enabling a keyboard control intercepts all shortcuts globally, it's easy to leave one enabled, scroll past it, and then be confused by none of your usual shortcuts working.

I have some thoughts on target improvements to these, but I think there may be an opportunity to address a larger issue of integration with the native keyboard bindings. Generally speaking, I think keyboard controls should be used to augment, not override, the existing bindings already available in Livebook.

Speaking personally, my optimal usage would be using j/k to navigate cells, and when I have a cell with keyboard controls focused, those bindings would be available.

There are plenty of How-To's here, but I'll have it here for now! Very curious to hear others' thoughts on this.

[jonatanklosko]

Generally speaking, I think keyboard controls should be used to augment, not override, the existing bindings already available in Livebook.

This doesn't really work, many keybindings in Livebook use just letters, so conflicts could easily occur if someone is building a game or anything that uses a couple characters. Also in the future we may add more keybindings, so new conflicts could emerge.

It would be helpful to see what are the use cases, the idea behind the keyboard control is to switch into "play mode" effectively without Liveobok in the picture :)

[josevalim]

s/exit/deactivate_only?

[jonatanklosko]

Sounds good :)

[zachallaun]

Would this be too verbose?

default_handlers_when_enabled: :off | :on | :allow_toggle

(edit: or deactivate_only instead of allow_toggle)

[jonatanklosko]

I'm fine with more verbose name, but to me :default_handlers_when_enabled is more confusing than clarifying :)

[zachallaun]

Fair enough!

[jannikbecher]

I was thinking of something similar in the past.
It would be nice to introduce something similar to emacs major/minor modes. So for example the major mode would be the global shortcuts for livebook and then depending on which cell is focused a minor mode is added and removed when the cell is not focused anymore.
I think with hotkeys-js this should not be that difficult to implement.
I can try to implement a PoC when I'm back :)

[jonatanklosko]

We already have two keyboard modes (navigation and insert), I don't think we should add more to avoid confusion :)

[zachallaun]

(Starting a new reply thread to keep the other thread focused on the key binding stuff.)

Another issue that I've run up against when a keyboard control is active, especially when default handlers are disabled (the default), is that it's fairly easy to accidentally leave one enabled, scroll somewhere else in the Livebook, and then be confused when no keybinding are working because a control off-screen is canceling all events. Depending on the length of the notebook, it can be difficult to find the active control to disable it.

It would be really nice to have some UI indicator that a keyboard control is active even when the cell is off screen. One option would be to make the keyboard control button sticky in some way -- when you scroll past it, it could stick to the gutter or something such that it's always visible.

Any thoughts?

[josevalim]

I can’t see a reason to enable more than one so maybe when we enable one, all others are disabled?

[zachallaun]

I agree and would enjoy that change, but I don't think it addresses the issue above? Imagine you've enabled a keyboard control, then scrolled very far down, then you try to use a global navigation shortcut and it doesn't work -- you've left a keyboard control that disables default handlers enabled, so now you have to scroll until you can find and disable it.

[josevalim]

A keyboard indicator would take too much "screen state", so I would try smaller QoL improvements first. :)

[zachallaun]

Agreed that other QOL changes come first. :)