> The real fix requires agreement at the protocol level, across terminal emulators, shell applications, and TUI frameworks simultaneously.
Yeah, and ideally you want the backward compatibility, so we don't have to recompile the world or patch things like e.g. cat.
But yeah, the root of the problem is that a) the TUI-like application that manually drives the terminal, with cursor movement/line folding/redrawing etc. needs to know, at every single moment, the precise location of the cursor and the coordinates of every single character it outputted (to e.g. properly handle backspacing, including over the line folds: \b explicitly doesn't move to the previous line by default and very wide and old convention), and b) getting that coordination info from the terminal in quick, reliable, and side-effect free manner is impossible, so you have to guess.
Like [0] that Windows has for its console? This API has just recently finally lost to UNIX's in-line signaling, because the in-band controls can be proxied through almost anything, including literal serial line with two RX-TX wires and a common ground; the downside, of course, is that you have to build "out-of-line" signalling on your own.
If getting the current cursor position in the terminal were as easy (and as fast) as calling GetConsoleScreenBufferInfo, instead of sending "\e[6n" to the terminal and then reading input from it and looking for "\e[(\d+);(\d+)R" inside and then somehow unreading the input before that device report, yeah, that'd be nice and would allow solving a lot of problems with mere brute force. Sadly, it is not, and so most reimplementations of e.g. readline/linenoise functionality in shells and prompts (Erlang's shell went through 2 internal reimplementations just in my time using it, for example) are flying blind, hoping that their implementation of wcwidth(3) matches the terminal's one.
What I personally have in mind is something very different-a terminal ioctl which gives you an fd which speaks a completely different protocol, such as JSON-RPC - or whatever else you prefer - to avoid endless disputes over which alternative protocol to use, it could be you pass the ioctl a string saying what alternative protocol you want, and it gives you an fd if the pty master has registered support for that protocol, an error otherwise. The “other fd” could actually be a Unix domain socket. This also means the kernel code change required would be minimal - it doesn’t have to actually understand the alternative protocol, just let the pty master register a mapping of protocol name strings to listening sockets.
> This API has just recently finally lost to UNIX's in-line signaling, because the in-band controls can be proxied through almost anything, including literal serial line with two RX-TX wires and a common ground; the downside, of course, is that you have to build "out-of-line" signalling on your own.
SSH/etc is totally capable of carrying “secondary channels” - that’s how it implements X11 forwarding, port forwarding, etc - admittedly would require some code change in SSH clients/servers but the change would be modest rather than massive.
The Windows approach can’t be so easily forwarded over the network because it is defined in terms of a C language API not a wire protocol. My idea doesn’t have that issue
Could you give an outline of an example scenarios? For e.g. "print a line with a highlighted word in it" (alternative to print 'This is \e[7mhighlighted\e[m text\n'), and "print some text, move cursor to the second last word, delete it, move to the new end":
print 'This is a line of text which is being line-edited in an imaginary editor.'
if no line-wrap happened:
# move left a manually calculated amount of 17 (wcswidth?), delete
# everything to the right, reprint the rest of the line
print '\e[17D\e[Keditor.'
else:
# You need to know how much fit on the previous line, and the
# terminal's width. Let's assume 80, and that wrap happened
# after 'imagi', so 'nary' was on the next line. Move up 1, move
# right 63 (or do \r and move right 75), clear the rest of the
# screen, reprint
print '\e[A\e[63C\e[Jeditor.'
One of the problems with secondary channels is their synchronization with the main one: say you printed some text to the main fd, requested cursor position from the control fd, and then immediately printed some more text to the main fd — what cursor position will be reported?
The issue is handled both by Emacs and Acme by eschewing the terminal. Instead they use the shell (and direct command execution) for commands. So that means no terminfo/termcap, ncurses, and escape codes (but Emacs have a library to parse some ansi codes).
How often are complex scripts rendered in terminal? What is the cost to scripts that are currently rendered accurately by terminal? Are there any group of tools that operate in complex scripts?
EDIT: Without saying that I think this is worthy and cool. I am just curious about the costs and benefits of such a tool.
14 comments
> The real fix requires agreement at the protocol level, across terminal emulators, shell applications, and TUI frameworks simultaneously.
Yeah, and ideally you want the backward compatibility, so we don't have to recompile the world or patch things like e.g. cat.
But yeah, the root of the problem is that a) the TUI-like application that manually drives the terminal, with cursor movement/line folding/redrawing etc. needs to know, at every single moment, the precise location of the cursor and the coordinates of every single character it outputted (to e.g. properly handle backspacing, including over the line folds: \b explicitly doesn't move to the previous line by default and very wide and old convention), and b) getting that coordination info from the terminal in quick, reliable, and side-effect free manner is impossible, so you have to guess.
It could just be the path to a Unix domain socket in an environment variable, where that socket speaks some kind of RPC protocol
If getting the current cursor position in the terminal were as easy (and as fast) as calling GetConsoleScreenBufferInfo, instead of sending "\e[6n" to the terminal and then reading input from it and looking for "\e[(\d+);(\d+)R" inside and then somehow unreading the input before that device report, yeah, that'd be nice and would allow solving a lot of problems with mere brute force. Sadly, it is not, and so most reimplementations of e.g. readline/linenoise functionality in shells and prompts (Erlang's shell went through 2 internal reimplementations just in my time using it, for example) are flying blind, hoping that their implementation of wcwidth(3) matches the terminal's one.
[0] https://learn.microsoft.com/en-us/windows/console/console-fu...
> Like [0] that Windows has for its console?
What I personally have in mind is something very different-a terminal ioctl which gives you an fd which speaks a completely different protocol, such as JSON-RPC - or whatever else you prefer - to avoid endless disputes over which alternative protocol to use, it could be you pass the ioctl a string saying what alternative protocol you want, and it gives you an fd if the pty master has registered support for that protocol, an error otherwise. The “other fd” could actually be a Unix domain socket. This also means the kernel code change required would be minimal - it doesn’t have to actually understand the alternative protocol, just let the pty master register a mapping of protocol name strings to listening sockets.
> This API has just recently finally lost to UNIX's in-line signaling, because the in-band controls can be proxied through almost anything, including literal serial line with two RX-TX wires and a common ground; the downside, of course, is that you have to build "out-of-line" signalling on your own.
SSH/etc is totally capable of carrying “secondary channels” - that’s how it implements X11 forwarding, port forwarding, etc - admittedly would require some code change in SSH clients/servers but the change would be modest rather than massive.
The Windows approach can’t be so easily forwarded over the network because it is defined in terms of a C language API not a wire protocol. My idea doesn’t have that issue
EDIT: Without saying that I think this is worthy and cool. I am just curious about the costs and benefits of such a tool.
- Everybody just uses english text, right?
- Ok, sometimes there might some weird accents or something
- Every character is about the same width
- Well, they're all integer numbers of characters wide
- No character is taller than an english I
- Everybody writes left to right
- Everyone writes horizontally
Also https://jeremyhussell.blogspot.com/2017/11/falsehoods-progra...
EDIT: How the hell do you format lists in HN comments
It can't handle terminal window resize and the layout gets messed up
I was surprised to see node based cli work much better with resize?
Anyone knows why?