Dear beta testers,

I’ve just shipped the latest version of Terminal Click: TC v0.8.5. Recall that we are re-examining the basics to get us out of the beta and become a “fully grown” Terminal Click.

Unicode Rendering

Terminal Click can now render extended latin, nerd fonts, emojis and CJK. Here’s a demo showing off some CJK glyphs:

As usual, whatever works on macOS will work on Windows and Linux too!

Friendly UTF-8 “Debate”

This is my first time taking on Unicode, which is particularly nasty with terminals, and there’s still future work for me to do. I really wanted an honest comparison against an expert so I asked a fellow Handmade author to react to my initial pass on it.

I got permission to reproduce our discussion:

Abner: Hey! I wanted to pick your brain about my Unicode rendering setup since you’ve been through the trenches with UTF-8 in your editor. Just general feedback is helpful. In particular I’m interested in the idea of fallback chains (assuming you’ve implemented it!)

My approach is to pre-rasterize glyphs into texture atlases. Each font gets its own atlas that’s packed with stb_rect_pack, and the atlas grows in power-of-two increments until everything fits. I decode UTF-8 codepoints at draw time, look up each glyph in an atlas via binary search, and then batch textured quads to the GPU.

For the fallback chain I have a primary user font, and then a series of fallback fonts: a Nerd Font for icons, an emoji font, and then CJK fonts for Simplified Chinese, Japanese, and Korean. When a codepoint isn’t found in the user font, I walk the chain in order until I get a hit. If nothing has it, I fall back to a replacement glyph (U+FFFD or '?'). The fallback atlases are loaded lazily… they only get rasterized and uploaded the first time a codepoint actually needs them. The upside is that startup is fast and memory stays low if you never hit CJK or emoji. The downside is there’s a one-time hitch when a fallback atlas first loads, and if the user changes font or size, I have to tear down and invalidate every atlas in the chain.

This is just my first pass at Unicode rendering so I know there’s room to improve. I’m curious how you handled the glyph coverage problem in your editor. Did you also go with a multi-atlas fallback approach, or did you do something entirely different? Especially interested in how you dealt with the sheer volume of CJK codepoints and whether you found a way to avoid rasterizing huge atlases upfront.

Starfreakclone: That approach isn’t all too different from how unicode glyphs work in fred. In fred I have one big atlas for fonts of a particular style. This is so I can easily batch at rendering time and it simplifies the surface area for a search.

I rasterize all the ASCII glyphs right away in to an O(1) slot lookup table because they’re by far the most common case.

For unicode, I have a lookup table that has the unicode glyph information. Since the glyphs themselves are already unique, I don’t need to hash anything. Then as unicode glyphs are requested I perform a lookup into the table to find the entry. If one doesn’t exist I load the fallback fonts (if not loaded). The fallback fonts are loaded in basically the same way you describe, so there’s a hitch if they’re not already loaded. Now, where we differ is that once I load the fallback font, I keep the font handle around in memory so I can look it up again later if the font sizes change. Once the fallback fonts are in place, I perform a lookup into the fonts to see if there’s a matching glyph for the codepoint. Once a glyph is found, it populates the map entry and returns the glyph information to the draw list.

The major difference between our approaches, it sounds like, is that you seem to allocate an atlas for each font regardless of if it will ever be used while fred selects individual glyphs from foreign fonts and rasterizes them directly into the main atlas.

Abner: Yes, that is our fundamental difference, thank you! I’ll chew on that. However I’ll immediately adopt this: “I rasterize all the ASCII glyphs right away in to an O(1) slot lookup table because they’re by far the most common case.” That’s basically having a glyph cache for anything below 128. It’s a little silly for me to binary search for the common case. We’re rendering English characters almost all of the time.

Starfreakclone: *thumbs up*

As we can see, my first impulse was to simplify development… which means users pay a price right now. Firstly, fully rasterizing font atlases bloats RAM unnecessarily. Second, this “one-time hitch” is unforgivable for terminal emulators. When I showed the hitch to a local community member he made me stress-test UTF-8 characters inside other mainstream terminals.

That was humbling.

Lastly, I’m concerned by our new download sizes. We went from 5MB downloads (which I already found appalling) all the way to 20MB+ because of these so-called “fallback fonts.” I’m definitely having second thoughts about bundling so many fonts.

Artist Feedback

An artist friend who goes by Zardium online responded to one of last month’s features: distance-based color interpolation. Check out the demo clip in case you forgot:

The TL;DR was basically “looks cool, but not practical.” Below is our discussion with permission to reproduce:

Zardium: I’ve got a UX papercut for you!

The interpolation of the path UI’s selection state might seem like a good idea but I’d point out that it’s actually obscuring intent.

“Will this element be interacted with when I click?” is a binary state, and your signifier for this information is the selection color.

But by interpolating the color, you’re muddying the information being presented. It’s no longer “is it light or dark” but “is it the very lightest color, or any other color, including colors that are very similar?"

You almost certainly want to keep it in two distinct colors that signify whether the path part will be retained, and save any in-between colors for a quick animation.

My other recommendation is that since paths are an inherently stacked concept and selecting a folder implies that all of its parents are included as well, the leftmost path parts before the hover should also be drawn as if they are part of the path.

You can change the metaphor to “light=will be part of the new path when I click” and “dark=won’t be part”

Abner: Excellent feedback! I agree wholeheartedly with this. I knew I needed a visual indicator now that folders are individually clickable, but had no design language for the best approach.

Zardium: Another thing is to try and have multiple signifiers if possible, and the important thing is that they’re of different types. So you have a color signifier which is good, but a spatial signifier like an underline or box could be very helpful as well.

It’s important that it’s not just changing the background color as well, which is common in TUIs. That’s just adding to the already existing color signifier which might get lost if someone has impaired color vision for instance.

Hopefully I’ll make these changes in time for the new builds this summer!

Pending Contributions to Terminal Click

Last month we announced the onboarding of contributors to TC. This is true and I’m excited to reveal more, but I’m waiting for our new volunteer to finish up a neat little blog post discussing the whole experience so far.

Let’s remember that contributors are willingly donating their time.

Next TC Release: Implement Missing Escape Codes

Any power user spending a lot of time on Vim, Emacs, ssh, or any interactive TUI will encounter glitchy behavior that turns into a nuisance. This is (largely) due to missing ANSI escape codes which I haven’t gotten around to supporting properly.

This is our next priority, alongside the UTF-8 bugs that get reported.

Until May,
Abner