For those interested, this guy is revamping the Emacs widget library with something more modern and platform agnostic, based on SDL: https://appetrosyan.github.io/posts/
Interesting, thanks for sharing! I've had thoughts about making vui.el backend-agnostic so it could target different widget implementations (like xwidgets or even native-GUI). An SDL-based widget library could potentially be one of those backends. Need to dig into appetrosyan's work before I can say anything intelligent about it though. And of course, it was an idea and I am unlikely to dive deep without practical need (time is limited, sadly).
I'm happy more people are interested in this critique! And thanks for d12frosted for joining the discussion.
I was searching for what's the latest in emacs widgets developments because I was interested in using widgets. Emacs Customize for example, renders text-based widgets in an editable buffer, which is very uncommon nowadays. Emacs seems like the best candidate for this kind of interactivity. I'm sure it's possible in other editors, but with significantly more effort and significantly slower rendering. I also considered lem, but the barrier also seems much higher.
His post also led me to his vui.el project, but I ended up not trying it, since after understanding the tradeoffs, I pushed the widget code creation to an LLM. I still get frequent unbalanced parentheses errors so I still stay close to the defaults.
The PoC is about testing a method to render a widgets-based, json-schema-validated input form that you can embed dynamically into an emacs buffer, enter the data, then do something post-validation. If anyone's interested, here's the latest state of the LLM-generated and human-fixed code: https://gist.github.com/whacked/6875c5117b48528e619755e13e24...
Thanks for sharing! The JSON schema → widget mapping is a solid use case, and your two-tier validation (keystroke + submit-time) is a nice touch.
On the unbalanced parentheses from LLMs - I've found Claude Code with Opus 4.5 is generally quite good at keeping them balanced, fwiw.
Curious about this though:
> after understanding the tradeoffs, I pushed the widget code creation to an LLM
What tradeoffs made you go with raw widget.el over vui.el? Genuine question - that's exactly the kind of feedback that helps me understand if vui.el is solving the right problems or missing something.
Your gist essentially reimplements some things vui.el provides generically (state tracking, inline form lifecycle, cleanup). Would be interesting to see if vui.el could simplify the widget/state parts while you focus on the schema translation - or learn what's missing if it can't.
If having conversation here is not convenient, we can continue under your gist or via email. Let me know if you are interested :) If not - that's also fine :)
The tradeoffs are very shallow, mainly I just wanted the maximum chance of seeing something working ASAP.
Staying vanilla reduces dependencies, which also makes testing easier during iteration. I forgot which agent (CC or codex) I used for the bulk of the code generation, but some times I manually do some one-off dialog and those get unbalanced parens.
In the agent I ask it to do `emacs -q -nw -l ...` to iterate, so it starts a bare emacs instance. This seems to have worked well when adding [Submit and Close] and [Cancel] buttons, as well as "move the cursor into the first text input widget when inserting a new array item" (the default action is to keep the cursor on the [INS] widget after inserting the list item).
The next consideration is just that I am less confident in the agent's ability to just ingest a .el library and use it correctly vs something more popular like python. Maybe it can, I just wanted results fast and not have to dig too deep myself. I had to go in and do some (setq debug-on-error t) toggles a few times while it was figuring out the json schema load into alist/hashmap differences and I didn't want to spend more time doing plumbing.
But as you probably can imagine, dynamic inline forms immediate gives us state issues, so I asked the agent to create a buffer-local alist (or hashmap?) to track the states of the form input, so it can be cleaned up on close. It's a bit unreliable now. If vui.el already has a solution I'll switch over next.
Thanks for the detailed response! This is really helpful feedback.
Looking at your gist, I think the code actually illustrates why I built vui.el in the first place. The schema→widget mapping logic is genuinely interesting work, but a significant chunk of the code (~400 lines) is dedicated to the inline form lifecycle: jsf--inline-forms registry, marker tracking, resync passes, advice on widget-before-change, overlay cleanup, etc. That's exactly the plumbing vui.el handles automatically.
With vui.el, your json-schema-form could potentially be just the schema translation + a component wrapper:
State tracking, cleanup on close, multiple forms per buffer - all handled by the framework. Your validation logic and schema mapping would be the same, just without the infrastructure code.
On the emacs -q -nw workflow: it works, but you might find eldev + buttercup tests even better for AI-assisted iteration. The agent can run eldev test after changes and self-correct on failures. Byte-compilation catches issues too. Claude Code handles eldev well out of the box.
Anyway, not trying to hard-sell vui.el - your approach clearly works and the schema-form idea is cool. But if you do hit more state/cleanup headaches, might be worth a look. Happy to help if you want to try porting the schema logic over.
I played with vui.el's hello world code earlier today, and it's a great showcase!
Is there a simple way to achieve in-buffer forms? The jsf-- stuff was instructed to make the widget form read-only except for the interactive widget components (took a few iterations to work), while keeping the rest of the buffer editable. The demos seem to all mount to a new buffer. Though I could also feed vui.el into an agent and ask :-)
Thanks for the tips. I guess I can add the deps to --eval "(load 'eldev)" in manual testing.
Glad you like it! I was curious to see the comparison myself, so I asked Claude Code to reimplement it with vui.el - wanted concrete numbers and code rather than just claims. The 60% reduction was a pleasant surprise.
Regarding in-buffer forms: I'd love to understand your use case better. When you say "injecting" forms into a buffer while keeping the rest editable - what's the actual workflow you're building? A few scenarios come to mind:
1. Inline editing in org/markdown - e.g., a structured data block that becomes a form when you enter it?
2. Mixed content - documentation with embedded interactive widgets?
3. Progressive disclosure - expand a section into a form, collapse back when done?
Right now vui.el mounts to a dedicated buffer, so it doesn't support inline injection. But depending on the use case, it might not be too complex to add - or there might be a different approach that works better. Would be interested to hear more about what you're trying to build.
(And yes, feeding vui.el to an agent works surprisingly well - that's exactly how the gist was created!)
I haven't thought through the entire use case. But it loosely revolves around the interaction designs provided by org-mode, ipython/jupyter notebooks, atuin desktop, aka "interactive documents", where the execution intent is embedded in text.
Jupyter is objectively more powerful from an interactivity standpoint, but the cell-based editing strategy has always fell short from a navigation and versioning standpoint. OTOH text-only interaction is sometimes too free, so after all these years, I still haven't found anything that feels just right.
Say you are writing devops documentation. Skipping the many ugly ways you can achieve this in jupyter (e.g. you need ssh logins and creds), or even slightly less ugly in atuin, in org-mode you'd use a begin_src / end_src block and eval that block. The block kind of breaks the text flow. A possibly better interaction mode would be a text "button", borrowing from GNU Hyperbole's terminology.
Continuing the button use case, say the devops document says "<run this> to check the uptimes of our servers". That button is simple enough. You can evaluate this and get a buffer, or like org-babel, insert the output into a block inline. Now we want to extend the functionality: "<query the ps of a given server>". We want parameterization, but without breaking flow. So here I'm thinking of injecting a disposable form at cursor. In a web-based toolkit, we'd use a popup or expandable form, but that switches focus out of the document and into a modal. Does that make sense?
I'm not sure how well it will all fit, but every now and then I feel the same itch, and it seems like in 10+ years of using different tools, Emacs is still the best for providing "structured, validated input for in-document interaction".
This makes a lot of sense now - thanks for the detailed walkthrough.
So the core idea is: ephemeral, inline forms that appear at point, collect structured input, execute, and disappear. Not persistent multi-form buffers, but disposable parameter prompts that don't break document flow or switch context to a modal.
The <query the ps of a given server> example is perfect - you want:
1. Click/activate the button
2. Small form expands inline (server selector, maybe filters)
3. Fill, submit
4. Form vanishes, result appears (inline or in separate buffer)
This is actually more tractable than what the gist's inline extension was doing. That was about persistent forms with state tracking, cleanup on manual deletion, etc. Your use case is simpler: mount form at point → collect input → unmount → done.
vui.el doesn't support this today, but it's not a huge leap. The pieces are there:
- Components with state and validation
- Controlled rendering
- What's missing: mounting into an existing buffer at point (vs taking over a buffer)
I'd need to think through how it interacts with the host buffer's text properties and undo - that's where the gist's widget-before-change advice complexity came from. But for disposable forms that clean up after themselves, it should be simpler.
If you want to experiment with this, I'd be curious to collaborate. It's a use case I hadn't fully considered, but "structured, validated input for in-document interaction" is a good framing.
I clicked through hoping you were wrong, saw the first page, and thought, ah, this is legible... then I got to the code blocks and was completely blinded.
I'm not sure what to tell authors of such pages...
Yes, copied it in to my scratch buffer to read it, not readable in the browser at all with a dark background. It did then make all the elisp nice to look at.
I only have a limited experience with GUI Widgets - by using JavaFX through `cljfx`
- vui.el seems like the right idea - You have a widget library, and then you add state management, you add layout management etc. It's sort of a blessing widget is simple enough to be reused this way
- ECS and inheritance. I have personally never really hit on this limitation. It's there in the abstract.. but don't GUIs generally fit the OO paradigm pretty well? Looking at the class tree for JavaFX, I don't see any really awkward widgets that are ambiguously placed.
- State Management. This can be bolted on - like with vui.el. But to me this feels like something that should be independent of GUIs. Something like Pathom looks more appealing to me here
- "Not a full reactive framework - Emacs doesn't need that complexity" .. why not? Maybe the library internals are complex, but it makes user code much simpler, no?
On vui.el's approach - yes, the blessing is that widget.el is simple enough to build on. It does the "rendering" and some "behaviour", vui.el handles the rest.
On ECS vs OO - I'll admit I don't have enough experience to speak about UI paradigms in general. But my critique of widget.el is that inheritance hierarchies don't compose well when you need orthogonal behaviors. Composition feels more natural to me - could be just how my brain works, but it scales better in my experience.
On state management being independent - I'd be curious to hear more. Pathom is interesting for data-driven architectures. vui.el's state is intentionally minimal and Emacs-native, but you're right it could potentially be decoupled further.
On "why not full reactive" - to clarify what vui.el has: React-style hooks with explicit dependency tracking (vui-use-effect, vui-use-memo, etc.), state changes trigger re-renders, batching for multiple updates. What it doesn't have: automatic dependency inference or fine-grained reactivity where only specific widgets update. The tradeoff was debuggability - explicit deps are easier to trace than magic. But I'm open to being wrong here. What would you want from a reactive layer?
- "don't compose well when you need orthogonal behavior" ah okay, you've actually hit this case. I guess I haven't done anything gnarly enough to encounter this
> Pathom is interesting for data-driven architectures. vui.el's state is intentionally minimal and Emacs-native, but you're right it could potentially be decoupled further.
I'll be honest, I haven't yet written a Pathom-backed GUI. But I'm hoping to experiment with this in the coming weeks :)) cljfx is structured in such a way that you can either use the provided subscription system or you can roll your own.
> What it doesn't have: automatic dependency inference or fine-grained reactivity where only specific widgets update
So all the derived states are recalculated? Probably in the 95% case this i fine
In the big picture I enjoyed the cljfx subscription system so much, that I'd like to use a "reactive layer" at the REPL and in general applications. You update some input and only the parts that are relevant get updated. With a subscription-style system the downside is that the code is effectively "instrumented" with subscription calls to the state. You aren't left with easily testable function calls and it's a bit uglier.
Pathom kind of solves this and introduces several awesome additional features. Now your "resolvers" can behave like dumb functions that take a map-of-input and return a map-of-output. They're nicer to play with at the REP and are more idiomatic Clojure. On top of that your code turns in to pipelines that can be injected in to at any point (so the API becomes a lot more flexible). And finally, the resolvers can auto parallelized as the engine can see which parts of the dependency graph (for the derived state you're prompting) can be run in parallel.
The downsides are mostly related to caching of results. You need an "engine" that has to run all the time to find "given my inputs, how do I construct the derived state the user wants". In theory these can be cached, but the cache is easily invalidated. You add a key on the input, and the engine has to rerun everything (maybe this can be bypassed somehow?). You also can concoct complex scenarios where the caching of the derived states is non-trivial. Derived state values are cached by the resolvers themselved, but they have a limited view of how often and where they're needed. If two derived states use one intermediary resolver but with different inputs, you need to find a way to adjust the cache size.. Unclear to me how to do this tbh
Thanks for the detailed breakdown on Pathom and cljfx subscriptions - this is exactly the kind of perspective I was hoping to hear.
The resolver model you describe (dumb functions, map-in → map-out, parallelizable) is appealing. It's similar to what I find elegant about React's model too - components as pure functions of props/state. The difference is where the "smarts" live: in the dependency graph engine vs in the reconciliation/diffing layer.
Your point about the 95% case resonates with vui.el's approach. We do have vui-use-memo for explicit memoization — so expensive computations can be cached with declared dependencies. It's the middle ground: you opt-in to memoization where it matters, rather than having an engine track everything automatically.
For typical Emacs UIs (settings panels, todo lists, file browsers), re-rendering the component tree on state change is fast enough that you rarely need it. But when you do — large derived data, expensive transformations — vui-use-memo is there. The tradeoff is explicit deps vs automatic tracking: you tell it what to cache and when to invalidate, rather than the framework inferring it.
That said, I'm planning to build a more complex UI for https://github.com/d12frosted/vulpea (my note-taking library) - browsing/filtering/viewing notes with potentially large datasets. That'll be a real test of whether my performance claims hold up against reality. So ff vui.el ever needs to go there, the component model doesn't preclude adding finer-grained updates later. The should-update hook already lets you short-circuit re-renders, and memoization could be added at the vnode level.
The caching/invalidation complexity you mention is what made me hesitant to start there. "Explicit deps are easier to trace than magic" was the tradeoff I consciously made. But I'm genuinely curious - if you do experiment with Pathom-backed GUI, I'd love to hear how it goes. Especially around the cache invalidation edge cases you mentioned.
I wrote my last message and then thought "oh gosh, it's so long, no one will read it". But I'm glad to find someone thinking about similar problems :))
> It's the middle ground: you opt-in to memoization where it matters, rather than having an engine track everything automatically.
What is the downside of just memoizing everything?
> you tell it what to cache and when to invalidate
I'd be curious to here more on this. I think the conceptual problem I'm hitting is more that these derived state systems (and this goes for a Pathom engine or a subscription model) seem to work optimally on "shallow" derived states. Ex: You have a "username" you fetch his avatar (one derived state) and then render it (maybe another derived state). That's fine.
But if now have a list of users... each has usernames and derived avatars and rendered images.. the list of users is changing with people being added and removed - this get mess.
With Pathom you can make username,avatar,render revolvers for the derived states. It's nicely decoupled and you can even put it in a separate library. With cljfx subscribers you can make subscriptions based on the state + username. It's more coupled, but you nolonger need an engine. Functionally it's the same.
But when it comes to the cache, I don't actually know any system that would handle this "correctly" - clearing cache entries when users are removed. Under the current solutions you seem to only have two solutions:
- You just sort of guess and make a "large enough" FIFO cache. You either eat memory or thrash the cache.
- Make Resolvers/Subscriptions on whole lists. So now you have usernames->avatars->renderings. This makes the derived states "shallow" again. If any entry is changed, the whole lists of derived states are recalculated. Memory optimal and fast to fetch (don't need to look check the memoization cache a ton). But if your list is rapidly changing, then you're doing a ton of recalculation.
I actually haven't worked with React directly, so I'd be curious to know how they propose dealing with this.
> if you do experiment with Pathom-backed GUI, I'd love to hear how it goes
When drawing outlines, tables, etc, I hope the new approach (vui) uses real Unicode box-drawing characters and not - | + etc. It's high time TUI people raise their standards a bit and consider unbroken line drawing (no gaps between repeated characters) a requirement rather than a luxury!
> The buffer is the UI, rendered by Emacs's extremely optimised text display machinery
Doesn't emacs lag like crazy in files with large lines. Why is this still a problem? Every modern editor handles this gracefully. I remember reading something about using regexes for syntax highlighting. This looks like a problem in the rendering layer which shouldn't be too hard to fix without touching the core engine. Are there any other problems that make it difficult to fix without disabling any useful features?
The problem with long lines was reportedly markedly improved in Emacs 29:
Emacs is now capable of editing files with very long lines.
The display of long lines has been optimized, and Emacs should no
longer choke when a buffer on display contains long lines. The
variable 'long-line-threshold' controls whether and when these display
optimizations are in effect.
A companion variable 'large-hscroll-threshold' controls when another
set of display optimizations are in effect, which are aimed
specifically at speeding up display of long lines that are truncated
on display.
If you still experience slowdowns while editing files with long lines,
this may be due to line truncation, or to one of the enabled minor
modes, or to the current major mode. Try turning off line truncation
with 'C-x x t', or try disabling all known slow minor modes with
'M-x so-long-minor-mode', or try disabling both known slow minor modes
and the major mode with 'M-x so-long-mode', or visit the file with
'M-x find-file-literally' instead of the usual 'C-x C-f'.
In buffers in which these display optimizations are in effect, the
'fontification-functions', 'pre-command-hook' and 'post-command-hook'
hooks are executed on a narrowed portion of the buffer, whose size is
controlled by the variables 'long-line-optimizations-region-size' and
'long-line-optimizations-bol-search-limit', as if they were in a
'with-restriction' form. This may, in particular, cause occasional
mis-fontifications in these buffers. Modes which are affected by
these optimizations and by the fact that the buffer is narrowed,
should adapt and either modify their algorithm so as not to expect the
entire buffer to be accessible, or, if accessing outside of the
narrowed region doesn't hurt performance, use the
'without-restriction' form to temporarily lift the restriction and
access portions of the buffer outside of the narrowed region.
The new function 'long-line-optimizations-p' returns non-nil when
these optimizations are in effect in the current buffer.
Right- but if you have a long line that is, for example, a JSON object, then surely it can't be properly be validated or syntax-highlighted before the entire line is scanned?
I do agree that Emacs can be slower than the terminal when handling long lines/files, although (depending on your case) this can be easily mitigated by running a terminal inside of Emacs.
Generally though, for everyday use, Emacs feels a lot snappier than VSCode.
Good point. Though for widget UIs you're typically rendering structured data you control, not parsing arbitrary text files. The syntax highlighting / validation concern applies to editing code, not to building interactive interfaces.
> Generally though, for everyday use, Emacs feels a lot snappier than VSCode.
So long mode is the best fix for this issue but it disables syntax highlighting and line numbers. Vscode can handle long lines just fine without disabling anything.
The long-line issue is real, though my statement was specifically about building UIs with widgets/overlays/text properties - not handling arbitrary files. In that context, Emacs's display engine is genuinely well-optimized: it handles overlays, faces, text properties, and redisplay regions efficiently.
When you're building a UI, you control the content. Lines are short by design (form fields, buttons, lists). The pathological case of a 50KB minified JSON line simply doesn't occur.
The long-line problem stems from how Emacs calculates display width for bidirectional text and variable-pitch fonts - it needs to scan the entire line. That's orthogonal to rendering widgets or interactive buffers.
The buffer is the UI, rendered by Emacs's extremely optimised text display machinery
The author is known in the community as a mere packager whose knowledge of the nitty-gritty derives entirely from hearsay. Perhaps he read the long-winded preamble to xdisp.c written in 1995 boasting of all manner of optimisations. But they were written so long ago, almost no one believes most of them matter anymore, what with thirty years of bitrot.
Argh, a few screenshots would’ve been nice. There’s none to be found in vui.el’s repository.
Agreed, especially since org-mode (which I bet this blog post was written in) supports (inline) images in its exporters.
For those interested, this guy is revamping the Emacs widget library with something more modern and platform agnostic, based on SDL: https://appetrosyan.github.io/posts/
His posts are very insightful.
Interesting, thanks for sharing! I've had thoughts about making vui.el backend-agnostic so it could target different widget implementations (like xwidgets or even native-GUI). An SDL-based widget library could potentially be one of those backends. Need to dig into appetrosyan's work before I can say anything intelligent about it though. And of course, it was an idea and I am unlikely to dive deep without practical need (time is limited, sadly).
He's building a new Emacs, or he's building a new library for the existing Emacs?
I couldn't tell from the list of blog posts about on non-Emacs topics.
I'm happy more people are interested in this critique! And thanks for d12frosted for joining the discussion.
I was searching for what's the latest in emacs widgets developments because I was interested in using widgets. Emacs Customize for example, renders text-based widgets in an editable buffer, which is very uncommon nowadays. Emacs seems like the best candidate for this kind of interactivity. I'm sure it's possible in other editors, but with significantly more effort and significantly slower rendering. I also considered lem, but the barrier also seems much higher.
His post also led me to his vui.el project, but I ended up not trying it, since after understanding the tradeoffs, I pushed the widget code creation to an LLM. I still get frequent unbalanced parentheses errors so I still stay close to the defaults.
The PoC is about testing a method to render a widgets-based, json-schema-validated input form that you can embed dynamically into an emacs buffer, enter the data, then do something post-validation. If anyone's interested, here's the latest state of the LLM-generated and human-fixed code: https://gist.github.com/whacked/6875c5117b48528e619755e13e24...
Thanks for sharing! The JSON schema → widget mapping is a solid use case, and your two-tier validation (keystroke + submit-time) is a nice touch.
On the unbalanced parentheses from LLMs - I've found Claude Code with Opus 4.5 is generally quite good at keeping them balanced, fwiw.
Curious about this though:
> after understanding the tradeoffs, I pushed the widget code creation to an LLM
What tradeoffs made you go with raw widget.el over vui.el? Genuine question - that's exactly the kind of feedback that helps me understand if vui.el is solving the right problems or missing something.
Your gist essentially reimplements some things vui.el provides generically (state tracking, inline form lifecycle, cleanup). Would be interesting to see if vui.el could simplify the widget/state parts while you focus on the schema translation - or learn what's missing if it can't.
If having conversation here is not convenient, we can continue under your gist or via email. Let me know if you are interested :) If not - that's also fine :)
The tradeoffs are very shallow, mainly I just wanted the maximum chance of seeing something working ASAP.
Staying vanilla reduces dependencies, which also makes testing easier during iteration. I forgot which agent (CC or codex) I used for the bulk of the code generation, but some times I manually do some one-off dialog and those get unbalanced parens.
In the agent I ask it to do `emacs -q -nw -l ...` to iterate, so it starts a bare emacs instance. This seems to have worked well when adding [Submit and Close] and [Cancel] buttons, as well as "move the cursor into the first text input widget when inserting a new array item" (the default action is to keep the cursor on the [INS] widget after inserting the list item).
The next consideration is just that I am less confident in the agent's ability to just ingest a .el library and use it correctly vs something more popular like python. Maybe it can, I just wanted results fast and not have to dig too deep myself. I had to go in and do some (setq debug-on-error t) toggles a few times while it was figuring out the json schema load into alist/hashmap differences and I didn't want to spend more time doing plumbing.
But as you probably can imagine, dynamic inline forms immediate gives us state issues, so I asked the agent to create a buffer-local alist (or hashmap?) to track the states of the form input, so it can be cleaned up on close. It's a bit unreliable now. If vui.el already has a solution I'll switch over next.
Thanks for the detailed response! This is really helpful feedback.
Looking at your gist, I think the code actually illustrates why I built vui.el in the first place. The schema→widget mapping logic is genuinely interesting work, but a significant chunk of the code (~400 lines) is dedicated to the inline form lifecycle: jsf--inline-forms registry, marker tracking, resync passes, advice on widget-before-change, overlay cleanup, etc. That's exactly the plumbing vui.el handles automatically.
With vui.el, your json-schema-form could potentially be just the schema translation + a component wrapper:
State tracking, cleanup on close, multiple forms per buffer - all handled by the framework. Your validation logic and schema mapping would be the same, just without the infrastructure code.On the emacs -q -nw workflow: it works, but you might find eldev + buttercup tests even better for AI-assisted iteration. The agent can run eldev test after changes and self-correct on failures. Byte-compilation catches issues too. Claude Code handles eldev well out of the box.
Anyway, not trying to hard-sell vui.el - your approach clearly works and the schema-form idea is cool. But if you do hit more state/cleanup headaches, might be worth a look. Happy to help if you want to try porting the schema logic over.
I played with vui.el's hello world code earlier today, and it's a great showcase!
Is there a simple way to achieve in-buffer forms? The jsf-- stuff was instructed to make the widget form read-only except for the interactive widget components (took a few iterations to work), while keeping the rest of the buffer editable. The demos seem to all mount to a new buffer. Though I could also feed vui.el into an agent and ask :-)
Thanks for the tips. I guess I can add the deps to --eval "(load 'eldev)" in manual testing.
edit: wow, I just saw your implementation at https://gist.github.com/d12frosted/51edb747167c0fa8fcce40fed...
this is awesome!
Glad you like it! I was curious to see the comparison myself, so I asked Claude Code to reimplement it with vui.el - wanted concrete numbers and code rather than just claims. The 60% reduction was a pleasant surprise.
Regarding in-buffer forms: I'd love to understand your use case better. When you say "injecting" forms into a buffer while keeping the rest editable - what's the actual workflow you're building? A few scenarios come to mind:
1. Inline editing in org/markdown - e.g., a structured data block that becomes a form when you enter it? 2. Mixed content - documentation with embedded interactive widgets? 3. Progressive disclosure - expand a section into a form, collapse back when done?
Right now vui.el mounts to a dedicated buffer, so it doesn't support inline injection. But depending on the use case, it might not be too complex to add - or there might be a different approach that works better. Would be interested to hear more about what you're trying to build.
(And yes, feeding vui.el to an agent works surprisingly well - that's exactly how the gist was created!)
I haven't thought through the entire use case. But it loosely revolves around the interaction designs provided by org-mode, ipython/jupyter notebooks, atuin desktop, aka "interactive documents", where the execution intent is embedded in text.
Jupyter is objectively more powerful from an interactivity standpoint, but the cell-based editing strategy has always fell short from a navigation and versioning standpoint. OTOH text-only interaction is sometimes too free, so after all these years, I still haven't found anything that feels just right.
Say you are writing devops documentation. Skipping the many ugly ways you can achieve this in jupyter (e.g. you need ssh logins and creds), or even slightly less ugly in atuin, in org-mode you'd use a begin_src / end_src block and eval that block. The block kind of breaks the text flow. A possibly better interaction mode would be a text "button", borrowing from GNU Hyperbole's terminology.
Continuing the button use case, say the devops document says "<run this> to check the uptimes of our servers". That button is simple enough. You can evaluate this and get a buffer, or like org-babel, insert the output into a block inline. Now we want to extend the functionality: "<query the ps of a given server>". We want parameterization, but without breaking flow. So here I'm thinking of injecting a disposable form at cursor. In a web-based toolkit, we'd use a popup or expandable form, but that switches focus out of the document and into a modal. Does that make sense?
I'm not sure how well it will all fit, but every now and then I feel the same itch, and it seems like in 10+ years of using different tools, Emacs is still the best for providing "structured, validated input for in-document interaction".
This makes a lot of sense now - thanks for the detailed walkthrough.
So the core idea is: ephemeral, inline forms that appear at point, collect structured input, execute, and disappear. Not persistent multi-form buffers, but disposable parameter prompts that don't break document flow or switch context to a modal.
The <query the ps of a given server> example is perfect - you want:
This is actually more tractable than what the gist's inline extension was doing. That was about persistent forms with state tracking, cleanup on manual deletion, etc. Your use case is simpler: mount form at point → collect input → unmount → done.vui.el doesn't support this today, but it's not a huge leap. The pieces are there:
I'd need to think through how it interacts with the host buffer's text properties and undo - that's where the gist's widget-before-change advice complexity came from. But for disposable forms that clean up after themselves, it should be simpler.If you want to experiment with this, I'd be curious to collaborate. It's a use case I hadn't fully considered, but "structured, validated input for in-document interaction" is a good framing.
This site is quite illegible if your system is set to prefers dark theme.
I clicked through hoping you were wrong, saw the first page, and thought, ah, this is legible... then I got to the code blocks and was completely blinded.
I'm not sure what to tell authors of such pages...
easy :) ask for dark theme; will try to implement one soon ;)
update: added; turns out I almost finished it before on a local branch but didn't push to master
Yes, copied it in to my scratch buffer to read it, not readable in the browser at all with a dark background. It did then make all the elisp nice to look at.
~M-x eww~ works nicely too.
sorry for that! added a dark theme, feedback is welcome :pray:
Oh, that's the guy who does `homebrew-emacs-plue` --- my preferred distribution of Emacs!
I only have a limited experience with GUI Widgets - by using JavaFX through `cljfx`
- vui.el seems like the right idea - You have a widget library, and then you add state management, you add layout management etc. It's sort of a blessing widget is simple enough to be reused this way
- ECS and inheritance. I have personally never really hit on this limitation. It's there in the abstract.. but don't GUIs generally fit the OO paradigm pretty well? Looking at the class tree for JavaFX, I don't see any really awkward widgets that are ambiguously placed.
- State Management. This can be bolted on - like with vui.el. But to me this feels like something that should be independent of GUIs. Something like Pathom looks more appealing to me here
- "Not a full reactive framework - Emacs doesn't need that complexity" .. why not? Maybe the library internals are complex, but it makes user code much simpler, no?
Good points, thanks for engaging thoughtfully.
On vui.el's approach - yes, the blessing is that widget.el is simple enough to build on. It does the "rendering" and some "behaviour", vui.el handles the rest.
On ECS vs OO - I'll admit I don't have enough experience to speak about UI paradigms in general. But my critique of widget.el is that inheritance hierarchies don't compose well when you need orthogonal behaviors. Composition feels more natural to me - could be just how my brain works, but it scales better in my experience.
On state management being independent - I'd be curious to hear more. Pathom is interesting for data-driven architectures. vui.el's state is intentionally minimal and Emacs-native, but you're right it could potentially be decoupled further.
On "why not full reactive" - to clarify what vui.el has: React-style hooks with explicit dependency tracking (vui-use-effect, vui-use-memo, etc.), state changes trigger re-renders, batching for multiple updates. What it doesn't have: automatic dependency inference or fine-grained reactivity where only specific widgets update. The tradeoff was debuggability - explicit deps are easier to trace than magic. But I'm open to being wrong here. What would you want from a reactive layer?
- "don't compose well when you need orthogonal behavior" ah okay, you've actually hit this case. I guess I haven't done anything gnarly enough to encounter this
> Pathom is interesting for data-driven architectures. vui.el's state is intentionally minimal and Emacs-native, but you're right it could potentially be decoupled further.
I'll be honest, I haven't yet written a Pathom-backed GUI. But I'm hoping to experiment with this in the coming weeks :)) cljfx is structured in such a way that you can either use the provided subscription system or you can roll your own.
> What it doesn't have: automatic dependency inference or fine-grained reactivity where only specific widgets update
So all the derived states are recalculated? Probably in the 95% case this i fine
In the big picture I enjoyed the cljfx subscription system so much, that I'd like to use a "reactive layer" at the REPL and in general applications. You update some input and only the parts that are relevant get updated. With a subscription-style system the downside is that the code is effectively "instrumented" with subscription calls to the state. You aren't left with easily testable function calls and it's a bit uglier.
Pathom kind of solves this and introduces several awesome additional features. Now your "resolvers" can behave like dumb functions that take a map-of-input and return a map-of-output. They're nicer to play with at the REP and are more idiomatic Clojure. On top of that your code turns in to pipelines that can be injected in to at any point (so the API becomes a lot more flexible). And finally, the resolvers can auto parallelized as the engine can see which parts of the dependency graph (for the derived state you're prompting) can be run in parallel.
The downsides are mostly related to caching of results. You need an "engine" that has to run all the time to find "given my inputs, how do I construct the derived state the user wants". In theory these can be cached, but the cache is easily invalidated. You add a key on the input, and the engine has to rerun everything (maybe this can be bypassed somehow?). You also can concoct complex scenarios where the caching of the derived states is non-trivial. Derived state values are cached by the resolvers themselved, but they have a limited view of how often and where they're needed. If two derived states use one intermediary resolver but with different inputs, you need to find a way to adjust the cache size.. Unclear to me how to do this tbh
Thanks for the detailed breakdown on Pathom and cljfx subscriptions - this is exactly the kind of perspective I was hoping to hear.
The resolver model you describe (dumb functions, map-in → map-out, parallelizable) is appealing. It's similar to what I find elegant about React's model too - components as pure functions of props/state. The difference is where the "smarts" live: in the dependency graph engine vs in the reconciliation/diffing layer.
Your point about the 95% case resonates with vui.el's approach. We do have vui-use-memo for explicit memoization — so expensive computations can be cached with declared dependencies. It's the middle ground: you opt-in to memoization where it matters, rather than having an engine track everything automatically.
For typical Emacs UIs (settings panels, todo lists, file browsers), re-rendering the component tree on state change is fast enough that you rarely need it. But when you do — large derived data, expensive transformations — vui-use-memo is there. The tradeoff is explicit deps vs automatic tracking: you tell it what to cache and when to invalidate, rather than the framework inferring it.
That said, I'm planning to build a more complex UI for https://github.com/d12frosted/vulpea (my note-taking library) - browsing/filtering/viewing notes with potentially large datasets. That'll be a real test of whether my performance claims hold up against reality. So ff vui.el ever needs to go there, the component model doesn't preclude adding finer-grained updates later. The should-update hook already lets you short-circuit re-renders, and memoization could be added at the vnode level.
The caching/invalidation complexity you mention is what made me hesitant to start there. "Explicit deps are easier to trace than magic" was the tradeoff I consciously made. But I'm genuinely curious - if you do experiment with Pathom-backed GUI, I'd love to hear how it goes. Especially around the cache invalidation edge cases you mentioned.
I wrote my last message and then thought "oh gosh, it's so long, no one will read it". But I'm glad to find someone thinking about similar problems :))
> It's the middle ground: you opt-in to memoization where it matters, rather than having an engine track everything automatically.
What is the downside of just memoizing everything?
> you tell it what to cache and when to invalidate
I'd be curious to here more on this. I think the conceptual problem I'm hitting is more that these derived state systems (and this goes for a Pathom engine or a subscription model) seem to work optimally on "shallow" derived states. Ex: You have a "username" you fetch his avatar (one derived state) and then render it (maybe another derived state). That's fine.
But if now have a list of users... each has usernames and derived avatars and rendered images.. the list of users is changing with people being added and removed - this get mess.
With Pathom you can make username,avatar,render revolvers for the derived states. It's nicely decoupled and you can even put it in a separate library. With cljfx subscribers you can make subscriptions based on the state + username. It's more coupled, but you nolonger need an engine. Functionally it's the same.
But when it comes to the cache, I don't actually know any system that would handle this "correctly" - clearing cache entries when users are removed. Under the current solutions you seem to only have two solutions:
- You just sort of guess and make a "large enough" FIFO cache. You either eat memory or thrash the cache.
- Make Resolvers/Subscriptions on whole lists. So now you have usernames->avatars->renderings. This makes the derived states "shallow" again. If any entry is changed, the whole lists of derived states are recalculated. Memory optimal and fast to fetch (don't need to look check the memoization cache a ton). But if your list is rapidly changing, then you're doing a ton of recalculation.
I actually haven't worked with React directly, so I'd be curious to know how they propose dealing with this.
> if you do experiment with Pathom-backed GUI, I'd love to hear how it goes
Will do :)
When drawing outlines, tables, etc, I hope the new approach (vui) uses real Unicode box-drawing characters and not - | + etc. It's high time TUI people raise their standards a bit and consider unbroken line drawing (no gaps between repeated characters) a requirement rather than a luxury!
vui-table supports three :border variants at the moment - :ascii, :unicode and nil (i.e. none)
:ascii
:unicode I think there is room for improvement (like use thicker lines for headers in unicode) and I also need to provide a way to override these.Thank whoever for posting this.
A blog post about widgets with zero screenshots, kind of tells where priorities lie.
vui.el, nice!
> The buffer is the UI, rendered by Emacs's extremely optimised text display machinery
Doesn't emacs lag like crazy in files with large lines. Why is this still a problem? Every modern editor handles this gracefully. I remember reading something about using regexes for syntax highlighting. This looks like a problem in the rendering layer which shouldn't be too hard to fix without touching the core engine. Are there any other problems that make it difficult to fix without disabling any useful features?
The problem with long lines was reportedly markedly improved in Emacs 29:
Emacs is now capable of editing files with very long lines. The display of long lines has been optimized, and Emacs should no longer choke when a buffer on display contains long lines. The variable 'long-line-threshold' controls whether and when these display optimizations are in effect.
A companion variable 'large-hscroll-threshold' controls when another set of display optimizations are in effect, which are aimed specifically at speeding up display of long lines that are truncated on display.
If you still experience slowdowns while editing files with long lines, this may be due to line truncation, or to one of the enabled minor modes, or to the current major mode. Try turning off line truncation with 'C-x x t', or try disabling all known slow minor modes with 'M-x so-long-minor-mode', or try disabling both known slow minor modes and the major mode with 'M-x so-long-mode', or visit the file with 'M-x find-file-literally' instead of the usual 'C-x C-f'.
In buffers in which these display optimizations are in effect, the 'fontification-functions', 'pre-command-hook' and 'post-command-hook' hooks are executed on a narrowed portion of the buffer, whose size is controlled by the variables 'long-line-optimizations-region-size' and 'long-line-optimizations-bol-search-limit', as if they were in a 'with-restriction' form. This may, in particular, cause occasional mis-fontifications in these buffers. Modes which are affected by these optimizations and by the fact that the buffer is narrowed, should adapt and either modify their algorithm so as not to expect the entire buffer to be accessible, or, if accessing outside of the narrowed region doesn't hurt performance, use the 'without-restriction' form to temporarily lift the restriction and access portions of the buffer outside of the narrowed region.
The new function 'long-line-optimizations-p' returns non-nil when these optimizations are in effect in the current buffer.
— <https://www.gnu.org/software/emacs/news/NEWS.29.1>
Right- but if you have a long line that is, for example, a JSON object, then surely it can't be properly be validated or syntax-highlighted before the entire line is scanned?
I do agree that Emacs can be slower than the terminal when handling long lines/files, although (depending on your case) this can be easily mitigated by running a terminal inside of Emacs.
Generally though, for everyday use, Emacs feels a lot snappier than VSCode.
Good point. Though for widget UIs you're typically rendering structured data you control, not parsing arbitrary text files. The syntax highlighting / validation concern applies to editing code, not to building interactive interfaces.
> Generally though, for everyday use, Emacs feels a lot snappier than VSCode.
+1
https://emacs.stackexchange.com/questions/598/how-do-i-preve... has some answers (and solutions).
So long mode is the best fix for this issue but it disables syntax highlighting and line numbers. Vscode can handle long lines just fine without disabling anything.
so-long-mode is sophomoric garbage.
The long-line issue is real, though my statement was specifically about building UIs with widgets/overlays/text properties - not handling arbitrary files. In that context, Emacs's display engine is genuinely well-optimized: it handles overlays, faces, text properties, and redisplay regions efficiently.
When you're building a UI, you control the content. Lines are short by design (form fields, buttons, lists). The pathological case of a 50KB minified JSON line simply doesn't occur.
The long-line problem stems from how Emacs calculates display width for bidirectional text and variable-pitch fonts - it needs to scan the entire line. That's orthogonal to rendering widgets or interactive buffers.
Not every modern editor. Neovim bogs on long lines too.
The buffer is the UI, rendered by Emacs's extremely optimised text display machinery
The author is known in the community as a mere packager whose knowledge of the nitty-gritty derives entirely from hearsay. Perhaps he read the long-winded preamble to xdisp.c written in 1995 boasting of all manner of optimisations. But they were written so long ago, almost no one believes most of them matter anymore, what with thirty years of bitrot.