Gonix: The Sixth Element

Unix is not just pipes and text streams. This is why I introduced a kernel like element into gonix: unix from userspace. To allow the system become close to the real Unix shell.

Beyond running cat | wc -l user will naturally ask

1. How many processes are running?
2. What are their name and arguments?
3. Can I stop a process when running?
4. Can I pause it?
5. And more.

A Monitor

The real Unix is an operating system. This means it has a kernel and it has superpowers that normal user-space code does not and cannot have. At the same time it is the kernel which is able to answer all the question above.

Welcome the fift^W sixth element of a gonix project. The monitor. While it can’t run or interrupt anything, it can track filters or it can facilitate sending the information to the filter. It can even stop or pause and resume well cooperating filters. So I’ve chosen the name monitor.

How to terminate or pause Go code?

The real kernel like unix can setup a signal handler of a process and on a signal delivery simply change the program counter of the processor to the address of the handler. The userspace has no say here and code of an interrupt handler is executed.

However Go itself has two concepts, which can map to the unix signals just well.

1. Context cancellation - This tells the well behaving Go code to stop whatever is doing and return early. Which can emulate SIGKILL and SIGTERM.
2. sync.Cond type which is good enough to emulate SIGSTOP and SIGCONT.

Both concepts are implemented via wrappers like gio.NewContextReader and once filter’s standard io is wrapped, then filter supports the semantics of those signals well. Because most Unix filters are designed around I/O, this simple trick usually works in most situations. You don’t have to add this support to each filter code.

That code is surprisingly short and easy to understand. The only thing to keep in mind is that the function wait() must return once the r.ctx has been canceled. However, it is up to the caller of this wrapper.

// Read implements Reader[T any] interface
func (r ContextReader[T]) Read(p []T) (n int, err error) {
  if r.wait != nil {
    r.wait()
  }

  if err := ctxErr(r.ctx); err != nil {
    return 0, err
  }
  return r.r.Read(p)
}

Signal delivery

As it was mentioned in previous paragraph, Go code can emulate the behavior of a Unix signals almost perfectly. However the Go solution will use channel, select, and goroutines. Even if it can implement the semantics of Unix signals, the delivery mechanism will be different.

This is the reason the solution is called like this.

type Pulser interface {
  Pulse() string
}

The pulse is a concept from a QNX Neutrino that involves an asynchronous message passing between userspace and a kernel. And asynchronous message passing in Go is idiomatic and native, making this a perfect fit.

With than in mind defining a Unix signal equivalent is easy.

var (
  sigkill = &signal{s: "killed"}
  sigstop = &signal{s: "stopped"}
  sigcont = &signal{s: "continued"}
  sigterm = &signal{s: "terminated"}
)

type signal struct {s string}

func (s *signal) Pulse() string {
  return s.s
}

Each wrapper of a unix.Filter has an internal channel for pulses, so all of them can receive them. This method guarantee that he pulse is sent to filter if it is running. Technically this spawns a goroutine for a delivery if the underlying channel is full. So it is not a good idea to send a lot of pulses or write the code which handles them slowly.

This is good enough for mechanism like signals. If you need a strong way to communicate, it’s better to use different Go mechanisms. It’s all just a user space anyway.

// Send sends a Pulser to the filter channel. The delivery is guaranteed and if
// the filter code is slow in a consuming, new goroutines are spawned to ensure
// the delivery. It is never a good idea to send too many pulses or have a slow
// pulse handling.
func (f *filter) Send(signal Pulser)

Unmaskable signals

Unix defines three important signals SIGKILL, SIGSTOP and SIGCONT. Those are never send to the process so it can never handle them. In the Unix terminology it can never mask them.

monitor defines their equivalents:

1. Sigkill() is an equivalent of SIGKILL - means hard stop, never forwarded.
2. Sigstop() is an equivalent of SIGSTOP - this uses sync.Cond to stop the execution until …
3. Sigcont() pulse is delivered, this is an equivalent of SIGCONT and causes the code to resume

Once those pulses are delivered, they are handled inside the wrapper.

Maskable signals and mailbox

The Sigterm() pulse is an equivalent to SIGTERM. It causes the filter’s context to be canceled. However the filter code can opt-in to the delivery. In this case monitor will forward it to mailbox. Which is the term borrowed from Erlang and acts as an asynchronous message queue.

In a case of gonix this is a channel, which makes the pulse-aware filter code looking like

handle, _ := monitor.Self(ctx)
mbox := handle.Mailbox()
for {
  select {
  case <-ctx.Done():
    return ctx.Err()
  case pulse := <- mbox:
    switch pulse {
    case monitor.Sigterm():
      fmt.Fprintf(stdio.Stderr(), "%s\n", pulse.Pulse())
      return nil
    default:
      fmt.Fprintf(stdio.Stdout(), "%s\n", pulse.Pulse())
    }
  }
}

OS Level signals

One cannot expect everything to be available as a native Go code. So gio has a support for a real execve and real OS processes via a Cmd wrapper. This integrates real processes into the project, except for pulses. Those are gonix’s own userspace abstractions, which has nothing to do with the operating system.

In order to solve this, the wrapper expose the fact that it can receive the OS level signals via Signal method. This allows the monitor to translate the pulses into real signals. Each pulse can define a method called Signal which translates the pulse into OS level counterpart if possible. The design is flexible enough that user can define own pulses and their rules how to translate them into to real OS signals.

Spawn

Spawning a filter in a gonix project may seem like a convoluted three-step process at first. There is no equivalent of CreateProcess or fork+execve, and the process may be a bit tedious at first. However, this complex^Wflexibility will soon pay off.

1. The Lookup operation maps the command name in a script to the …
2. the Builder which injects the command name and arguments and shell option to the …
3. correct Filter

This design allows the caller to precisely determine and control the state of the sub-filter. Most importantly, it enables a seamless integration between the various components of the gonix stack.

For a monitor to perform its monitoring function, it must run a dedicated wrapper that provides all the needed functionality. The exposed Lookup method does exactly this. It ensures what that caller get the builder function which will run the properly wrapped and tracked filter.

// codeberg.org/gonix/utils/xargs
// codeberg.org/gonix/sh
// codeberg.org/gonix/monitor

// unix.FilterLookupMap provides compatible FilterLookupFunc
// wrapping the go map - this is handy for examples or tests
var binaries unix.FilterLookupMap
m := monitor.New(binaries.Lookup)
// integrating with xargs-like tool
func XargsNew(unix.FilterLookupFunction) Xargs {}
// integration with a sh
func NewFilters(filters unix.FilterLookupFunc, 
  opts ...unix.ShellContextOption) Filters {}

The beauty of this design is that the utils, sh, and monitor do not depend on each other. The only one dependency is on gio itself. Combined together, they provide a fully user-space Unix scripting with signals and native xargs for gonix.

Additionally the caller has a full control over how it will execute the children.

1. Run and exit? This is not a problem as gonix never reassigns the parent id, does not expect parent to call waitpid.
2. Run and wait? No problem.
3. Run concurrently? No problem. Use sync.Waitgroup, Luke.
4. Run concurrently and cancel on the first failure? No problem. Use errgroup.Group, Anakin.

Final thoughts

This is a huge milestone for me. It’s been almost four years since I’ve been watching the Dennis Ritchie and experimenting with Unix pipeline and fully userspace Unix tools. I’ve never had a side project like this. While the original idea was simple, it led to one or more new ideas each time.

• How hard would it be to write a unix tool (utils repository)?
• There is goawk, how hard it would be write a cat on top of it?
• Having a unified base would be nice (gio).
• There is sh in pure Go, let’s integrate it.
• There are Go native unix tools, let’s integrate the u-root project.
• Rust coreutils has WebAssembly build, let’s integrate them.
• The tools needs a working directory and env variables, figure out.
• I would want to stop or pause the filter, welcome monitor and pulses.
• I would want to know how many were started, how many are in progress and so on.

I can’t believe that the entire project has less than 10,000 (9657 accroding tokei) lines of code. Yet, by integrating excellent third party packages, it can do so much. I am convinced this stance is important especially in days of agentic coding - everyone with an agent would be able to generate 10000, 50,000 or 100,000 lines in no time. We, the humans, can compete only on a metric of less and better designed code.

This minimalism is reflected in the project layout itself. Each repository does one thing. gio itself is so fundamental base of the project I dropped even testify from its dependencies from it. Only standard library is allowed here. Similarly, the dependency chains are kept minimal. Integration tests, like the one integrating sh, utils/xargs and monitor, are in their specific go packages with own go.mod. This way the test dependencies do not pollute the main module. At the same time I refused to better error library to monitor. All popular packages felt extra heavy given the little of a functionality I really needed.

To AI or not to AI?

It is 2026! This topic must be made clear. I wrote every production line of code by myself. At the same time, I must admit that LLMs were useful when working on this project. Given the little time I had, I would never been able to finish it so fast. So I took this as a good opportunity to learn how to use the LLMs. So far, I have found the chats and agents good at

1. In providing code samples. For example getting code examples for wazero.
2. In generating unit test cases, especially bug reproducers.
3. In research and helping me to shape my vague ideas - similarly to rubber duck debugging.
4. In suggesting me the Pulse and QNX naming, as well as Mailbox and Erlang, or discovering uutils has web assembly build.
5. In generating a lot of plausible sounding overly complicated designs I was able to either reject or simplify.
6. In helping me to debug bugs, crashes and deadlocks I had.
7. In keeping the track about the state of the project.

I haven’t found it a good tool for writing a code, though. Using it on this project has actually made me more skeptical. If you care about the code, you absolutely must write it yourself. LLMs, especially agents, can help you with everything else. But a machine cannot replace the human understanding. The more you know, the better the LLMs become as tools for you.

Writing the code is the best way how to develop a deep understanding.

Logo

Legs, beak and colors from Confused Tux (Public Domain) mixed into gonix.svg.