How hard would it be to replicate the unix pipe in Go?

Given the fact I began experimenting 4 years ago, it might be tempting to say yes. It is hard! However, implementing a Unix command and a pipe itself is trivial in Go. All the necessary ingredients are already present in a standard library.

• pipe is io.Pipe
• the producer and consumer runs concurrently and Go has goroutines

TL;DR;

Let me introduce the project quicky. https://codeberg.org/gonix provides a Unix utilities and scripts as a (Go) programming library. Since the utilities and shell interpreter itself themselves are Go - there’s no fork+execve, no extra syscalls, it is all standard programming in userspace. And thanks to people working on reimplementation of GNU core utilities to Rust GNU compatible coreutils are available right now.

Consider this trivial example of the script.

> gonix/cat /etc/passwd | rust/wc -l
42

While it does look like one and behave like a script, it is all pure Go all the way down.

1. https://github.com/mvdan/sh implements the shell itself.
2. cat is implemented by a gonix project itself. https://codeberg.org/gonix/gonix/src/branch/main/cat
3. wc uses WebAssembly build of https://github.com/uutils/coreutils project.

The names gonix/cat and rust/wc are arbitrary - scripts can be called by any name and these names has been chosen in order to showcase the fact this is not a script in a traditional sense.

Motivation

Despite using Linux for almost two decades, I have never liked the strict separation between scripting and a programming. There is no libgrep or libsed or libawk that combines the higher level tools with a proper programming environment. Projects like libcurl/curl do exist though. Even then - if a script solves the problem by calling a curl, the solution must be written again using interfaces tailored to C or other real programming language.

So while command line tools and scripting languages themselves are written in C (or Go or Rust or Python or Node). Consuming those back as a software package is awkward (see gpgme as an infamous example). This project is an attempt to merge both worlds together and keep scripting useful part and integrated part of a programming.

The cat | wc example above can be implemented using a few lines of Go and all runs in userspace inside the same process without a single additional fork+exec in sight.

// register commands
var cmds = map[string]unix.FilterBuilderFunc{
  "gonix/cat": cat.FromArgs,
  "rust/wc":  coreutils.Wc(),
}
// parse the script
file, _ := syntax.NewParser().
  Parse(strings.NewReader(script), "gonix/cat ${FILE} | rust/wc")

runner, _ := interp.New(
  interp.Env(expand.ListEnviron("FILE=test.me")),
  interp.StdIO(nil, os.Stdout, os.Stdout),
  interp.ExecHandlers(c.ExecHandler),
)
) = runner.Run(context.Background(), file)

Shoulders of giants

The most fascinating aspect of this project is how little code I needed to write. It is also impressive how much I can rely on work of others. The whole entire project contains less 10,500 lines of a code. This is a cool metric, especially in this AI agentic era.

This project would not be possible without the creators of Unix and Go and

• https://github.com/mvdan/sh for an excellent implementation of a shell for Go
• https://github.com/benhoyt/goawk for amazing implementation of awk. The gonix implementation of cat is implemented as awk scripts.
• https://pkg.go.dev/github.com/tetratelabs/wazero which allowed me to use
• https://github.com/uutils/coreutils despite being written in Rust
• https://github.com/u-root/u-root project for having a native implementation of some utilites too.

The unix architecture

Consider this very simple Unix shell pipeline.

> cat /etc/passwd | wc -l
42

In a traditional unix architecture there are many components, which must be there for this to work.

1. unix kernel
2. sh, cat and wc binaries installed
3. kernel starts the sh process
4. sh must open(2) and read(2) the script
5. sh must call pipe(2) to create a pipe
6. sh must call fork(2) for each sub-process
7. sh must call dup(2) connecting the stdout of cat with stdin of wc
8. sh must call execve(2) for both subprocesses
9. sh calls waitpid(2) to get a report from kernel when and how tools ended
10. sh can finally exit(2) and report the status

Do not forget this is simplified version of all the work computer does. It ignores stuff like signal handling and all other stuff, dynamic linking, locales, memory calls like brk and mmap, opening locale and other files and so on. The GNU userspace executed as sh test.sh ran 567 syscalls.

Additionally, a script will only work if these are installed on the target system. This can become a problem if

• You cannot modify the system.
• The required tool is not easily available.
• Your system don’t have latest GNU utilities and all it has are outdated unix tools.
• your system is alien enough, that even if a Unix environment has been ported, this will never feel native.

With gonix shell scripting becomes 100% portable.

Gonix architecture

The core of unix command is the standard io. The C abstraction is called a file descriptor and is represented by a type int.

/* #include<unistd.h> */

/* File number of stdin; 0. */
#define STDIN_FILENO 0
/* File number of stdout; 1. */
#define STDOUT_FILENO 1
/* File number of stderr; 2. */
#define STDERR_FILENO 2

This concept can be easily expressed in Go.

type StandardIO interface {
  Stdin() io.Reader
  Stdout() io.Writer
  Stderr() io.Writer
}

Which is exactly what gio/unix does.

And while each C unix tool starts with a main function

#include <unistd.h>

int main() {
  write(STDOUT_FILENO, "stdout\n", strlen("stdout\n"));
  write(STDERR_FILENO, "stderr\n", strlen("stderr\n"));
}

For our purpose its better to define an interface

type Filter interface {
  Run(context.Context, StandardIO) error
}

and implement it

type Demo struct{}
func(Demo) Run(_ context.Context, stdio unix.StandardIO) error {
  fmt.Fprintf(stdio.Stdout(), "stdout\n")
  fmt.Fprintf(stdio.Stderr(), "stderr\n")
  return nil
}

Example: cat

The cat without any arguments is literary an one-liner with io.Copy doing all the work.

// import codeberg.org/gonix/gio/unix
func (Cat) Run(_ context.Context, stdio unix.StandardIO) error {
  _, err := io.Copy(stdio.Stdout(), stdio.Stdin())
  return err
}

Example: wc -l

The wc -l is a bit more lines, but still not very complicated.

// import codeberg.org/gonix/gio/unix
func (Lines) Run(_ context.Context, stdio unix.StandardIO) error {
  count := 0
  scanner := bufio.NewScanner(stdio.Stdin())
  for scanner.Scan() {
      count++
  }
  err := scanner.Err()
  if err != nil {
    return err
  }
  fmt.Fprintf(stdio.Stdout(), "%d\n", count)
  return nil
}

Command line arguments

Run and unix.Filter interface can satisfy the easiest tools without a command line arguments or any notion of an outer context like working directory. The typical Unix tool expects and receives command line arguments when executed. This is solved by filter builder pattern

func(command string, args []string, opts ...ShellContextOption) (Filter, error)

What way implementation can get more context than context and standard io only.

type CatWithArgs struct {
  command string
  args []string
}

// import codeberg.org/gonix/gio/unix
func (c CatWithArgs) Run(_ context.Context, stdio unix.StandardIO) error {
  if err := c.parse(c.args); err != nil {
    return fmt.Errorf("%s: %w", c.command, err)
  }
  _, err := io.Copy(stdio.Stdout(), stdio.Stdin())
  return err
}

The shell context

Let’s consider this example

FOO=42 printenv

Unix tools expects environment variables too. This is achieved by opts ...ShellContextOption of a builder. Every tool can then get proper functions which can be used to get an access. The key design concern here is to decouple the interface for such functionality with an actual implementation

1. interface is defined in gio/unix
2. injection is a part of unix.FilterBuilderFunc
3. actual implementation if done in sh package for mvdan.cc/sh/v3.

Tools can register those helpers to obtain enough from the context as seen in a following example.

var b uroot.Builder
var builders = unix.FilterLookupMap{
  "mktemp": b.Mktemp(),
}
filters := sh.NewFilters(
  builders.Lookup,
  unix.WithLookupEnvFunc(sh.LookupEnvFunc),
  )

const source = "TMPDIR=/tmp42 mktemp -d t.XXXXXX"
parsed, err := syntax.NewParser().
  Parse(strings.NewReader(source), "")
require.NoError(t, err)

var got bytes.Buffer

runner, err := interp.New(
  interp.StdIO(nil, &got, nil),
  interp.ExecHandlers(filters.ExecHandler),
)
// expect error as /tmp42 does not exists

Available options are

1. get working directory via unix.GetDirFunc
2. lookup environment variable via unix.LookupEnvFunc
3. get all environment via unix.GetEnvFunc

What did the gonix did for us?

Nothing at the moment. However I think this can improve an existing container tooling for example.

FROM scratch

# I need to create a directory
# but can't as mkdir is not there
RUN mkdir /app

or the debugging sessions

# go away: cannot execute /bin/ls: file not found
docker exec scratch ls -lh /app

At this moment all the tools are supposed to be installed inside the system itself. And consider a hypothetical improved tooling

FROM scratch

# mkdir is available as a part of Go runtime
# no need to install a thing
GONIX rust/mkdir /app

# ls is a part of runtime, so rust/ls will work
better-docker exec --gonix 'rust/ls -lh /app' scratch

The other possible use cases are

• And monitoring probes
• Tools which setups systems using scripts like distrobox
• Experimenting with more exotic tools like those written for Plan9
• AI companions writing and executing scripts
• MCP server
• And many more

Roadmap?

There is any as this is a project of love. However the interesting areas to consider are

1. context aware io - at this moment individual filters much implement context checks individually - idea here is to wrap context and io together, so Read/Write methods will stop working once context is canceled
2. busybox-like tool which packs as much other tools as possible
3. interactive shell - being in Go it can have nice things like native fuzzy search

Limitations

There are more aspects to a real Unix system than just pipes and environment variables. Some are fundamental limits of how the kernel works, while others may be implemented later. These limitations include things like

Kernel properties

The project is an emulation of a real Unix system completely bypassing kernel and all its features. So process isolation, process tracking, signals, control groups, security context, SELinux labels, user and groups, extended attributes and so on - nothing does work inside gonix.

Unless the fork+exec is involved - https://codeberg.org/gonix/gio/src/branch/main/unix/exec.go#L30 is available.

In other words gonix is not a security layer. Additionally running inside the same OS process, you will never get SUID binaries like sudo working in userspace.

Can’t execve by default

Other part are unit tools calling fork+exec themselves like xargs. There is no good solution for a reverse wrapper which will hijack exec.Command and covert it to unix.Filter. So there is GNU compatible implementation of https://codeberg.org/gonix/gonix/tree/main/xargs/ which gets the unix.FilterBuilderFunc as a parameter and can execute other userspace tools.

This design allows setups like the main shell having an access to different filters than xargs itself. This is how it is unit tested by the way.

Signals and process tracking

While this one can be emulated, at the moment individual utilities don’t respect context.Context by default unless specified elsewhere. The design of Go context will most likely allow emulating the signals, but this is not yet done. The relevant part is a process tracking, this is not implemented - if something get stuck, you must kill a whole process.

Interactivity

While interesting, neither mvdan.cc/sh/v3 neither the utilities are developed and tested with interactivity in mind.

Are there any other similar projects?

I haven’t searched thoroughly and those are two examples I am aware of.

https://github.com/ewhauser/gbash seems to be the closest. However it seems to be more AI driven and I expect that most of code there is generated.

The gbash project itself references https://github.com/vercel-labs/just-bash which seems to be the same, only in Typescript.

AI

While code in the project was written by me, I must admit I would not be able to progress so quickly without a help of LLMs.

1. Proof of concept code including wazero integration has been provided by various AI tools as well as a few test cases.
2. The ideas behind unix.ShellContextOption has been investigated by Claude. It was nice to be able to see and reject all the wrong solutions it produced, which helped me to figure out the proper design.
3. Experimental sbase web assembly port has been vibe-coded by Claude.
4. deepl.com/write reviewed this text as usual.
5. Google AI explained me how to use Inkscape to convert the png logo into svg.

Logo

Gopher by Rene French CC license https://upload.wikimedia.org/wikipedia/commons/2/2d/Go_gopher_favicon.svg and Unix Terminal Logo Dollar Akshay

Meanwhile, it was necessary to fork it it due growing needs. Most notably it rely on SCSS, where I want CSS written by hand without any Javascript post-processing. With modernish CSS features like variables supporting the color scheme is easy and this blog is based on gruvbox.

/* https://github.com/morhetz/gruvbox?tab=readme-ov-file#light-mode-1 */
:root {
  --bg: #fbf1c7;
  --red: #cc241d;
  --green: #98971a;
}

Going dark

With CSS variables in place, the dark theme simply redefines variables using prefers-color-scheme: dark @media query.

@media (prefers-color-scheme: dark) {
  :root {
    --bg: #282828;
    --red: #cc241d;
    --green: #98971a;
  }
}

And the page itself can advertise supported schemes and its priority.

<meta name="color-scheme" content="light dark" />

Fixing Mastodon comments

Taking an inspiration from https://carlswchwan.eu, I added support for Mastodon comments. The problem was that I originally reused a CSS from a different blog and didn’t actually used site’s CSS variables. This has been fixed and styling for comments gor merged to main style.css.

/* Mastodon comments */
#load-comment {
  background-color: var(--blue);
  color: var(--bg);
  border: 1px solid var(--blue12);
}

Color scheme based screenshots

One of the coolest things about this exercise is that HTML itself supports @media queries, making it easy to render different images based on a preferred color scheme. This means that page with a light style will display light screenshots. And vice versa.

<picture>
  <source srcset="screenshot-dark.png" media="(prefers-color-scheme: dark)">
  <source srcset="screenshot-light.png" media="(prefers-color-scheme: light)">
  <img src="screenshot-light.png" alt="Miblog screenshot">
</picture>

Color scheme thumbnails

There is another problem on the main index page. Unfortunately, most of article thumbnails were not created with dark theme support in mind. The responsive ByteKit is the first article to feature the support for dark themed thumbnails and dark themed screenshots.

I released version 0.3.0 of a bytekit.app, which is a privacy oriented AGPLv3 Web application for developers that I had been working on.

There are a few new features to highlight:

• AWK, which is based on the impressive goawk package from Ben Hoyt.
• jq, which is based on the pure Go version of jq
• new design, logo and responsive layout and a footer

I must admit that ChatGPT helped me generate ideas for the logo and its color scheme. My minuscule Inkscape skills handled the rest then.

Responsive design

However much bigger change than visual was a responsiveness. Bytekit is intentionally light on elements. I wanted to have something, which just works and to not overwhelm users or myself with UI options. Additionally writing UI elements is not a funniest or most creative part of the work. So once I made elements like working. Well then every other problem can be solved by adding a new input box, right?

However, the most significant change was the responsiveness. Not the visuals. Bytekit intentionally uses only a few elements. I wanted something that just works and to not overwhelm users or myself with UI options. And I must admit that writing UI elements is not the most funnies part of the work. Quite the opposite. So once I made elements like the InputBox work. Then every problem became a nail. Or something input box can solve.

Let’s say the awk page.

1. Command line parameters? InputBox!
2. Program itself? Well InputBox, of course.
3. Input?
4. Output?

You get it. The Input Box has a consistent layout, the copy and paste buttons, can display an error like a wrong AWK program being passed and can be bind to on text changed event providing an interactivity. And as most of the pages are based on this element, this helps a lot with a consistent user experience.

Despite using light HTML or using a CSS framework, the design was not well-suited to the smaller screens. The biggest culprit was the <aside> menu, which got rendered at the top of the screen every time, making the site next to useless on mobile. Fortunately Bulma.css, the CSS framework behind the page, supports a responsiveness.

Menu is twice

The problem is how to ensure that mobile users gets no menu and can show one, while desktop users has their menu always displayed as they do have much free space. The solution was to render the menu twice and use CSS to hide/show it depending on a screen size.

Bigger screens are now defined with is-hidden-touch CSS class, which ensures it gets hidden on a smaller screens.

The menu for smaller screens is inside the navbar and has the class is-hidden-desktop. This means, that by default, the aside and navbar menus are hidden. In order to display it, there is a button which on a click pass the is-active class to the proper element. This is a few lines of Go and this is basically what drives a lot of interactivity of the bytekit itself.

There are still areas to improve, such as the aforementioned InputBox which has too much padding on a mobile devices. In overall the site looks much better now.

Better footer

Last but not least change is the new footer. Previously, it displayed a nerdy sha256 hash to everyone, making it unpleasant and especially on a mobile. This has been changed, it got a name, em dash, short description, links to source code and hashes are hidden for those interested.

Logo for the article is remixed https://github.com/egonelbre/gophers/blob/master/vector/science/power-to-the-linux.svg with https://upload.wikimedia.org/wikipedia/commons/c/cd/IPhone_size_comparison.svg.

It may sound odd to use an online tool for something as sensitive as password. The good thing is that https://bytekit.app/password-generator has the necessary properties to make it trustworthy.

1. It runs inside browser.
2. It does not send data anywhere - this can be verified.
3. It is https://codeberg.org/vyskocilm/bytekit Free Software, so the source code can be inspected.
4. Anyone can deploy the static binary or a container quay.io/vyskocilm/bytekit himself.

How to generate a secure password

In a not-so-good days of enterprise systems a few rules were made mandatory.

1. Passwords must have a minimum length
2. Passwords must contains D digits and S special characters
3. One must change your password every M months
4. The system remembers all previous passwords, so it’ll reject too similar new passwords. This is very secure, indeed.

Xkcd made them obsolete.

In fact the sane rules are

1. Passwords must be stored in a password manager
2. It prevents a phishing attempts among others
3. Passwords must be randomly generated (ie no admin, 12345, or a password or a football)
4. Passwords must not be reused
5. The password manager is typically protected by a master password and this is where xkcd comes into a place.
6. Use https://haveibeenpwned.com/ and https://haveibeenpwned.com/Passwords to verify your email and a password.

Algorithms

So to make the traditional enterprise systems happy as well as readers of xkcd and to enjoy a bit of coding, https://bytekit.app/password-generator generates 4 types of passwords at once.

1. random is password which has uppercases, lowercases, digits and symbols. Password rules are respected.
2. wovel-consontant-wovel is a simplified version of A Random Word Generator For Pronounceable Passwords National Technical Information Service (NTIS) AD-A-017676, which aims to generate memorable passwords. Now is deprecated. The wovel consonant tries to generate readable passwords using a simplified algorithm. In a reality they’re not if a password is long enough. Rules are respected by placing digits and special characters to random places in a generated string.
3. koremutake generated passwords based on algorithm from shorl and apg-go and its predefined syllables. Bytekit randomly switches between all lowecase, all uppercase or first uppercase for each syllable. Rules are respected by placing digits and special characters to random places in a generated string.
4. xkcd - the recommended algorithm for passwords intended to be memorized. The CorrectHorseBatteryStaple from xkcd: Password Strength is an example of great and safe password. Except not this one, because it has been breached 150 times already (lower case variant more than 4000 times) and today it is very likely be a part of a dictionary. Rules related to digits and specials are ignored in this case.

When in doubt use a random option or xkcd if special characters or digits does not matter.

Password Rules

Many sites require passwords to follow specific rules. To avoid implementing many HTML controls, I use a simple algorithm that tries to identify rules in plain English and apply them. The best results are when each rule is on its own line, like the placeholder.

It works by searching for numbers or well known word numbers like zero and then within three words long window some keyword is searched. The “Password must have three digits” will find a number 3 and a keyword digits, so minimal number of digits is set to 3.

The parseRules and TestParse can shed more light to anyone interested.

Introduction

After working with computers for almost two decades, vim’s modal editing is wired into my brain and a muscle memory. I do not consider myself as an advanced user, but yanking/pasting, %, splits, textobjects, macros and a few movements that I use are enough for me to stick with it. Not being forced to use mouse is a nice bonus too.

Enter neovim. As a fork of vim, it inherits all good parts and adds dozens of advantages making it a plausible IDE. Among other is brings tree-sitter, native LSP client and Lua as a scripting language.

Unlike int my favorite language Go, where vim/neovim sits proudly in a 3rd place with 19% of respondents claiming use is. The situation in Java world is most likely different. I could not find any specific numbers for it. However, since its inception, the Java world has been dominated by IDEs. Starting with Netbeans - which to my surprise is still an active project under Apache Foundation - through Eclipse IDE, and, most recently Intellij, probably the most popular IDE today. Then there are Spring Tools Suite, JBoss Developer studio as well as a Visual Studio Code. Java has never been short of IDEs.

Neovim plugin(s)

When I’m checking out anew programming language, I check out the nvim-lspconfig first. However, in the case of Java, this is the wrong place to look.

mfussenegger/nvim-jdtls is the plugin I used. It integrates neovim with Eclipse JDT LS. Yes, that is is the same Eclipse, that produces the IDE mentioned earlier. There is also nvim-java/nvim-java too, which I haven’t tested. The nvim-java project itself says

nvim-jdtls is a plugin that follows “Keep it simple, stupid!” approach. If you love customizing things by yourself, then give nvim-jdtls a try.

Which, to be honest, haven’t shed any lights whenever I want to use this or that plugin.

Installation

The installation string is a bit longer, because project is hosted on codeberg.

{
  "mfussenegger/nvim-jdtls",
  url = "https://codeberg.org/mfussenegger/nvim-jdtls",
  ft = "java",
}

Then configure it and enable.

vim.lsp.config("jdtls", {capabilities = capabilities,})
vim.lsp.enable("jdtls")

And that’s all folks. Or isn’t?

Project Lombok

A typical Java class has a few things: the private attributes, constructor and the infamous getter and setter methods. Unlike in Go, making the class attributes public is considered a bad style. However as both languages emphasizes interfaces, you can’t place an attribute to interface defintion - ignoring final static constants in Java. So having getter setters makes a sense.

So here is the typical Java code

public class Foo {
  private int bar;

  public Foo(int bar) {
    this.bar = bar;
  }

  public int getBar() {
    return bar;
  }

  public void setBar(int bar) {
    this.bar = bar;
  }
}

The getter/setter code is very repetitive. The original Java solution was to use IDE to generate the getter and setter code. Project lombok automates the whole thing via plugging into JVM and via special feature called annotation it generates needed methods in a background.

import lombok.AllArgsConstructor;
import lombok.Data;

@Data
@AllArgsConstructor
public class Foo {
  private int bar;
}

It turns out the nvim-jdtls does not support this out of the box, so LSP compaints all the time about missing methods.

The HOWTO

The TL;DR answer is to check jdtls.lua in my dotfiles. It contains complete setup for it.

I do not want to use Mason, which can automate a lot of LSP related downloads. So the process is a bit manual.

1. download the lombok.jar and place it somewhere
2. define lombok_path variable pointing to the lombok.jar
3. pass it as "-javaagent:" .. lombok_path to the configuration.

And that’s it.

I used the Lua variable lombok_path to debug issues I was facing while configuring neovim. Using the variable made it easier to write a proper notification. So now, when I open a Java project, the confirmation about properly found lombok.jar properly appears at the top of the messages.

local notify = vim.notify
local ok, fidget = pcall(require, "fidget")
if ok then
  notify = fidget.notify
end

-- Verify lombok exists
if vim.fn.filereadable(lombok_path) == 0 then
  notify("Lombok jar not found at: " .. lombok_path, vim.log.levels.ERROR)
else
  notify("Lombok jar found at: " .. lombok_path, vim.log.levels.INFO)
end

Acknowledgments

The following images has been hurt during the logo creation process. And ChatGPT has been used to sketch the original idea.

• neovim.svg
• java-icon.svg
• project-lombook-avatar

Bytekit is written in Go. That might sound an odd choice for a browser app, but Go itself targets WebAssembly. Using GOARCH=wasm together with go-app let me run Go in the browser.

The motivation

I mostly use the command line, but I still rely on online tools from time to time. Many are available, yet most have annoying issues:

• no privacy - “we and 635 trustworthy partners swear we won’t share your data”
• dubious security - is data processing done client-side or server-side? Are results shared with third parties?
• no consistency - each tool uses a different UI, and some need extra clicks for simple actions
• no source code

My main motivation was a better JWT decoder. I personally find the UI of https://jwt.iot as confusing with regards to verifying of signatures. So I wanted to have a cleaner and simpler interface. As a GNOME user, I like the Adwaita and GNOME HIG in general. So the goal was to have something similar to Dev Toolbox only on web.

1. Security first

Developer tools often handle sensitive data like auth tokens and API keys. Sending that data through a web app can be risky. Bytekit runs entirely in the browser

• nothing is ever sent outside your computer. You can verify this easily with your browser’s developer tools.

2. Privacy

The Privacy policy for https://bytekit.app is simple. I run the service alone - there are no partners, no trackers, no ads and no analytics. The HTTP requests are sent to journald and are placed to 100MB big in-memory ring buffer together with all other services. I inspect them manually from time to time for a debugging purposes.

The site links to its source code hosted on Codeberg and the it runs from quay.io/vyskocilm/bytekit container image. So self-hosting is always an option.

3. Open source / Free software

I chose AGPLv3 for the main app because I want to encourage people to self-host, reuse, or improve the project. At the same time, the license asks that any changes to be shared back with the community.

4. Easy to use

I avoided forcing clicks for simple tasks. The app runs locally in your browser and processes data as you type. The UI is meant to be clear and minimal — no unnecessary bells and whistles. One example is the hash/checksum page. Instead of making you choose an algorithm, it computes them all at once.

Using Go on a frontend?

Let’s start with an elephant in the room first. Which is the artifacts size.

$ ls -lh public/web/*.wasm*
-rw-r--r--. 1 michal michal 9.1M Jan  7 16:14 public/web/app.wasm
-rw-r--r--. 1 michal michal 2.4M Jan  7 16:14 public/web/app.wasm.gz
-rw-r--r--. 1 michal michal 2.1M Jan  7 16:14 public/web/app.wasm.zst

For a perspective, a main Javascript file from YouTube says 1,835kB (resource size 9,446 kB). So given the fact this comes with full Go runtime, I consider this as pretty small and acceptable. The wasm binary and other assets are compressed to save bandwidth. Clients request the compressed version via Content Negotiation (Accept-Encoding). Browsers do this by default, so users get the smaller transfer automatically.

The server actually refuses to serve an uncompressed wasm file.

The application itself is just a directory of a static assets inside a public/ folder. I also included optional Go net/http server to serve those files. That server has a few advantages over serving static files:

• it is compatible with OCI containers and can be placed behind a reverse proxy
• use of embed means the app is a single executable which works from memory
  • it doesn’t need external services, neither a filesystem and can be deployed anywhere
• it handles content negotiation for compressed assets and refuses to serve a plain wasm
• it makes app cache friendly by handling Etags, If-None-Match and Vary HTTP headers
• it ensures a correct Content-Type for all requests
• it has a rewrite rules included - so paths never ends with .html
• it takes a care about Content-Security-Policy headers

How the frontend Go look like?

This is not a tutorial, only a short showcase of a frontent Go. The example is a base64 encoding/decoding page. All UI components are plain Go structs which must embed the framework’s app.Compo.

package base64

import (
  bk "codeberg.org/vyskocilm/bytekit/internal/kit"

  "github.com/maxence-charriere/go-app/v10/pkg/app"
)
type Content struct {
  app.Compo

  showError bk.ToastFunc
  encoded   string
  decoded   string
  encoding  *base64.Encoding
}

To render a content as HTML one must implement the Render method from app.Composer. Those components are HTML components defined by go-app or those can be other structs implementing app.Composer interface. The app defines a few reusable UI components this way, like and InputBox.

func (c *Content) Render() app.UI {
  return app.Div().Body(
    &title{
      showError:  c.showError,
      onSelected: c.onEncodingChanged,
    },
    bk.InputBox().
      SetText(c.encoded).
      ShowError(c.showError).
      Label("Base64 encoded content").
      OnChangedText(c.onEncodedChange),
    bk.InputBox().
      SetText(c.decoded).
      ShowError(c.showError).
      Label("Decoded text").
      OnChangedText(c.onDecodedChange),
    )
}

The interactivity is done via callbacks - the framework calls a defined callback a particular event.

func (c *Content) onDecodedChange(ctx app.Context, decoded string) {
  ctx.Async((func() {
    encoded := c.encoding.EncodeToString([]byte(decoded))
    ctx.Dispatch(func(ctx app.Context) {
      c.encoded = encoded
      c.decoded = decoded
    })
  }))
}

In the browser, JavaScript runs on a single main thread. Heavy computations block that thread and freeze the UI because no other JavaScript can run at the same time. There is a concept of a Worker which lets you run code off the main thread.

In Concurrency model of go-app all things runs in a single UI goroutine which updates the UI and a state of the components. Async then runs the code inside another goroutine, which do not block the main UI. When code is finished running, the Dispatch call passes results safely back to the UI goroutine.

Javascript interop

No frontend code can live without an ability to call Javascript. This is the only way for accessing browser APIs. The app itself exposes some of the DOM API.

  ctx.Async(func() {
    app.Window().Get("navigator").
      Get("clipboard").
      Call("writeText", text)
  })

An access to elements triggering an event is possible too.

  ctx.Async(func() {
    title := ctx.JSSrc().Get("innerText").String()
  })

As well as a support for Promises, which is the only way of reading from a clipboard.

clipboard := app.Window().Get("navigator").
  Get("clipboard")
promise := clipboard.Call("readText")
thenFunc := app.FuncOf(...)
catchFunc := app.FuncOf(...)
promise.Call("then", thenFunc).Call("catch", catchFunc)

The frontend itself is generated by https://go-app.dev via

$ GOARCH=wasm GOOS=js go build -o "public/web/app.wasm" \
codeberg.org/vyskocilm/bytekit/cmd/bytekit

This populates the content of a public folder where all web assets are placed. The go-app framework is the one doing all the magic behind the scenes.

So how’s dev experience without npm?

The npm (pnpm, yarn, …) is pervasive in a FE development world. Not using the standard frontend toolbox means you often ends up solving problems yourself. That said, using npm for parts of the workflow can be reasonable. For example, bytekit uses purgecss to minify bulma.css by removing unused declarations, making assets as lean as possible.

On the other hand sometimes writing a part of a build chain is easy and very convenient. As an example code compressing assets with gzip and zstd turned out to be about 100 lines of straightforward, idiomatic Go code. This is a tradeoff worth making. Especially the build system used, https://magefile.org/, is Go code itself.

So there are gophers all the way down.

Having complete codebase in a single language including the build system makes the developer experience very pleasant. There is a little magic and build logic is only a code inside internal/mage and can be inspected, changed or tested exactly like the rest of the other code. The top level mage.go merely defines a build targets.

And as go supports go tool the build system is a dev dependency listed in go.mod.

tool github.com/magefile/mage

And a CI is a buch of calls of a form go tool mage $target

•  smaller ecosystem to pull from
•  coherent codebase - everything is written in one language
•  speed - once is built, it runs faster than npm tooling

Closing thoughts

Bytekit started as a small experiment in running Go on the web. It stays small and focused on a few useful tools. It’s privacy-first, open source, and easy to self‑host if you prefer.

Try it at [https://bytekit.app]. The source is on Codeberg and the container image is on quay.io, so running your own copy is straightforward. Issues, ideas, and pull requests are welcome.

If you like the approach or find a rough edge, tell me - especially about the UI and privacy parts. Thanks for reading and for trying Bytekit.

Logo is remixed [https://github.com/egonelbre/gophers/blob/master/vector/science/power-to-the-linux.svg].

Problem

Consider this code

package main

import (
  "encoding/json"
  "fmt"
  "net/url"
)

type Foo struct {
  BaseURL *url.URL `json:"base_url"`
}

func main() {
  const src = `{"base_url": "https://example.com"}`
  var aFoo Foo
  err := json.Unmarshal([]byte(src), &aFoo)
  if err != nil {
    panic(err)
  }
  fmt.Printf("%#+v\n", aFoo.BaseURL.String())
}

The standard *url.URL can’t do that. There’s UnmarshalBinary, but no other method available.

panic: json: cannot unmarshal string into Go struct field Foo.base_url of type url.URL

UnmarshalText

json.Unmarshal documents how it works

To unmarshal JSON into a value implementing [Unmarshaler], Unmarshal calls that value’s [Unmarshaler.UnmarshalJSON] method, including when the input is a JSON null. Otherwise, if the value implements [encoding.TextUnmarshaler] and the input is a JSON quoted string, Unmarshal calls [encoding.TextUnmarshaler.UnmarshalText] with the unquoted form of the string.

The only thing needed to be implemented is TextUnmarshaler interface. Because URL is a string in JSON.

type URL struct {
  *url.URL
}

func (u *URL) UnmarshalText(text []byte) error {
  parsed, err := url.Parse(string(text))
  if err != nil {
    return err
  }
  u.URL = parsed
  return nil
}

And the output is "https://example.com".

Marshal it back

When marshaled back to JSON, the URL looks like this: No one wants to write URLs like this.

{
  "u": {
    "Scheme": "https",
    "Opaque": "",
    "User": null,
    "Host": "example.com",
    "Path": "",
    "RawPath": "",
    "OmitHost": false,
    "ForceQuery": false,
    "RawQuery": "",
    "Fragment": "",
    "RawFragment": ""
  }
}

Obviously there is an interface for that

func (u URL) MarshalText() ([]byte, error) {
  if u.URL == nil {
    return []byte{}, nil
  }
  return []byte(u.URL.String()), nil
}

{"base_url":"https://example.com"}

Conclusion

While the text encoding methods are somewhat less popular than their encoding/json counterparts, they can be still useful.

See complete example on Go playground.

Logo is remixed github.com/egonelbre/gophers and is under CC0 license like an original.

This is a story about how I failed. I am writing this because I did my best to make it work, but it turned out its impossible

At the same time it has a some good hints about user systemd, quadlets, rootless podman and what I managed to get working.

The problem

When it comes to your own stuff, overengineering is always the preferred option, isn’t it? My problem was fairly simple. After finishing a Markdown document in neovim, I have to run hugo, install it, or find a (distrobox)[https://distrobox.it/] container with it installed. Then, I need to push the source and built content to the Codeberg repository. After that, I need to log into the server, switch to the user, and run git pull to finally publish the changes.

Except for the git, this is a pretty 90s approach. And as a popular saying goes

Developer will rather spent 3 days writing script* rather than doing the same 3 minutes of manual work twice.

Which is again pretty 90s, isn’t it? Nowadays developers would spent 3 months deploying a multi-cloud multi-region k8s cluster with Terraform, Helm, ArgoCD and dozens of microservices than writing a script.

The idea

The solution exists and is called CI/CD. Because Codeberg itself is a public instance of a software called Forgejo, the most natural way to do it is with actions powered by forgejo-runner. There are others who successfully did that.

Here I wanted to mention that both forgejo-runner and codeberg are wonderful tools, which has nothing to do with why I failed.

So the idea is to build roughly this.

1. On every push, Hugo Build translates the content.
2. Things will be packaged into an OCI container and pushed to an internal repository.
3. The blog itself will be an OCI container running behind Caddy.
4. The whole server will be behind the front-end Caddy, which will be in charge of logging, TLS, and certificate management.

It’s pretty standard, isn’t it? Given the state of the industry in 2025, it doesn’t sound too complicated.

The container curse

I must say that I like containers. When used properly, they’re amazing. For example the openSUSE cnf. This is a Rust tool built inside ubuntu-latest image. In order to test that it works on openSUSE itself, the resulting binary is mounted to a tested openSUSE Tumbleweed container. This allows for a fast integration test using a real system as well as testing of all supported scenarios like zypper, dnf5 or dnf.

What I don’t like is Docker, specifically dockerd, and the fact that it runs as root. While I understand why it is like that. And after this saga I understand it more than ever. Its simplicity is what made it so popular in the first place. Docker like dockerd is a great solution for developers. Not for my server. This brings me to the real curse of containers: rootless ones.

Specifically, I dislike dockerd because it runs as root. I understand why it is like that, though. After this saga, I understand it more than ever. Its simplicity is what made it popular in the first place. Docker, the all might dockerd, is a great solution for developers. But not for my server. This brings me to the real curse of containers: rootless ones.

It’s 2025, and I must say, you can really go far with them. Most of their limitations are in the form of /etc/subuid allocations, and sometimes one encounters a strange permission error. However, it’s much better than it was in 2020 when I was experimenting with them.

How to setup a rootless container

1. create dedicated user

Create a new dedicated system user; systemd-sysuser is probably the best option nowadays.

> cat /etc/sysusers.d/forgejo.conf
#Type Name    ID   GECOS               Home directory  Shell
u     actions -    "forgejo actions"   /home/actions   /sbin/nologin
r     -       1000-1999
m     actions systemd-journal

For rootless Podman containers, it is best to start an user instance via systemd so that the containers will be managed by it like a regular services. Note that systemd-sysuser does not create a home directory.

# Create new user
> systemd-sysusers /etc/sysusers.d/forgejo.conf
# Run systemd's user instance for a newly user
> systemctl start user@1999.service

Quadlet is a Red Hat technology that integrates Podman with a systemd. By writing a .container file that describes all of its properties, quadlet generates an appropriate systemd unit.

This means that forgejo runner itself runs as a container. To do its job, it needs a “docker” socket. In this case it is podman socket of a actions user mounted to the proper place. This allows runner to pull and execute other containers without having a chance to obtain root privileges.

This setup mounts /home/actions/runner into the runner container, ensuring cache is saved between runs.

[Unit]
Description=Forgejo runner

[Container]
Image=code.forgejo.org/forgejo/runner:11
ContainerName=forgejo-runner
User=0
Group=0
Environment=DOCKER_HOST=unix:///var/run/docker.sock
Exec=forgejo-runner --config /data/config/config.yml daemon
Volume=%h/runner:/data:Z
Volume=/run/user/%U/podman/podman.sock:/var/run/docker.sock:rw
Volume=/etc/localtime:/etc/localtime:ro
AutoUpdate=registry

[Service]
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=default.target

# Check actions user services
> systemctl --machine actions@.host --user status forgejo-runner
● forgejo-runner.service - Forgejo runner
     Loaded: loaded (/home/actions/.config/containers/systemd/forgejo-runner.container; generated)
     Active: active (running) since Thu 2025-10-23 18:37:46 CEST; 1h 35min ago

SElinux intermezzo

In the grand scheme of things, this was really a minor annoyance. I got a “permission denied” on /var/run/docker.sock inside the container. The problem was that openSUSE Leap default policy don’t allow for container_t to use connectto.

$ ausearch -m AVC -ts recent

type=AVC msg=audit(1761167553.886:1210): avc:  denied  { connectto } for
pid=34382 comm="forgejo-runner" path="/run/user/1999/podman/podman.sock"
scontext=system_u:system_r:container_t:s0:c167,c401
tcontext=unconfined_u:unconfined_r:container_runtime_t:s0-s0:c0.c1023
tclass=unix_stream_socket permissive=0

> ausearch -m AVC -ts recent | audit2allow -M forgejorunner
> semodule -i forgejorunner.pp

module forgejorunner 1.0;

require {
        type container_t;
        type container_runtime_t;
        class unix_stream_socket connectto;
}

#============= container_t ==============
allow container_t container_runtime_t:unix_stream_socket connectto;

Running own registry

So far I was making a good progress. The runner was registered against Codeberg and was running a printf just fine. The second problem was decifing to store the containers with blog. I did not want to store them in any external registry. They are private artifact of the server itself, so why expose them?

As I got confident with systemd, quadlets and container, I deployed another container file under registry user.

[Unit]
Description=Zot containers registry

[Container]
Image=ghcr.io/project-zot/zot-linux-amd64:latest
ContainerName=zotregistry
User=0
Group=0
Exec=serve /data/config/config.json
Volume=%h/registry:/data:Z
Volume=/etc/localtime:/etc/localtime:ro
PublishPort=127.0.0.1:2999:2999
AutoUpdate=registry

[Service]
Restart=on-failure
RestartSec=30s

[Install]
WantedBy=default.target

Long story short. It is a good idea that does not work. In a hindsight, it makes a sense. You can’t connect two rootless Podmans running under a different user accounts. Not without doing their networking equal to host. Which kinda makes no sense. Why go through all the troubles isolating things and then let everything bind everywhere?

One simple alternative would be to expose the registry under a public name, such as registry.vyskocil.me. The container was exposed to the (localhost)host. However, I never wanted to deal with yet another public service. And the intent was always to keep the registry internal to the VM.

The easiest solution was to move registry under actions user and to define a Podman network.

> cat /home/actions/.config/containers/systemd/ci-net.network
[Network]
Driver=bridge

Add Network=ci-net.network to both container files. And a started containers must have an access to the same network. As this is managed by systemd it has a prefix.

container:
  network: "systemd-ci-net"

The dead end

I must admit that, during this phase, I thought all the difficult problems had been solved. Permissions, SELinux, and networking were all configured and set up. However, item 3 of my plan turned out to be a show stopper.

You cannot build a container. Inside a container. Period! Altough there seems to be workarounds like gcr.io/kaniko-project/executor:latest they always require elevated privileges like access to /dev/fuse for overlayfs and possibly the --priviledged flag to gain more capabilities. I was not comfortable doing this. The purpose of using separate user accounts and a rootless Podman containers was to make the system more secure and less susceptible to breaches.

Sync like its 1996

So, how did I solve the problem? rsync! Since runner runs on my own host, I can mount the host’s file system write the results there.

container:
  image: docker.io/library/node:22-bookworm
  options: >-
    -v /home/actions/runner/public:/public:Z

  steps:
  - name: Trigger a sync
    run: touch /public/.ready-for-sync

Or rather 2010?

Then how are we going to sync files? systemd comes to the rescue. The easiect way is to have unit watching .ready-for-sync and run the rsync script. It runs as a root as it needs to adapt the privileges once more. However systemd itself is great because t provides a lot of tunables that limit the permissions to an absolute minimum.

[Service]
# Hardening
ProtectSystem=strict
ProtectHome=read-only
BindReadOnlyPaths=/home/actions/runner/public
ReadWritePaths=/run/lock/ /srv/www...
InaccessiblePaths=/root
PrivateTmp=yes
PrivateMounts=yes
NoNewPrivileges=yes
ProtectControlGroups=yes
...

Conclusion

Altough I didn’t reach my original goal, I can still call this journey as a success. At least I can uninstall git from VM and hugo from laptop and can enjoy at least a partially modern, fully automated setup.

Codeberg/Forgejo and Forgejo-runner are amazing pieces of a software. I was surprised that I could deploy and run the software successfully, even with rather unusual setup.

And rootless pPdman containers in 2025? They are super cool. The integration to systemd via quadlets is amazing. I feel good knowing they are limited by real Unix privileges and not by namespace/container magic.

However as you give up all the privileges, you can’t build a new OCI container.

Last remark

This is dedicated to all the script kiddies bombing my sshd with ubuntu, admin, wordpress and other non sense accounts. Good luck with actions!

> getent passwd actions
actions:x:1999:1999:forgejo actions:/home/actions:/sbin/nologin

After some time, I realized that this is essentially how protobuf’s oneof is implemented by protoc and protoc-gen-go. Surprisingly, the naming convention it uses is almost identical to mine. This protobuf definition

message Msg {
  oneof payload {
    A a = 1;
    B b = 2;
  }
}

is implemented using nearly the same naming convention.

type isMsg_Payload interface {
	isMsg_Payload()
}

oneoflint

Welcome oneoflint. This linter matches protobuf’s oneof pattern and provides an exhaustive checks for code implementing the protobuf.

Because the naming convention is almost the same, the two linters shares the same code (and bugs). They are exposed as two binaries, so everyone can be opinionated and there is not configuration. It is exactly how a proper Go tooling should be.

var Sum = &analysis.Analyzer{
	Name: "sumlint",
	Run:  analyzer{prefix: "Sum"}.run,
}

var Oneof = &analysis.Analyzer{
	Name: "oneoflint",
	Run:  analyzer{prefix: "is"}.run,

Be honest

There is no configuration at all. Be honest! I am being honest! So what about CLI arguments? Okay, there is a verbose flag.

When developing a linter, there’s nothing worse than starting with an empty output when it was supposed to work. The worst part is that, due to the clever caching of the Go toolchain, you may see your tool doing something, but then nothing at the subsequent step. The cache simply assumes that linters are bug-free, but that’s not the case during development. After some back-and-forth with Copilot and the package documentation, I figured out how to properly add a flag.

This defines the flag.FlagSet. Each one has a proper name so that each linter can have its own configuration.

func init() {
	var verboseFlag bool

	fs1 := flag.NewFlagSet(Sum.Name, flag.ExitOnError)
	fs1.BoolVar(&verboseFlag, "verbose", false, "enable verbose output")
	Sum.Flags = *fs1
	fs2 := flag.NewFlagSet(Oneof.Name, flag.ExitOnError)
	fs2.BoolVar(&verboseFlag, "verbose", false, "enable verbose output")
	Oneof.Flags = *fs2
}

Detected shared state anomaly: both FlagSet instances bind to identical bool storage (verboseFlag)

This is a valid comment from our AI overlord. In fact I asked it to output it in less human form for an effect. In this case the value is accessed using Lookup method and not by reading the variable.

Usage on a command line is simple. The FlagSet name is a prefix of an argument.

$ go vet -vettool=/path/to/sumlint -sumlint.verbose .

Unit tests

Unit testing the code with golang.org/x/tools/go/analysis/analysisTest proved challenging. To use it, you need to put the Go source code into testdata/src/$package and let the analyzer compare the results with the expected data.

This is great as long as it works. However, if the data does not match, analysistest becomes very difficult to understand.

First, you need to assert the list of exported Facts. In the case of oneoflint, this is a list of structs that implement the isMsg_Payload interface.

The second assertion is the error message that is supposed to be printed by the linter. Which gave this test case.

package one_of

type isMsg_Payload interface { //want isMsg_Payload:`one_of\.Msg_A,one_of\.Msg_B`
	isMsg_Payload()
}

type Msg_A struct {
	A *A `protobuf:"bytes,1,opt,name=a,proto3,oneof"`
}

type Msg_B struct {
	B *B `protobuf:"bytes,2,opt,name=b,proto3,oneof"`
}

func (*Msg_A) isMsg_Payload() {}

func (*Msg_B) isMsg_Payload() {}

func process(msg *Msg) {
	switch msg.GetPayload().(type) { // want `non-exhaustive type switch on isMsg_Payload: missing cases for: one_of.Msg_B`
	case *Msg_A:
	default:
	}

Integration testing

To my surprise unit testing was not enough. While sumlint worked like a charm, the oneoflint failed and printed nothing. This is when the verbose flag goes handy. It allows to print just enough state of a linter to be able to diagnose a problem.

Much to my surprise, unit testing was not enough. sumlint worked like a charm, but oneoflint failed and printed nothing. This is when the verbose flag comes in handy. It is supposed to print enough information about the linter to diagnose a problem.

With enough logs, Copilot was able to suggest a fix. The issue was related to the fact that the proto interface was not exported, and Copilot suggested adding a new pass for this case.

However, my trust in unit tests was shaken. The first problem I had with this approach was that the analyzer test could not load the external package. The source code for the unit test was generated by protoc, but due to its inability to import non-standard library packages, it had to be manually changed. Of course, not being able to check a real world code is not good.

The github.com/gomoni/sumlint/test package contains realistic test data with a file generated by protoc. The test file then imports real Go code, tests calls to go vet via os/exec. And compares the results.

package test

import "github.com/gomoni/sumlint/test/one_of"

type oneofTest struct{}

func (oneofTest) good(msg *one_of.Msg) {
	switch msg.GetPayload().(type) {
	case *one_of.Msg_A:
	case *one_of.Msg_B:
	default:
	}
}

func (oneofTest) noDefault(msg *one_of.Msg) {
	switch msg.GetPayload().(type) {
	case *one_of.Msg_A:
	case *one_of.Msg_B:
	}
}

func (oneofTest) noB(msg *one_of.Msg) {
	payload := msg.GetPayload()
	switch payload.(type) {
	case *one_of.Msg_A:
	default:
	}
}

In other words, integration test

1. build linters
2. cd test
3. run go vet -vettool=/path/to/oneoflint .
4. compare the stderr with expected output

$ (cd test; go vet -vettool=/path/to/oneoflint .)
# github.com/gomoni/sumlint/test
./oneof.go:16:2: missing default case on isMsg_Payload: code cannot handle nil interface
./oneof.go:24:2: non-exhaustive type switch on isMsg_Payload: missing cases for: github.com/gomoni/sumlint/test/one_of.Msg_B

Introduction

This concept exists in many functional languages. Take Elm, for example

import Html

type Shape
    = Circle Float
    | Point

area : Shape -> Float
area shape =
    case shape of
        Circle r ->
            pi * r * r
        Point ->
            0

main =
  let shape = Point
  in
  area shape
    |> Debug.toString
    |> Html.text

In this case, an idiomatic Go implementation would be straightforward.

type Areaer interface {
  Area() float64
}

type Point struct {}
func (p Point) Area() float64 {
  return 0
}

Consider creating a new case in Elm.

type Shape
    = Circle Float
    | Point
    | Rectangle Float Float

This results in a compile error that leads the developer to all places where this case must be handled. Additionally, using interfaces can be impractical when the underlying structs have different layouts.

This case does not account for all possibilities.

10|>    case shape of
11|>        Circle r ->
12|>            pi * r * r
13|>        Point ->
14|>            0


Missing possibilities include:

What have Go ever done for us?

Interfaces? Generics? Type switches?

All right, but apart of interfaces, generics and type switches?

Iterators?

Iterators? Oh shut up!

Go itself seems to be very close to having sum types. The following code is valid Go code that looks almost exactly like Elm examples.

type Shape interface {
  Circle | Point
}

Excellent proposal, sir! There are only two problems. The first is this is a generic type constraint only. And the second is this is only a generic type constraint.

Technically this is a single problem, but I considered it to be so important, that I mentioned it twice!

In other words, if you declare a variable of type Shape, the Go compiler will complain.

cannot use type Shape outside a type constraint: interface contains type constraints

However, since Go 1, there has been a nearly perfect construct in Go: a type switch. However, as this is Go, there is no way to perform advanced pattern matching; a switch can be based on types or values, but not in a single statement.

var shape Shape
var area float64
switch x.(Shape) {
case Point:
  area = 0.0
case Circle:
  area = math.Pi * x.R * x.R
}

Except for exhaustiveness. The Go compiler is unaware of which structs implement which interfaces, so adding a new case has no impact. The code will compile and crash at runtime. While possible, type switches are not idiomatic Go.

BurntSushi/go-sumtype

I am not the first person to try to solve this problem.Take https://github.com/BurntSushi/go-sumtype as an example. BurntSushi is a Rust developer and the author of ripgrep, among other projects. So it’s not surprising that he’s interested in having at least one type of sum type available in Go.

//go-sumtype:decl MySumType
type MySumType interface {
        sealed()
}

$ go-sumtype mysumtype.go
mysumtype.go:18:2: exhaustiveness check failed for sum type 'MySumType': missing cases f

Can we do it better?

The Rust code looks like this, and this approach is considered normal and idiomatic.

#[derive(Debug)]
struct Rectangle {
    width: u32,
    height: u32,
}

Not in Go. The magic comments commonly seen in Go source code are directives of the compiler itself. Such as go:embed, go:build, go:tool. The only exception is nolint, which is used to turn off false positives of a linter.

In Go world, there is actually a great precedent. The Test, Bench, and Fuzz prefixes are merely conventions that have an actual meaning for testing code. So let’s have a linter that implements the following simple convention

// SumFoo declares a sum type, which is recognized by sumlint
type SumFoo interface {
	sumFoo()
}

The rules are simple.

The Sum interface must be public and have Sum as a prefix.

It must implement a private method with the same name and the sum prefix.

Each definition of this type is considered a sum type. Sumlint detects all structs that implement the interface and enforces exhaustiveness in type switches. No magic comments are needed, as the special status is visible from the type name itself.

An example

// SumFoo declares a sum type, which is recognized by sumlint
type SumFoo interface {
	sumFoo()
}

// A is an implementation of a SumFoo
type A struct{}
func (A) sumFoo() {}

// B is an implementation of a SumFoo
type B struct{}
func (B) sumFoo() {}

There are three cases. The first one is ideal because all cases are covered. Unlike in other languages, Go requires a default case to handle the nil interface case. This particular property may clash with a nilness or nilaway analyzers.

func good(x SumFoo) {
	switch x.(type) {
	case A, B:
	default:
	}
}

Removing B case leads to an error.

// missing B
func noB(x SumFoo) {
	switch x.(type) {
	case A:
	default:
	}
}

go vet -vettool=${HOME}/go/bin/sumlint .
./src.go:29:2: non-exhaustive type switch on SumFoo: missing cases for: github.com/gomoni/sumlint/tests.B

As well as a missing default. This case is intended to handle the nil interface value, which can’t be easily work-arounded in a Go.

// missing default, is reported
func noDefault(x SumFoo) {
	switch x.(type) {
	case A, B:
	}
}

go vet -vettool=${HOME}/go/bin/sumlint .
./src.go:23:2: missing default case on SumFoo: code cannot handle nil interface

Edited on 2021-10-18: fix the grammar, improved wording.

I rarely write content here. Perhaps I should work on making writing a habit. The blog itself does uses hugo to generate the HTML from Markdown input. This works well. However. Hugo is huge big, super flexible and makes incompatible changes from time to time.

ERROR deprecated: .Site.Author was deprecated in Hugo v0.124.0 and subsequently
removed. Implement taxonomy 'author' or use .Site.Params.Author instead.

As I no expert on Hugo and only use it occasionally, I must admit that such errors do not help me much. However, it turns out that this is a problem that can be solver with AI. And in fact ChatGPT provided me with some useful answers regaring taxonomy and .Site.Params.Author.

.Site.Author

I fixed an error in config.toml by simply moving data around, because taxonomy is not needed for a single user blog.

- [author]
-   author = "Michal Vyskocil"
-   profile = "I am software developer and a former field manager with 12+ years of experience, Prague, Czech Republic"

[Params]
+   author = "Michal Vyskocil"
+   profile = "I am software developer and a former field manager with 12+ years of experience, Prague, Czech Republic"

Modifying the config.toml is only a half of the story. The way the Markdown is rendered is the responsibility of the used template. This also needs to be updated. This blog uses (miblog-hestia-pure)[https://codeberg.org/vyskocilm/miblog-hestia-pure], which is a fork of https://github.com/diwao/hestia-pure. Therefore with the change above, updating the template was straightforward.

  <div class="author__info pure-u-4-5 pure-u-md-7-8">
-   {{ if and (.Site.Author.author) (ne .Site.Author.author "") }}
-   <h3>{{ .Site.Author.author }}</h3>
+   {{ if and (.Site.Params.author) (ne .Site.Params.author "") }}
+   <h3>{{ .Site.Params.author }}</h3>
    {{ else }}

Truncate output

Hugo added a new divider for summaries, which is HTML comment `

`. This broke the index.html and the RSS output. I only realised this once I put the blog online 🤦. As I had no intention of adding the marker to all the articles I had already written, the solution was to truncate each article to 280 characters.

-   <p >{{ if .IsPage }}{{ .Summary }}...{{ else }}{{ .Site.Params.description }}{{ end }}</p>
+   <p >{{ if .IsPage }}{{ .Summary | plainify | truncate 280 }}...{{ else }}{{ .Site.Params.description }}{{ end }}</p>

Fixing RSS

Changing the output of RSS was more work. This was because I basically had to fork the default RSS template. And then I had to adapt it to my needs.

The installation

Is easy with lazy.nvim.

{
  'saghen/blink.cmp',
  -- optional: provides snippets for the snippet source
  dependencies = { 'rafamadriz/friendly-snippets' },

  -- use a release tag to download pre-built binaries
  version = '1.*',
}

The configuration

While my complete configuration file blink.lua is a big one, the plugin is actually pretty easy to install and start with. What is really nice is that it offers a lot of completions by default. The "lsp", "buffer", "path" are all builtin. The "snippets" too, just require an external plugin with defined snippets.

What I really like are the defaults. The keymap = { preset = "default" } uses the ['<C-y>'] = { 'select_and_accept' }, so Ctrl+y accepts the completion as every vim nerd expects.

However when needed, it is easy to configure the plugin to the linking. So I enabled documentation to be opened by default or to enable ghost_text so the completion will be visible as a virtual text in the buffer. The documentation is simply amazing.

What really amazed me was a menu-draw configuration. What was problem and I did not appreciated was it does not show the completion source by default. The configuration was pretty straightforward. First add a source_name into columns.

columns = { { "kind_icon" }, { "label", "label_description", gap = 1 }, { "source_name" } },

And then define how the source_name will look like. This define max width, the highlight class and most importantly the content of the field like [LSP] or [Buffer] or [Copilot].

components = {
	source_name = {
		width = { max = 30 },
		text = function(ctx)
			return "[" .. ctx.source_name .. "]"
		end,
		highlight = "BlinkCmpSource",
	},
},

Support for snippets is builtin, given the external plugin is installed. Either luasnip or mini.snippets are supported.

snippets = { preset = "luasnip" },

And besides documentation blink has a completion for function signatures, so it can provide a context help when using a function too.

signature = { enabled = true },

To Rust or not to Rust

The name blink does refer to the speed plugin. So except native engine written in Lua, there is a Rust based one claiming a significant speedup. The Rust based frizbee is a SIMD fuzzy matcher which achieves ~6x the performance of fzf while ignoring typos.

1. lua
2. rust

I ended up using fuzzy = { implementation = "lua" }, which is the one Kickstart offers. It worked well for me - except the emoji and nerdfont providers. But more specifically I do not want neovim plugins to download and install random binaries from the Internet. And that’s sadly the only one option for a frizbee at the moment.

So it is not surprising that I ended up with 37 plugins installed and loaded every time I run nvim. While none of the plugins are particularly heavy and the startup time is below 100 ms. It is still interesting to find out why is the package manager is named Lazy.

Make Lazy.nvim lazy again

folke/lazy.nvim already advertises the lazy loading feature

• 🔌 Automatic lazy-loading of Lua modules and lazy-loading on events, commands, filetypes, and key mappings

The problem is that most of the configuration examples do not care. After all, loading on startup doesn’t break anything, so it is a good default after all. This leaves the exercise of lazily loading stuff to the curious user.

Lazy itself provides all the necessary hints. Just type :Lazy and check the output. In my case nvim loads 5 mandatory plugins, tht can’t be postponed without breaking the editor.

  Total: 37 plugins

  Loaded (5)
    ● gruvbox 3.57ms  start
    ● lazy.nvim 5.56ms  init.lua
    ● lualine.nvim 6.19ms  VeryLazy
    ● nvim-lastplace 0.57ms  start
    ● vim-sleuth 0.47ms  start

VeryLazy

That is actually a cool name. In short, lazy allows you to start a plugin in a case of an event. VeryLazy is specific one. Documentation says

VeryLazy: triggered after LazyDone and processing VimEnter auto commands

VeryLazy plugins are not counted in the startup time. Just press P in :Lazy view to compare lualine.nvim 3.82ms  VeryLazy

  Startuptime: 36.28ms

  Based on the actual CPU time of the Neovim process till UIEnter.
  This is more accurate than `nvim --startuptime`.
    LazyStart 14.69ms
    LazyDone  28.89ms (+14.2ms)
    UIEnter   36.28ms (+7.39ms)


    ●  startuptime 36.28ms
    ●  VeryLazy 5.33ms
      ➜  lualine.nvim 4.06ms

And as lualine.nvim 4.11ms  start

  Startuptime: 47.56ms

  Based on the actual CPU time of the Neovim process till UIEnter.
  This is more accurate than `nvim --startuptime`.
    LazyStart 12.86ms
    LazyDone  37.18ms (+24.32ms)
    UIEnter   47.56ms (+10.38ms)

    ●  startuptime 47.56ms
    ●  VeryLazy 1.56ms

VeryLazy is then the simplest optimization. But the plugin will still load on every startup. So it is indeed a simple but lazy optimization. With a cool name.

Events

Lazy Loading offers an excellent place to start. Unfortunately a bit short on examples.

event: Lazy-load on event. Events can be specified as BufEnter or with a pattern like BufEnter *.lua

The value can be a string, an array of strings, or a function or a patterns that allows you to start the plugin after some action. Here are gitsigns and nvim-lint plugins configured to get loaded after a buffer is read. As neither is language dependent, there isn’t much to do.

  ○ gitsigns.nvim  BufReadPost 
  ○ nvim-lint  BufReadPost  InsertLeave  BufWritePost 

~/.config/nvim/lua/plugins/gitsigns.lua contains

return {
  "lewis6991/gitsigns.nvim",
  event = { "BufReadPost" },
}

File types

I tend to use a language agnostic tools, so have a lspconfig/neotest/conform combo instead of go.nvim. Nothing against go.nvim, it is a great plugin I use as a base for my configuration. This means that most of the plugins I have can’t be restricted to a file type. There are notable exceptions though.

  ○ kulala.nvim  http 
  ○ lazydev.nvim  lua 
  ○ luvit-meta 

Lazydev is for Lua files, and since it’s a plugin from folke himself, it would be odd to not to suggest a lazy loading by default.

return {
  {
    "folke/lazydev.nvim",
    -- only load on lua files
    ft = "lua",
  },
  { -- optional `vim.uv` typings
    "Bilal2453/luvit-meta",
    lazy = true
  },
}

lazy=true options is handy for library plugins. Such plugin will be loaded when needed by other plugin.

Command

Since I use some commands only via neovim’s command line, there’s nothing easier than loading the plugin after typing it on a command line. This is a case of vim-fugitive and :G command. Lazy registers the command and loads the plugin when it is really needed.

  ○ vim-fugitive  G 

return {
  "tpope/vim-fugitive",
  cmd = "G",
}

Keys

Some other plugins have a defined shortcut(s) to activate. In my case it’s the powerful Telescope plugin. Lazy can also handle this scenario too. The plugin can be activated with a command or a shortcut. This form expects the actual keymap to be activated inside config function of a plugin.

  ○ neotest  <leader>t 
  ○ telescope.nvim  Telescope  <leader>sr  <leader>s.

return {
  {
    "nvim-telescope/telescope.nvim",
    cmd = "Telescope",
    keys = {
      {"<leader><leader>"},
      {"<leader>sh"},
    },
    config = function()
      -- set <leader>sh here
      vim.keymap.set("n", "<leader>sh", ...)
    end
  },
}

Conclusion

Was it worth it? I reduced the startup time of an empty Neovim from 90ms to ~40ms, where opening a Go file increases the time to 80ms. So in terms of numbers it was not. On the other hand, it was cool and fun, and I hope you’ve enjoyed reading this.

Fonts

FiraCode Nerd Font under SIL OPEN FONT LICENSE Version 1.1, converted to woff2 format by cloudconvert.com

• The iter package provides common type definitions.
• The slices package has several functions that work with iterators
• The maps package has several functions that work with iter.Seq2 and maps

Installing a release candidate of Go

Since Go 1.23 has not been released yet, you need to install a release candidate. With recent changes and a GOTOOLCHAIN support, this is easier than ever.

# Given an installed go compiler
$ go version
go version go1.22.5 linux/amd64
# Get the release candidate and set it in a go.mod
$ go get go@1.23rc2 toolchain@1.23rc2
# And make it a default
$ GOTOOLCHAIN="go1.23rc2" go version
go version go1.23rc2 linux/amd64
$ cat go.mod
module github.com/gomoni/it

go 1.23rc2

A simple export GOTOOLCHAIN=go1.23rc2 is all that is needed to make go and a language server and other tools to work with a new release. See the Forward Compatibility and Toolchain Management in Go 1.21 from the Go Blog for more details.

it v0.1.0

Changes to a standard library made most of github.com/gomoni/it obsolete. And it is never a good idea for third-party packages to compete and duplicate it. Surprisingly neither Filter, neither Map helpers got there, making it@v0.1.0 a perfect place for them. To be closer to the standard library these helpers exists in a respective subpackage

• islices provides filter and map on top of iter.Seq
• imaps provides filter and map on top of iter.Seq2

The it itself provides a Chain and Mappable structs providing a chainable API for a helper functions.

package main

import (
	"fmt"
	"slices"

	"github.com/gomoni/it"
)

func main() {
	n := []string{"aa", "aaa", "aaaaaaa", "a"}
	ch := it.NewMapable[string, int](slices.Values(n))
	slice := ch.
		Filter(func(s string) bool { return len(s) >= 2 }).
		Map(func(s string) int { return len(s) }).
		Collect()
	fmt.Println(slice)
}

Request pkg.go.dev upgrade

As soon as the new release v0.1.0 was tagged, pkg.go.dev showed obsolete information. It turns out that there is an easy process for requesting an update.

Let me use a hypothetical v0.1.109 release, to make a link work even for the future releases.

1. Go to the version which does not exists yet (https://pkg.go.dev/github.com/gomoni/it@v0.1.109)
2. Click on a button that says Request “github.com.../it@v0.1.109”
3. Profit!

The trigger

The xz backdoor meant that I immediately nuked the server from the Internet. While MicroOS itself uses SELinux .. the Kubernetes is known for running everything as a root by default.

Leap forward

Frankly speaking I am not a devops guy and Kubernetes is a hell of layers and indirections. I was actually quite surprised when realizing how little vanilla Kubernetes does and how many things you have to get from the broader ecosystem.

Even the Ingress - the way you assign public URLS to your services - is not provided and you must select and learn an external Ingress controller. And configuring it is no fun.

{{ < highlight yaml > }} metadata: annotations: ingress.kubernetes.io/redirect-regex: “^https://vyskocil.org/?(.*)” ingress.kubernetes.io/redirect-replacement: “https://vyskocil.me/$1" {{ < /highlight > }}

And do not try to start with Helm. That is a horrible abstraction, that has become an industry standard for some reason. All in all managing the Kubernetes is a full time job. Which I do not enjoy doing.

Web server

You can use Apache httpd or nginx as your web server. I found the Caddy to be much easier to configure and it comes with some bells and whistles like a built-in acme protocol support, so https via letsencrypt is automatically configured. Setting up this blog was as easy as

vyskocil.me {
        tls email@example.net
	root * /srv/www/htdocs/vyskocil.me/miblog/public/
	encode zstd gzip
	file_server
}

And sudo systemctl reload caddy.service and voila!

curl --head https://vyskocil.me
HTTP/2 200 
accept-ranges: bytes
alt-svc: h3=":443"; ma=2592000
content-type: text/html; charset=utf-8
etag: "sb7i6ejr7"
last-modified: Sun, 31 Mar 2024 09:54:14 GMT
server: Caddy
content-length: 25603
date: Wed, 03 Apr 2024 20:34:05 GMT