Ship GoLang to WebAssembly in Days Using These Techniques

Why Convert Projects to WASM

There are thousands of GoLang projects that are written as CLI tools and forgotten by users. We built our own API client tool called Lama2. It is a file-based API executor implemented in GoLang.

It is working based on Markdown-inspired API language. It supports git-based collaboration. API files having a .l2 extension can be executed by Lama2. Here are some basic valid Lama2 API files:

Get API call:

GET
https://httpbin.org/get

Post API call with parameters:

POST
https://httpbin.org/post
{
    "a": "b",
    "c": "d"
}

We want to build interfaces that help users interact with the tool more efficiently in their browsers, desktops, and mobiles. So we decided to make a WebAssembly (WASM) binary out of it.

WASM is a technology we can use to convert projects in low-level languages like C/C++, and Rust to work on browsers.

Nowadays all browsers support WebAssembly. Top Products like Figma and Google Sheets use WebAssembly and work like a charm.

In this article, I will explain how I converted Lama2 written in GoLang to WebAssembly, and what I learned on the way about GoLang development and WASM.

The Basics of WASM to Get You Started

First, let me explain how WASM works. It's a binary file that is executable from the browser. It's used mainly for the compilation of low-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.

GoLang has full language support for WebAssembly, but it's in the experimental stage.

For compiling the GoLang code to WASM we use GOOS=js and GOARCH=wasm environment variables it will create a new binary with the .wasm extension.

You probably won't get it correctly the first time around because of the errors and issues that I have explained in this article.

Use the syscall/js function and create some functions that can be called from the js code from the browser or whatever framework you use.

In the main function add the function js names and the GoLang code inside js.global.set(). This is how the file will look:

package main

import (
	controller "github.com/HexmosTech/lama2/controller"
    "syscall/js"
)

func main() {
	//Set the global JavaScript property "goWebRequestFunc" to the result of wasmLamaPromise
	js.Global().Set("goWebRequestFunc", wasmLamaPromise())
	js.Global().Set("goCmdConvertFunc", wasmCodeConverter())

	// Block the main function to keep the Go WebAssembly running
	select {}
}

At the end of the main function use the select{} which blocks the current goroutine, which blocks GoLang code from exiting after executing the functions.

Let's create a js function using js.Func as a return type here is an example. If you are working on a function that makes an HTTP request. Your function should return a promise to make it work properly. Here is my previous article on making the HTTP request work.

func wasmCodeConverter() js.Func {
	return js.FuncOf(func(this js.Value, args []js.Value) interface{} {
		inputdata := args[0].String()
		convertLang := args[1].String()
		result, _ := controller.ProcessConverterInput(inputdata,convertLang)
		return result
	})
}

Sea of Issues And Headaches from GoLang Dependencies

The major issue I faced while building was unsupported GoLang packages. While converting to WASM some standard Unix operations and the mpb package were not supported which shows a download progressbar.

To remove the dependency not supported issue we need to use either alternative packages that support WASM or use our own code which is a time-consuming task. I hope GoLang fixes all the dependency issues in the new versions. This is why a lot of GoLang projects are difficult to compile to WASM in a single run.

Another major issue we faced was how to get the API calls working from the browser. We faced CORS errors due to the same-origin policy. For example, while making a GET request to google.com fails with CORS errors. This was solved by using a proxy. Explained in detail this article by my friend Rijul.

How to Maintain Sane Source Code And a Single Source of Truth Using GoLang Build Tags

The next major problem was the isolation of the WASM code. It is a standard practice to let the code have a single source of truth.

If we want to add a new feature we may not need to make changes in two places. We needed everything inside the Lama2 repo. If we have separate places will cause the core implementation to diverge and the code will become a confusion hell for future developers.

Also, Lama2 is a language and it's expected to run at the core in the same way from all platforms. So To solve this we used build tags.

Build tags are represented as comments at the top of the file. Leave a line empty after that. This is how it will look.

//go:build cli

package main

// rest of code

In our case the file has a tag cli, we use it for commandline-based build. In WASM files, we used wasm as the build tag. The wasm build will not consider files with cli tag and vice versa. This helped to separate the different logic and functions and fix the code duplication. In cli there were a lot of file-based operations, in wasm, there were more string-based operations.

We can use -tag= parameter to set the tags in the build command. This is also applicable to running tests.

This is how the build command looks:

go build -tags=cli -o build/l2 

Our main aim for building wasm binary was to use it for a widget format for our Lama2 API docs. We are planning to release it as a service so that people can make API docs interactive. The main two features are running APIs and code conversion to different languages. There are also color themes that are customizable. Do reach out if you are interested in early adoption.

Fixing Painful Library Compatibility for WASM Using a GoLang Workspace

Back to the issue I faced while developing the binary was the httpie-go library. We used a customized htttpie-go repo as the underlying API to make the API calls. This was not WASM compatible, so I had to change its structure to support new changes in Lama2. The build tags were used here also. The main issue here was development was taking time due to the github code pull not working roadblock.

While pushing httpie-go changes to github I am working on my own branch. To reach the new updates into lama2 I need to run:

go get github.com/HexmosTech/httpie-go@wasm

Here wasm is my branch name. Sometimes the new changes are not reflected while making a get. This was a major roadblock while doing continuous iterations and bug fixes. I needed the changes quickly available.

After basic googling, I found workspaces in Go. In a workspace, you can have multiple GoLang projects. In my case, I had lama2 and httpie-go. If we make any changes in the httpie-go code, it will get instantly reflected in the lama2 where we are using it as a package. This will save a lot of time for debugging. Also, the build tags get propagated into the library while you build.

Here are the steps for setting up a workspace in the GoLang project. The first step is to create an empty repo name workspace and then add the projects you need to work on, into that directory.

This is the structure.

workspace -|
           |- Lama2
           |- httpie-go

Create a go.work file inside the workspace.

go 1.21.1

use (
	./Lama2
	./httpie-go
)

This is all you need to do to get the workspace working. Changes in httpie-go will be reflected back in Lama2 where we are using it as a library.

Handling Size Issues While Deploying

While trying to use the WASM binary from a website, the WASM binary file size became an issue. My initial file size was 25.3 MB. This is not practical to use such a huge file. Here are some tricks to reduce the size of WASM. If you are working on a WASM project you are also likely to face this problem.

1. Remove useless Code and Dependencies: Remove unused code, try to optimize the code, and check out individual package sizes. You can check the size of the individual packages using goweight tool. In our code, there was goja dependency which was taking a lot of space. So we removed that used simpler functions.

2. Use build flags that significantly reduce the size while building. This reduced the WASM file size to 11.4MB.

GOOS=js GOARCH=wasm go build -a -gcflags=all="-l -B -wb=false" -ldflags="-w -s" -o main.wasm

-gcflags allows you to pass arguments to the Go compiler for all packages (all).
-l disables inlining.
-B disables bounds checking.
-wb=false disables write barrier checks (garbage collector related).
-ldflags allows you to pass arguments to the Go linker.
-w omits the DWARF symbol table, which is used for debugging.
-s omits the symbol table and debug information, making the binary smaller.

3. Using Nginx gzip to compress the file: In your nginx config you can use the following rules to use gzip compression. gzip_comp_level=9 is the max compression.

 location /wasm {
        root /var/www;
        index index.html;
        try_files $uri $uri/ =404;
        add_header Access-Control-Allow-Origin *;
      
        gzip on;
        gzip_disable "msie6";
        gzip_min_length 256;

        gzip_vary on;
        gzip_proxied any;
        gzip_comp_level 9;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;
        gzip_types text/plain application/octet-stream  text/css application/json application/javascript text/xml application/xml application/xml+rss;

   }

Using these tricks I got the size reduced to 2.5MB. Checkout my exploration on reducing the size here: docs

Conclusion

Let's conclude this article with a small overview. We discussed the issues faced while converting a Go project to WASM. We also discussed the solutions to tackle them using workspaces and build tags. We also discussed the size issues that we faced while deploying the WASM binary. If you find this article helpful, please do check out Lama2 and give it a star. Also if you can contribute to the projects in any way, please do let us know. I hope this article was useful for you. Thanks for the read and subscribe to this journal for weekly blogs.

FeedZap: Read 2X Books This Year

FeedZap helps you consume your books through a healthy, snackable feed, so that you can read more with less time, effort and energy.

BUY