Not long ago I forked a single-file CLI tool with the intention of submitting a couple of PRs but I ran into two issues:

• The project itself used black and isort although they weren't listed as dependencies in the pyproject.toml, (a nix flake was used instead).

• The .gitignore file was lacking an Environments section and normally I use direnv to load environment variables from a .envrc.

Problem One: Formatting

I didn't have nix installed at the time and my editor is configured for Ruff, so I settled on using pre-commit with the following config:

repos:
  - repo: https://github.com/psf/black-pre-commit-mirror
    rev: 25.1.0
    hooks:
      - id: black
        language_version: python3.12

  - repo: https://github.com/pycqa/isort
    rev: 6.0.1
    hooks:
      - id: isort
        name: isort (python)

That's great, however, I had no intention to submit this file in a PR but globally ignoring it would be inappropriate because I almost always do want to commit pre-commit configs.

Problem Two: Loading the Environment

The PR I submitted involved adding environment variable configuration for the full surface of the CLI, along with it, I added an Environments section to the .gitignore file. However, this second addition was quickly rejected for being unrelated to the PR 😕. Ultimately, this encouraged me to implement a local solution.

Solution

It turns out git supports an exclude file located at .git/info/exclude. In it you can add files/patterns that you wish to ignore for that specific repository.

So I wrote a CLI tool exclude that makes it effortless to add, del, list and reset this file. While I won’t delve into the internals of the CLI, I want to highlight one important aspect related to testing: the use of interfaces.

Take the runAddCommand function, it accepts both an io.Writer and an io.ReadWriter:

func runAddCommand(out io.Writer, f io.ReadWriter, args []string) error 

This means I can pass it an os.Stdout (printing to console) and an os.File (the actual exclude file) for production use, meanwhile, for testing I can pass it two bytes.Buffer.

for _, tt := range tests {
	t.Run(tt.name, func(t *testing.T) {
		var out bytes.Buffer
		f := bytes.NewBufferString(tt.existing)

		err := runAddCommand(&out, f, tt.args)
		if err != nil {
			t.Fatalf("runAddCommand returned an error: %v", err)
		}

Go's powerful interfaces are just one of the reasons I love the language. By using them to enforce contracts we can keep our functions flexible and testable while ensuring type safety.

Conclusion

The exclude CLI may be a niche tool but it works and solved my problem.

There are some other articles that touch on the issue of file exclusions from tracking, I won't link to them but they're readily findable with a quick Google search.

Subscribe to this blog's RSS feed

I’ll preface this by saying that while Kong is now my favourite CLI framework, it wasn’t always my first choice. Early on, I built many of my CLI tools using Cobra, inspired by Ricardo Gerardi’s excellent book, Powerful Command-Line Applications in Go, which I still highly recommend. For smaller-scale CLIs, I usually turn to urfave/cli or Peter Bourgon’s Flags First package, both of which are excellent libraries as well.

Why Kong is King

Declarative approach

Kong’s approach, leveraging Go structs and tags, eases the transition from CLI concept to code. For the purposes of these examples I use the xair-cli package. The CLI struct defines the root of the CLI and embeds a Config struct:

type CLI struct {
	Config `embed:"" prefix:"" help:"The configuration for the CLI."`
}

That Config struct then uses tags to define:

• default values
• environment variables
• long and short versions of the flag
• help output

This is clean, declarative and easy to read.

Subcommands as structs

Many CLIs require complex subcommand structures and how a library allows you to layout your structure can make a big difference to readability.

By defining subcommands as struct fields tagged with cmd defining complex structures becomes possible:

type CLI struct {
	Main     MainCmdGroup      `cmd:""`
	Strip    StripCmdGroup     `cmd:""`
	Bus      BusCmdGroup       `cmd:""`
	Headamp  HeadampCmdGroup   `cmd:""`
	Snapshot SnapshotCmdGroup  `cmd:""`
	Dca      DCACmdGroup       `cmd:""`
}

Where each one of these can be it's own command or subcommand group:

type MainCmdGroup struct {
	Mute    MainMuteCmd       `cmd:""`

	Fader   MainFaderCmd      `cmd:""`
	Fadein  MainFadeinCmd     `cmd:""`
	Fadeout MainFadeoutCmd    `cmd:""`

	Eq      MainEqCmdGroup    `cmd:""`
	Comp    MainCompCmdGroup  `cmd:""`
}

This approach makes it straightforward to design CLI structures of arbitrary depth.

Supports all the goodies

Kong offers many features allowing developers to create powerful and intuitive interfaces to fit a variety of domains. These include:

• Custom validators
• Branching off positional arguments
• Argument definitions with slices, maps and pointers
• Help customisation at a granular level

For example, the xair-cli package models an 18-channel rack mixer consisting of 18 input strips, 6 auxiliary buses, a Main L/R bus and a wide variety of control types such as EQ, effects, and gain sliders. When we look at the OSC spec we see addresses such as /ch/01/mix/01/level and /bus/1/eq/1/f.

The pattern here is:

• select the channel type and index into it
• select the parameter type and index into it
• select the item you wish to control.

So ideally we want to represent these commands like so:

xair-cli strip <index> send <send-index> [<level>]

xair-cli bus <index> eq <band> freq [<freq>]

Since Kong allows us to branch off positional arguments we can achieve this by embedding structs tagged with arg into command structs:

type BusCmdGroup struct {
	Index struct {
		Index   int           `arg:""`
	} `arg:"" help:"Control a specific bus by index."`
}

If we repeat this pattern:

type BusEqCmdGroup struct {
	Band struct {
		Band *int             `arg:""`
	} `arg:"" help:"Control a specific EQ band of the bus."`
}

The final result is an interface that closely matches the format typically expected in audio CLIs.

Highly extensible

Being a well designed library it supports all kinds of plugins, here are two of my favourites:

• mango-kong for generating man pages.
• kong-completion for generating shell completion scripts.

Both integrate seamlessly with the library allowing a developer to extend their CLI by defining fields on the CLI struct:

type CLI struct {
	Man     mangokong.ManFlag `help:"Print man page."`

	Completion kongcompletion.Completion `help:"Generate completions."`
}

Quick, simple and effective!

Conclusion

Kong is a beautiful, powerful and highly flexible library perfect for writing all kinds of command-line interfaces. It's my personal favourite but I encourage you to explore the different libraries available, they all offer their own unique take.

Further notes:

• Historically, Kong has lacked robust shell completion, but the recently released kong-completion plugin is compatible with the latest version. While this plugin works for bash, zsh, and fish, it does not currently extend to PowerShell.

Subscribe to this blog's RSS feed

In order to fully utilise remote controlling over VBAN you need two way communication which requires implementing both TEXT (outgoing) and SERVICE (incoming) subprotocols.

Text Implementation

Text is fairly straightforward in that you are required to build a packet comprised of a header matching the specification along with a payload and the VBAN server should process it.

A barebones example:

import socket
import struct

# fmt: off
BPS_OPTS: list[int] = [
    0, 110, 150, 300, 600, 1200, 2400, 4800, 9600, 14400, 19200, 31250,
    38400, 57600, 115200, 128000, 230400, 250000, 256000, 460800, 921600,
    1000000, 1500000, 2000000, 3000000
]
# fmt: on
SUBPROTOCOL_TXT = 0x40
CHANNEL = 0
STREAMTYPE_UTF8 = 0x10


def main(
    command: str,
    host: str = "localhost",
    port: int = 6980,
    streamname: str = "Command1",
) -> None:
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
        header = struct.pack(
            "<4s4B16sI",
            b"VBAN",
            BPS_OPTS.index(256000) | SUBPROTOCOL_TXT,
            0,
            CHANNEL,
            STREAMTYPE_UTF8,
            streamname.encode("utf-8").ljust(16, b"\0"),
            0,
        )

        sock.sendto(header + command.encode("utf-8"), (host, port))

Service Implementation

Service is more involved in that you are required to:

• Subscribe to the service to receive the data
• Parse the incoming data packets.

For the first step we can fire a subscription packet matching the protocol specification but we must do this repeatedly on an interval less than the time we subscribe for.

import socket
import struct
import threading
import time

SUBPROTOCOL_SERVICE = 0x60
RTPACKETREGISTER = 32
RTPACKET = 33
SUBSCRIPTION_TIMEOUT = 5
PACKET_IDENT = 0


def subscribe_to_service(
    sock: socket.socket, host: str, port: int, stop_event: threading.Event
):
    framecounter = 0
    while not stop_event.is_set():
        header = struct.pack(
            "<4s4B16sI",
            b"VBAN",
            SUBPROTOCOL_SERVICE,
            PACKET_IDENT & 0xFF,
            RTPACKETREGISTER,
            SUBSCRIPTION_TIMEOUT & 0xFF,
            b"Register-RTP".ljust(16, b"\0"),
            framecounter,
        )
        framecounter += 1

        sock.sendto(header, (host, port))

        time.sleep(SUBSCRIPTION_TIMEOUT - 1)


def main(
    host: str = "localhost",
    port: int = 6980,
):
    stop_event = threading.Event()
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
        t = threading.Thread(
            target=subscribe_to_service, args=(sock, host, port, stop_event)
        )
        t.start()

        while not stop_event.is_set():
            try:
                data, addr = sock.recvfrom(2048)
                print(f"Received data from {addr}: {data}")
            except socket.timeout:
                continue
            except KeyboardInterrupt:
                stop_event.set()

        t.join()

What we'll receive in the output is a large dump of data:

Received data from ('localhost', 6980): b'VBAN`\x00!\x00Voicemeeter-RTP\x00\x1f\x9e\t\x00\x03\x00\x00\x04\x02\x02\x01\x03\x00\x00\x00\x00\x80\xbb\x00\x00\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\x83\xfa\x83\xfa\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\x83\xfa\x83\xfa\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\x83\xfa\x83\xfa\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1\xe0\xb1...

This isn't useful to us unless we parse and convert it to python types. Here is a simple class that parses bytes 28-43 of an incoming RT Packet:

@dataclass
class VbanRTPacket:
    """Represents bytes 28-43 of an incoming RTPacket"""

    HEADER_SIZE = 4 + 1 + 1 + 1 + 1 + 16

    _voicemeeterType: bytes
    _reserved: bytes
    _buffersize: bytes
    _voicemeeterVersion: bytes
    _optionBits: bytes
    _samplerate: bytes

    def __str__(self) -> str:
        return ", ".join(
            [
                f"{self.voicemeetertype=}",
                f"{self.voicemeeterversion=}",
                f"{self.samplerate=}",
            ]
        )

    @property
    def voicemeetertype(self) -> str:
        """returns voicemeeter type as a string"""
        return ["", "basic", "banana", "potato"][
            int.from_bytes(self._voicemeeterType, "little")
        ]

    @property
    def voicemeeterversion(self) -> tuple:
        """returns voicemeeter version as a tuple"""
        return tuple(self._voicemeeterVersion[i] for i in range(3, -1, -1))

    @property
    def samplerate(self) -> int:
        """returns samplerate as an int"""
        return int.from_bytes(self._samplerate, "little")

    @classmethod
    def from_bytes(cls, data: bytes) -> "VbanRTPacket":
        """Returns a dataclass representing the RTPacket data 
        from bytes 28-43 of the incoming packet
        """
        return cls(
            _voicemeeterType=data[28:29],
            _reserved=data[29:30],
            _buffersize=data[30:32],
            _voicemeeterVersion=data[32:36],
            _optionBits=data[36:40],
            _samplerate=data[40:44],
        )

However, a VBAN server can throw a lot of different kinds of data to a listening socket so it's important to filter out the data you need. This can be done by adding in some guard clauses:

def main(
    host: str = "localhost",
    port: int = 6980,
):
    stop_event = threading.Event()
    with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
        t = threading.Thread(
            target=subscribe_to_service, args=(sock, host, port, stop_event)
        )
        t.start()

        while not stop_event.is_set():
            try:
                data, addr = sock.recvfrom(2048)
                if len(data) < VbanRTPacket.HEADER_SIZE:
                    continue

                if data[0:4] != b"VBAN":
                    continue

                protocol = data[4] & 0xE0
                if protocol != SUBPROTOCOL_SERVICE:
                    continue

                if data[6] != RTPACKET:
                    continue

                packet = VbanRTPacket.from_bytes(data)
                print(packet)

            except socket.timeout:
                continue
            except KeyboardInterrupt:
                stop_event.set()

        t.join()

The final output of the script is now:

self.voicemeetertype='potato', self.voicemeeterversion=(3, 1, 2, 2), self.samplerate=48000

Demonstrating how we can subscribe for real time data from the RTPacket service and convert the returned data into usable python types.

Conclusion

There's a lot more to the specification than that which has been demonstrated in this blog post. You can find a more complete implementation of the TEXT/SERVICE subprotocols in the vban-cmd python package along with a python interface offering an abstraction layer over the dataclasses making scripts like the following possible:

class ManyThings:
    def __init__(self, vban):
        self.vban = vban

    def things(self):
        self.vban.strip[0].label = 'podmic'
        self.vban.strip[0].mute = True

    def other_things(self):
        self.vban.bus[3].gain = -6.3
        self.vban.bus[4].eq = True
        info = (
            f'bus 3 gain has been set to {self.vban.bus[3].gain}',
            f'bus 4 eq has been set to {self.vban.bus[4].eq}',
        )
        print('\n'.join(info))


def main():
    conn = {'host': 'localhost', 'port': 6980, 'streamname': 'Command1'}
    with vban_cmd.api('banana', **conn) as vban:
        do = ManyThings(vban)
        do.things()
        do.other_things()

        # set many parameters at once
        vban.apply(
            {
                'strip-2': {'A1': True, 'B1': True, 'gain': -6.0},
                'bus-2': {'mute': True},
                'vban-in-0': {'on': True},
            }
        )

Or perhaps even include it in another package altogether:

• vban-cli: A command-line interface.
• vban-tui: A Textual based TUI.

The possibilities are endless.

Other fantastic projects implementing various VBAN subprotocols:

• vban A pure C implementation of AUDIO/TEXT.
• pyVBAN A python implementation of AUDIO/SERIAL and TEXT.
• obs-vban An OBS plugin implementing AUDIO.
• vbantxt A Go implementation of TEXT offering a single binary

Even more with a quick search.

Subscribe to this blog's RSS feed

Background

It turns out the AmpliGame D6 does ship with a Streamlabs plugin:

But you have to jump through hoops to get it to connect and even if you get it working (I had problems authorising), it's not obvious how to switch scenes with it, if it's even possible.

Solution

So how else can we control Streamlabs from the AmpliGame D6? One option is via a CLI leveraging websockets but we need a way to run it. The D6 does include a feature Toolbox > Open which allows a user to load an application with arguments. This sounds great but in my testing, it opens a CMD window which threw me out of game, so no good.

So how else can we run a CLI?

ScriptDeck

Enter ScriptDeck, a wonderful plugin by the StartAutomating guys. It was created for the Elgato StreamDeck but can be installed on the AmpliGame D6 by following these instructions by TheBeardOfKnowledge.

To summarise:

• Install Elgato StreamDeck software, register for a free account on their site
• Download any plugins/icon packs you want
• Copy and paste them into the D6 plugins directory.
• Restart the D6 client.

Note, at the time of writing this still works for the ScriptDeck plugin although it no longer works for many other plugins.

When you visit the Elgato store you'll notice there are two version of the ScriptDeck plugin:

• WindowsScriptDeck: Compatible with Powershell 5
• ScriptDeck: Compatible with Powershell 7 (Core)

Now that we have a way to run our CLI, we need a way to install the CLI and make it discoverable.

SLOBS-CLI

slobs-cli is a python CLI, so you'll need python 3.11 or greater at the very least. Thankfully the guys at Astral have created uv a python version/package manager which makes installing python CLIs a cinch. There are several ways to install uv on Windows so select any one of them from their installation instructions.

Once that's done, installing the slobs-cli is a one-liner:

uv tool install slobs-cli

In order to communicate with Streamlabs the slobs-cli expects to know the websocket connection information. In recent versions of Streamlabs you'll find the information in Settings > Mobile > Third Party Connections:

Once you have the ip, port and websocket token it's time to test. First open a Powershell session in Windows and run:

slobs-cli --domain localhost --port 59650 --token <API token> scene list

You should be met with a response like:

This is great, it's working! However, passing the connection info on every invocation is cumbersome. A better way is to use environment variables. The way to manage this in Powershell is to store them in a Powershell profile, so in your Powershell session enter $profile and open the file (or create if it doesn't exist). Then in your Microsoft.PowerShell_profile.ps1 file enter the following:

# slobs-cli
$Env:SLOBS_DOMAIN = "localhost"
$Env:SLOBS_PORT = "59650"
$Env:SLOBS_TOKEN = <API Token>

To be sure this works, restart your Powershell session then retry the command above but without passing the connection flags:

slobs-cli scene list

Running slobs-cli on the D6

Now we have a way to run our CLI and we have the CLI installed, we just need to run it from the D6 controller!

Expand the ScriptDeck plugin in the Fifine Control Deck software and drag-n-drop Powershell Script onto a button. Then in When Presssed just enter your slobs-cli command like so:

Fingers crossed, press the button on your D6 and watch the scene switch. Voilà!

You can do a lot more than just scene switching with slobs-cli so check the README for a full list of available commands.

Conclusion

If you're unfamiliar with python or CLIs in general then this process might seem daunting but if you follow the steps carefully it shouldn't take long to set up.

Furthermore, you can now control other software using CLIs, for example, OBS, Voicemeeter, Meld Studio etc.

Further Notes:

• The same process can be carried out for Meld Studio swapping out the slobs-cli part for meld-cli. However, it does require node instead of python and at the time of writing its still possible to port the Meld Studio plugin as detailed in the above section ScriptDeck.

Subscribe to this blog's RSS feed

In November of last year I decided to take part in Manning's manuscript review program as I was already reading one of their MEAP books, Learn Go with Pocket-Sized Projects. Here are my thoughts after reading the entire manuscript.

Target Audience and Goals of the Book

As outlined in the first chapter the book's target audience includes:

• Those with previous experience in other languages who would like to extend their professional skills.
• Teams who are considering Go for their next project by providing a broad and thorough insight of the language.
• Busy people who are looking to complete projects that are both rewarding but doable in a reasonable amount of time.

The author's aim to achieve the following goals:

• Teach using an iterative process with each chapter split into sections that guide you through project implementations on a commit-by-commit basis.
• Provide exemplary and clear examples of industry-level Go code.
• Leave the reader inspired and with enough knowledge to go on to write excellent Go code themselves.

Structure

The book is comprised of 12 chapters, each one walking the reader through a single project although the final chapter technically covers two subjects and is designed to round off the book.

The kinds of projects you will develop include:

• A 3-level logging library with a stable, exported API.
• A CLI money converter app
• A concurrent maze solver
• A gRCP web service and client.

Each chapter follows a similar structure:

• Introductory text written in a narrative style often providing historical or cultural context to the problem at hand.
• An outline of the projects Requirements and Limitations.
• A step by step walkthrough of the project's implementation
• A summary section detailing the salient points.

I don't have enough space in this article to talk about every project in the book but I'll briefly cover two of them.

Bookworm's digest (chapter 3)

If you're like me and you love reading books, then anything bookworm related is probably interesting to you! In this chapter you'll load virtual bookshelves stored in JSON files and use Go to read, sort and analyse the data.

This chapter covers the following:

• Use of maps as an in-memory storage
• Use of deferred functions and their stacked execution
• Decoding JSON files into Go structs
• Sorting slices with custom comparators as well as implementing sort.Interface{}

Concurrent maze solver (chapter 9)

This chapter introduces you to the world of maze solving.

This chapter covers the following:

• Use of the image/png library to load and write mazes to disk.
• How to spin up many goroutines to search paths simultaneously.
• How to record the paths searched and colour them on the PNG.
• How to synchronise goroutines using channels and waitgroups.

Do the authors achieve their goals?

In my opinion the answer to this is a clear yes.

Goal one: Iterative process

Each chapter is split into sections with each section building on the previous. The chapters themselves are also sufficiently organised by difficulty.

The authors take a bottom up approach, for each project they start with the basics from initializing the module, defining the project specification, thinking through the design choices and finally onto implementation.

The reader is encouraged to commit their work to source control before any major additions.

The authors go to great lengths to explain their thought processes. This is very important from the readers perspective because it allows you to think about 'why' something should be done and not just 'how'.

Goal two: Production level code

Testing is a primary concern throughout the book and encouraged at every step.

The authors stress the importance of thinking through design choices carefully, not only their impact in the moment but also into the future.

Although anti-patterns are used to demonstrate less than ideal code, the authors do a good job of explaining the shortcomings and then go on to demonstrate a preferred implementation.

Goal three: Projects scaled for busy people

As is addressed in the book's appendix, several of the chapters have you write projects that rely on in-memory databases primarily to keep the projects sized reasonably (doable in a day or two). The authors stated clearly both in the chapters and in the appendix that this is normally an unacceptable solution and would not be suitable for production code.

Conclusion

I think that Learn Go with Pocket-Sized Projects is a beautifully written book that serves its stated goals.

The book does a good job of presenting the strength and flexibility of Go, offering projects that deliver a broad coverage of the language.

The writing style is a mixture of conversational, humorous and terse but for the most part explanations are straightforward and to the point.

I wouldn't recommend this book as a first read in Go, better a second or third book. Certainly I would recommend it to anyone looking for a book to further their knowledge in Go best practices and project design.

Further Notes:

• The downloadable code that accompanies the book was very well organised!

Subscribe to this blog's RSS feed

Background

Behringer offer an official app XAIR Edit that can be used to remote control a mixer from an android device or a Windows PC. It uses the OSC protocol over UDP to communicate with the mixer.

Patrick-Gilles Maillot has done a lot of fantastic work (mostly in C) with the XAIR/MAIR series. Of particular help to me was the documentation he'd drawn up along with an X32 Emulator which I found very useful in testing my own code.

Discovering the base class

It turns out that one thing I really enjoy doing is writing interfaces that represent families of products. I've spent a lot of time over the past few years programming with the Voicemeeter API, which itself is a family of products, so I thought why not give it a go here as well.

To start with I went searching to see if anyone had already done this and I stumbled across the Xair-Remote python package by Peter Dikant. It's a useful package that allows a user to connect an X-TOUCH MINI MIDI Controller to an XAIR mixer. With it you can control parameter states including volumes, mute states and bus sends. Digging into the code a little I noticed he'd written a base XAirClient class and used it's send() method to communicate directly with the mixer. So it occurred to me, perhaps we can decouple this base class from its implementation, write an abstraction layer over it, scale it according to each kind of mixer and present a pythonic interface that represents the XAIR/MAIR family of mixers.

Developing the Interface

Step one, extract the base class

This was mostly a copy-paste, I'm very grateful to the original developer of the Xair-Remote package, it sped up the process of writing this interface.

Now that we're dealing with an interface that represents a family of products it makes perfect sense to define the base class as an ABC, as it will serve as the launching point for the various mixer classes.

Step two, lay out the kind maps

Since we want our abstraction layer to scale correctly, it helps us to create dataclasses that map precisely the structure of each kind of mixer.

For example, this kind map would represent the XR18 mixer:

@dataclass(frozen=True)
class XR18KindMap(KindMap):
    # note ch 17-18 defined as aux return
    num_dca: int = 4
    num_strip: int = 16
    num_bus: int = 6
    num_fx: int = 4

Note, we expect the kind maps to remain frozen for the lifetime of the program, this way they behave more like named tuples.

Step three, write the abstraction layer

In writing the abstraction layer I relied heavily on documentation written up by others. I also had to rely somewhat on intuition and a lot of testing. Since I'm not an audio engineer and I only have access to a single product in the family of products at points I just did my best.

I'll take the Strip class as a single example. First, an abstract base class that defines some default implementation:

class IStrip(abc.ABC):
    def __init__(self, remote, index: int):
        self._remote = remote
        self.index = index + 1

    def getter(self, param: str) -> tuple:
        return self._remote.query(f"{self.address}/{param}")

    def setter(self, param: str, val: int):
        self._remote.send(f"{self.address}/{param}", val)

    @abc.abstractmethod
    def address(self):
        pass

Then, a concrete class that mixes in a whole bunch of other classes that precisely define the layout for a single strip:

class Strip(IStrip):
    @classmethod
    def make(cls, remote, index):
        STRIP_cls = type(
            f"Strip{remote.kind}",
            (cls,),
            {
                **{
                    _cls.__name__.lower(): type(
                        f"{_cls.__name__}{remote.kind}", (_cls, cls), {}
                    )(remote, index)
                    for _cls in (
                        Config,
                        Preamp,
                        Gate,
                        ...
                    )
                },
                ...
            },
        )
        return STRIP_cls(remote, index)

    @property
    def address(self) -> str:
        return f"/ch/{str(self.index).zfill(2)}"

Finally, a factory function for composing each XAirRemote{kind} object:

def init_xair(self, *args, **kwargs):
    XAirRemote.__init__(self, *args, **kwargs)
    self.kind = kind
    self.strip = tuple(Strip.make(self, i) for i in range(kind.num_strip))

All of the classes are built and loaded into memory at import time ready to be requested by the package entry point.

Extending the interface to support the X32

When I first wrote the XAIR-API package I'd originally intended to support the XAIR/MAIR series only. Some of the OSC addresses differ slightly for the X32 because it is (physically) a substantially different mixer. Whereas the XAIR/MAIR are digital rack mixers, the X32 is a full blown desk mixer with many more channels and physical controls. However, due to a particular request from a particular user of the interface I decided to investigate support for the X32.

To that end I wrote some adapter classes, for example:

class Bus(IBus):
    @property
    def address(self):
        return f"/bus/{str(self.index).zfill(2)}"

They override the addresses for the XAIR series modifying them according to the X32 specification. In the case of Bus addresses, the XAIR series use /bus/1/ whereas the X32 uses /bus/01, as you can see numbers are left padded with zeros.

Then I wrote a separate factory function for the x32, using the adapter classes to build the layout for the interface:

def init_x32(self, *args, **kwargs):
    XAirRemote.__init__(self, *args, **kwargs)
    self.kind = kind
    self.bus = tuple(adapter.Bus.make(self, i) for i in range(kind.num_bus))

Conclusion

All in all I found the exercise of decoupling a base class written by another developer and writing it to an interface an eye-opening experience. It forced me to really think about the following:

• The best way to implement the interface internally.
• What it would be like to use from the consumer's perspective.
• Which parts to expose.
• How to present a pythonic interface that abstracts away from the details of OSC.

I have made public the full source code.

Subscribe to this blog's RSS feed

Earlier this month I was lucky enough to get my hands on a complimentary copy of Head First C# Fifth Edition. There were things I expected and things that surprised me, here are my thoughts.

About the Review

This review will be an opinion piece from the perspective of a learner. I'm not a beginner in programming but I am new to C#. That puts me in the target audience for this book. I went into this primarily interested in GUI development. The scope of this review will be limited to the learning experience, for a more technical review you will need to look elsewhere.

Style

It's impossible to write about my experience with a Head First book without discussing the general Head First style. Books in this series tend to employ a lot of visual aids, a conversational tone, a wide variety of exercises and deliberate repetition. For a more detailed explanation you may check this page on the O'Reilly site.

Since this review reflects my personal perspective, I will share my own opinion. There are certain aspects of this teaching style that I prefer over others. For instance, I find the conversational tone quite appealing. However, do I feel that it is necessary to keep me engaged? Not really—I am equally comfortable reading books written in a more formal style.

The redundancy is a factor I consider to be a major strength, sure it's repetition but it's explanation of the same idea in different ways. Having said that, I didn't find it helpful every time and in fact on one or two occasions I found myself skipping an (already explained) point in order not to break my train of thought.

I like the use of images to break up text. As you can see from the example page linked there is a clear structure to the page working its way from top to bottom, at the same time the reader's eye is drawn between locations. This page is a clear demonstration of the cognitive friendly approach this book aims for.

Regarding the presentation, the first thing that caught my attention upon opening the book was the use of hand-drawn characters. Traditionally, the Head First series has relied on photographic images of people—for example, see the image below:

In my opinion, this is a significant improvement. The hand-drawn illustrations make the pages appear cleaner, less distracting, and more relatable. It’s also worth noting that this update seems to be applied across the entire series, as I’ve observed the same style in other recent Head First editions.

Topics Covered

The book has 12 main chapters and then intertwined with them six Unity Labs. Topics covered in this book, in no particular order, include but are not limited to:

• Good code style, ie sensible variable, method naming.
• Refactoring.
• Basic types and flow control constructs.
• General OOP concepts: inheritance, composition, polymorphism, encapsulation, cohesion, separation of concerns, DRY etc.
• Abstract and Concrete classes.
• Using the Visual Studio debugger.
• The encouragement and use of paper prototypes.
• XAML and C# code behind in .NET MAUI apps.
• Data binding.
• Automatic properties with backing fields.
• Interfaces (with and without default implementations), how to use them and why they are important.
• Upcasting and downcasting.
• Sorting techniques with IComparable and IComparer interfaces.
• Collections, specifically covered: List, Dictionary, Queues, Stacks.
• LINQ queries, LINQ methods and deferred evaluation.
• File and memory streams. Network and Gzip streams are briefly mentioned but not explored.
• IDisposable interface for handling the cleanup of unmanaged resources.
• Object serialization.
• The garbage collector.
• Exception handling.
• Nullable value types and the null-coalescing operator.
• Extending sealed classes.
• Unit testing with MSTest, writing for edge cases and unpredictable input.
• Logging with Serilog
• The encouragement and use of AI assisted learning and an introduction to prompt engineering.
• Feedback loops, emergence and how they affect dynamics both in games and other areas of programming.

As a learner, the parts I particularly liked

The book opens with a clear and detailed walkthrough with screenshots guiding the reader on setting up their development environment. This is important because perhaps the reader has never used Visual Studio. I've read other books before that lacked detailed setup instructions, this can lead to great confusion and leave a bad initial impression.

The first project you're tasked with is an Animal Matching game.

This is a great introduction to the book because it gives a fast demonstration of the power and flexibility of C# and XAML with just a few lines of code.

The .NET MAUI project Random Cards walks you through a structured process. The discussion around Ana's game also covers point one.

1. Plan in advance how you intend to model your ideas into classes, possibly with the aid of a paper prototype.
2. Write a working command line application.
3. Complete the project by using those same classes in a GUI application. All the while reinforcing the importance of accessibility.

I very much like this methodology, first of all because it encourages the learner to really think about what they are doing. As a beginner, it's way too easy to want to rush in and start writing code, then later regret some of the choices that a better design would have avoided. By putting careful thought and planning into how you organise your classes and which methods you will make public you can separate the internal design of a class from its consuming code. Second, the process is incremental. It gives the learner the opportunity to start with the basics and build up as they go along. Last of all, having personally spent time working with users with accessibility needs, I am intimately familiar with the frustrations that can arise from inaccessible GUI design. Computers are used by all kinds of users around the world and it's important that beginners are encouraged to follow best practices concerning accessibility.

Each chapter ends with a Q&A style section. As the authors state, some of these questions are actual questions they've been asked by readers of past editions of the book. This is a great format for a few reasons:

• It gives the opportunity for readers to process topics covered in a back and forth manner
• It encourages the reader to ask questions themselves
• It may well answer a specific question they already had. The example that comes to mind personally, the IDisposable interface had been covered in chapter 10 but only in the context of files and streams. I found myself asking, is this an interface appropriately used with other types of classes? The Q&A in chapter 12 asks and answers this question almost verbatim.

Criticisms

As I mentioned in the introduction of this review, my primary interest is in GUI development and there was plenty of that present. I am less interested in game development and there was a lot of that too. Now I have to be fair in my assessment, the authors go to a lot of effort to explain their reasoning behind the game-centred focus. The lessons taught throughout the book while building the game projects are broadly applicable across programming. Nevertheless, my question is simple, would it have been possible to teach some of those same programming concepts while focusing on a wider variety of non game-centred projects?

Possible improvements

The lumberjack exercise in chapter eight has you build a console app to demonstrate the Stack and Queue ADTs. It does a sufficient job but I have to wonder whether this was a great opportunity to demonstrate the nature of the stack and queue in a more visual, .NET MAUI GUI app.

Overall thoughts

I believe Head First C# Fifth Edition is a very strong effort. It's clear to me that a lot of care and attention went into producing a complete and thorough learning experience for the reader. The emphasis on good programming practices, GUI accessibility and self learning techniques especially impressed me.

The first several chapters are gently paced and introduce fundamental concepts in C# but also programming in general. If you prefer a fast paced introduction to a language you should bear this in mind.

I've talked about the general style of Head First books and given my personal opinion but each person's experience will vary. I will say this much, if you've never tried reading a Head First book and C# piques your interest then I encourage you to give this book a go.

Further Notes:

• The authors have made available on their Github repository Blazor versions of all the .NET MAUI apps.
• They have a YouTube channel where they post guides related to the book.

Subscribe to this blog's RSS feed

The plugin API offers a broad range of functions for managing cvars, interacting with registered commands, file operations and so on.

As well as that, it offers callback functions for game events:

PCL void OnPreFastRestart();
PCL void OnExitLevel();
PCL void OnPostFastRestart();
PCL void OnPreGameRestart(int savepersist);
PCL void OnPostGameRestart(int savepersist);
PCL void OnSpawnServer();

First things first, compile suitable regexes for later use:

qboolean create_regex()
{
    return regcomp(&regex.ip, REG_IP, REG_EXTENDED) == 0 &&
           regcomp(&regex.url, REG_URL, REG_EXTENDED) == 0;
}

We indicate to the server if they fail to compile in the OnInit() function:

PCL int OnInit()
{
    if (create_regex() == qfalse)
    {
        log_info("AdStop: Failed to compile RegEx, exiting...");
        return 1;
    }

    ...
}

Then we register our plugin cvars, this gives users of the plugin the ability to configure its behaviour from config files:

PCL int OnInit()
{
    ...

    cvars.sub = Plugin_Cvar_RegisterString(..., ..., ...);
    cvars.sub_ip = Plugin_Cvar_RegisterBool(..., ..., ...);
    cvars.sub_url = Plugin_Cvar_RegisterBool(..., ..., ...);

    return 0;
}

Before checking for regex matches we must first clean the incoming message of colour codes. If we don't do this coloured messages may fail a regex test.

The following snippet uses pointer arithmetic to step through the string until we meet ^[0-9], skips those characters if present, otherwise the char at p is copied into the position at q. The message string is effectively cleaned in situ.

ptrdiff_t remove_colours(char *message)
{
    char *p, *q;

    p = q = message;
    while (*p)
    {
        if (*p == '^' && *(p + 1) && isdigit(*(p + 1)))
        {
            p++;
        }
        else if (isprint(*p))
        {
            *q++ = *p;
        }
        p++;
    }
    *q = '\0';

    return q - message;
}

Finally, if a match occurs we overwrite the original message. All of this is handled in the APIs OnMessageSent callback function:

PCL void OnMessageSent(char *msg, int slot, qboolean *show, int mode)
{
    remove_colours(msg);

    enum match_type match = matches(msg);
    if (match == IP  || match == URL)
    {
        snprintf(message,
                 MAX_SAY_TEXT,
                 Plugin_Cvar_GetString(cvars.sub));
    }

    ...
}

Once the plugin is loaded abusers attempting to advertise on the server will have their messages replaced.

Note. if the replacement text is set to a blank string the users message will not show at all, not even a prompt indicating an attempted message.

Subscribe to this blog's RSS feed

My goal was to create a command-line interface (CLI) program that implements the following features:

• Direct mode.
• Interactive mode.
• Possibility to set, get and toggle parameters.
• Load commands from script files.
• Launch the GUI (and possibly other tools) using flags.
• Load configuration files (xml profiles).
• Provide logging

First step, have the CLI accept instructions as arguments and execute each in turn. Then define an -i flag to enable interactive mode. This allows a user to operate the program in two distinct modes.

while ((opt = getopt(argc, argv, OPTSTR)) != -1)
{
	switch (opt)
	{
	case 'i':
		iflag = true;
		break;
	case 'h':
		[[fallthrough]];
	default:
		usage();
	}
}

if (iflag)
{
	puts("Interactive mode enabled. Enter 'Q' to exit.");
	interactive(vmr);
}
else
{
	for (int i = optind; i < argc; i++)
	{
		parse_input(vmr, argv[i]);
	}
}

Interactive mode should read repeatedly from stdin:

void interactive(PT_VMR vmr)
{
    ...

    while (fgets(input, MAX_LINE, stdin) != NULL)
    {
        input[(len = strcspn(input, "\n"))] = 0;
        if (len == 1 && toupper(input[0]) == 'Q')
            break;

        parse_input(vmr, input);
        
        ...
    }
}

Allowing a user to enter commands until an exit instruction is given:

Since reading from script files may use either of the CLI's modes we must parse the input in both cases. Consider that a single line in a script may contain multiple instructions, for example:

strip[0].gain=5 strip[1].comp+=4.8 strip[2].label=podmic

So it's important that we split lines into separate instructions:

void parse_input(PT_VMR vmr, char *input)
{
    ...

    token = strtok_r(input, DELIMITERS, &p);
    while (token != NULL)
    {
        parse_command(vmr, token);
        token = strtok_r(NULL, DELIMITERS, &p);
    }
}

Here is an example run with verbose output enabled:

Examples of where this may become problematic:

• strip[0].label="my podmic"
• bus[2].device.wdm="Realtek Digital Output (Realtek(R) Audio)"

The solution was to modify the parser to track when we step in and out of quoted strings, skip the delimeters when in quote and write the resulting token to a buffer, passing that to parse_command() instead.

The CLI application should correctly handle get, set and toggle operations.

void parse_command(PT_VMR vmr, char *command)
{
    log_debug("Parsing %s", command);

    if (command[0] == '!') /* toggle */
    {
        ...

        return;
    }

    if (strchr(command, '=') != NULL) /* set */
    {
        ...
    }
    else /* get */
    {
        ...
    }
}

Set is trivial enough, we can simply use the VBVMR_SetParameters api call. This handles both float and string parameters.

Get is a little tricker because C is statically typed, meaning the compiler must be made aware of parameter and return types. To avoid having to explicitly track which commands are expected to return which type of response we can use the fact that a failed get_parameter_float will return an error code and then try get_parameter_string.

void get(PT_VMR vmr, char *command, struct result *res)
{
    clear_dirty(vmr);
    if (get_parameter_float(vmr, command, &res->val.f) != 0)
    {
        res->type = STRING_T;
        if (get_parameter_string(vmr, command, res->val.s) != 0)
        {
            res->val.s[0] = 0;
            log_error("Unknown parameter '%s'", command);
        }
    }
}

As well as that, C offers Unions for building mixed data structures (covered in Chapter 16 of C Programming A Modern Approach). By defining a Struct with a Union member I was able to store and track the result.

struct result
{
    enum restype type;
    union val
    {
        float f;
        wchar_t s[RES_SZ];
    } val;
};

Toggle is then simply an implementation of a get into a set. The only noteworth detail is that we should guard against unsafe gain changes. I handled this by first testing if the response was of type float, and then testing it against 1 and 0. Strictly speaking this doesn't guarantee a boolean parameter, but it does protect against dangerous operations such as Strip 0 Gain = (1 - (-18)) which could be hazardous to health or audio equipment.

if (res.type == FLOAT_T)
{
	if (res.val.f == 1 || res.val.f == 0)
	{
		set_parameter_float(vmr, command, 1 - res.val.f);
	}
	else
	{
		...
	}
}

I decided to use the log.c package by rxi to offer various levels of logging. Here is a demonstration of the CLI run in direct mode with TRACE logging enabled.

As you can see, it gives a low level perspective of the API calls.

This has been a very fun project to tackle, it's easy to see why people fall in love with programming in C.

I have made public the full source code for this package.

Further Notes:

• The binary in Releases was compiled with coloured logging enabled. Unfortunately it doesn't work properly on all terminals. So rebuilding the application with coloured logging disabled may be necessary.

Subscribe to this blog's RSS feed

Background

To address this, I opted for a centralized system, which offers several advantages. Primarily, it enables me to manage all webhooks from a single location. Additionally, it provides the flexibility to parse and, if needed, modify a webhook’s payload before forwarding it to Discord.

For instance, consider the excellent Linux backup tool GoBackup. It supports sending webhook notifications — on backup completion — to various platforms including Discord. However, the default webhook message looks like this:

systemd: [GoBackup] OK: Backup service has successfully

Backup of service completed successfully at 2023-10-23 09:17:47.443308376 +0100 BST

This format is adequate, offering plenty of detail, however, I prefer notifications that are structured in the following way:

<hostname> :: <Service>: <status message>

By including the webhook’s origin (the server hostname), followed by the Linux service name, and finally the status message, I get the essential information. Discord provides a timestamp and if I need more context, I simply refer to the logs.

Implementing the Webhook Proxy

Define our Endpoint

Only a single endpoint is needed. However, validating the incoming request is important. This is achieved by using FastAPI's dependency injection in the path operation decorator.

@app.post(
    "/github/{repo_name}",
    dependencies=[
        Depends(validate_source_ip), Depends(validate_webhook_secret)
    ],
)
async def github_webhook(
    repo_name: str,
    request: Request,
    client: niquests.AsyncSession = Depends(get_request_client),
    x_real_ip: str = Header(...),
    x_hub_signature: str = Header(...),
    x_github_event: str = Header(...),
):
    ...

Validate the Source IP

In my case, the server is placed behind a reverse proxy. To determine the source IP, it is read from the X-Real-IP header. This IP is then checked against a list of valid IP ranges, which can be retrieved via the GitHub API.

async def validate_source_ip(
    request_client: Annotated[
        niquests.AsyncSession, Depends(get_request_client)
    ],
    x_real_ip: str = Header(...),
) -> None:
    resp = await request_client.get("https://api.github.com/meta")
    allowed_ips = resp.json().get("hooks", [])

    if not any(
        ip_address(x_real_ip) in ip_network(valid_ip)
        for valid_ip in allowed_ips
    ):
        ERR_MSG = f"IP address {x_real_ip} is not allowed"
        raise HTTPException(status_code=403, detail=ERR_MSG)

Validate the Webhook Secret

Next, the webhook secret is tested. If either of these dependency functions fails, a HTTPException is raised to prevent the request from being processed.

async def validate_webhook_secret(
    request: Request, x_hub_signature: str = Header(...)
) -> None:
    if not x_hub_signature:
        raise HTTPException(
            status_code=400, detail="Missing X-Hub-Signature header"
        )

    mac = hmac.new(
        settings.GITHUB_WEBHOOK_SECRET.encode(),
        msg=await request.body(),
        digestmod=hashlib.sha1,
    )
    if not hmac.compare_digest("sha1=" + mac.hexdigest(), x_hub_signature):
        raise HTTPException(status_code=403, detail="Invalid signature")

Match the Event and Trigger the Handler

If all validation checks succeed, the event type is read from the X-Github-Event header, and the appropriate handler is triggered.

match x_github_event:
	case "ping":
		return {"message": "pong"}
	case "issues":
		await handlers.issues(payload, client)
	case _:
		logger.debug(f"unhandled event {x_github_event}")

The request payload can now be processed and converted into a Discord webhook with embedded content:

async def issues(payload, request_client: AsyncSession):
    if payload["action"] not in ["opened", "closed", "reopened", "deleted"]:
        return

    embed = {
        ...
    }
    data = {
        "username": "Github-OpenGist",
        "embeds": [embed],
    }
    await send_discord_webhook(request_client=request_client, data=data)

Finally the Discord webhook is sent:

async def send_discord_webhook(request_client: AsyncSession, data):
    resp = await request_client.post(settings.WEBHOOK_URL, json=data)
    if resp.status_code != 204:
        raise HTTPException(
            status_code=400, detail="Failed to send Discord webhook"
        )

The result is a nicely formatted Discord message:

Conclusion

With everything up and running I can relay and receive Github webhook notification through a proxy server. Other than being written with a different framework the codebase was also converted from sync to async which is much faster for these types of I/O operations.

I've posted a version of this proxy server that implements only Github Issues. It can easily be extended to support other Github events.

Gist notes:

• Sending the Discord webhook is handled by the excellent niquests library.
• Managing the request client resource is done via a FastAPI lifespan.
• Loading the proxy server configuration is achieved with Pydantic Settings.

Subscribe to this blog's RSS feed