Introduction

What this book is not

The primary purpose of this book is to explain the internal architecture of popcorn2. This is not a replacement for documentation for any public or private kernel APIs. If you are trying to write a kernel module, please refer to the documentation.

popcorn2

popcorn2 is a hobby microkernel written by @egkoppel, as way to experiment with OS design ideas (which he claims are "improvements") not present in the major three OSes.

Structure

The Core

The core popcorn2 kernel is split across three main crates: kernel, kernel_hal and kernel_api.

kernel_hal

kernel_hal contains all architecture specific code required for the core kernel to operate, and no more. This means it specifically does not include any peripheral ¹ drivers, with the exception of a UART controller, to aid in debugging the preboot environment. This includes paging code, platform initialisation, etc.

kernel_api

kernel

Kernel modules

1. The line is not clear cut. For example, on x86, the x87 was originally a coprocessor, however it would be very difficult to write userspace drivers for it due to how it integrates with context switching. Therefore this would be something that would be considered close enough to part of the architecture that it should be in the HAL. ↩

Popcorn2 Driver Model

Drivers are located in /System/drivers, [should there be a non system folder too?] and have a name of the format [TBD].exec.

Device manager initially starts the root driver for the system as a fixed pseudo-device, and then a recursive enumeration procedure is followed.

For each driver, [an IPC channel] is initialised between the device manager and the driver. The device manager starts by sending an INIT command to the driver. The driver responds with a driver descriptor object. The NEXT command is then sent repeatedly. If the driver is a bus-type driver, it responds to each request with a child descriptor object, returning DONE

Driver descriptor object

or should this be part of the binary or something instead?

Child descriptor object

Loosely based on UDI device enumeration.

IRQ VM idea????

Using event queues for userspace IRQ handling poses a latency problem. Due to scheduler design or something, the driver may not get CPU time quickly enough and for long enough to handle IRQs immediately. For something like a PS/2 keyboard or mouse controller, this poses a problem due to the ease with which data can be overwritten under high workload.

It is less immediately obvious why approaching this with POSIX style signals is not the most optimal idea. This could be designed to reduce the latency requirements, but it adds some overhead and complexity in the scheduler design. It also requires driver devlopers to ensure all interrupt code is signal safe, as well as ensuring proper syncronisation between the IRQ handlers and the main driver code.

A potentially silly idea to solve this is to define a basic virtual machine to execute IRQ handlers, designed to be versatile enough to read IRQ data and send EOIs, but not complex enough to pose a security risk (as it is run directly in the kernel) or lock up the kernel. Programs are limited to [TBD] bytes, and are compiled and statically checked by the kernel upon creation.

IRQ handlers are expected to read any required data from the device, and push it into the data queue, where the main driver process will be able to read and process it.

There is an implicit return at the end of all programs.

Handlers may be run with interrupts enabled. A CLI instruction is provided to be called before any end-of-interrupt is signalled, to prevent recursive calls that result in stack overflow. Stack overflows will result in the entire driver process being terminated.

The machine has X and Y registers, both 64 bits in size. The X register is populated with the logical interrupt number (specific to the driver) upon start. The Y register is zeroed.

Is 64 bits fine?

TODO: memory operations, improve branching, basic arithmetic(?)

IO port and memory access is only allowed to ports and memory the driver has prior permission to access. If access to required IO/memory is lost while an interrupt handler is in place, it will be removed and the driver notified [mechanism tbd].

Example routines

(; used to begin a line comment)

i8042 PS/2 controller handler

PUSH X      ; So the driver knows which device this interrupt is from
INB Y, 0x60 ; Read from the i8042 data port
PUSH Y      ; Pass the incoming data onto the driver
; The i8042 has no EOI routine so we just implicitly return here

i8259 PIC EOI handler

Assume here that logical interrupt numbers map to standard dual PIC lines. This example does not handle spurious interrupts.

CLI          ; Disable interrupts before sending the EOI to prevent any loops
IMM Y, 0x20  ; Load the Y register with the EOI command
BLE X, 7, 2  ; If interrupt lines 0-7, jump 2 instructions ahead
OUTB 0xA0, Y ; Otherwise send the EOI command to the second PIC
OUTB 0x20, Y ; In all cases, send the EOI command to the first PIC

Loosely modelled on RP2040 PIO blocks

IPC

IPC is a fundamental part of the design of Popcorn2, and underpins all interaction between processes and the outside world. To support this, a rich protocol called the Popcorn2 IPC Protocol (PIP) is used.

PIP at its core is an extension of Unix's "everything is a file" philosophy to an "everything is an object" philosophy. Instead of opening a file, a process creates an object, returning an object handle instead of a file descriptor. When created this way, a path to the object must be specified. Objects can also be returned as the result of a method call, in which case they may not have an underlying path.

Interfaces

Objects implement one or more interfaces. An interface is defined in a PIP Source file, and contains a number of methods, each associated with a name and a numeric identifier.

Interfaces are registered with the kernel by passing it a PIP Binary file. This is normally handled by the interfaced daemon, which searches for .pipb files in /System/proto (TBD: other locations) and registers interfaces it finds.

Core interfaces

Popcorn2 defines

Default objects

In the same way that UNIX systems provide a set of already open file descriptors on process start, Popcorn2 provides a set of already created objects.

Object handles 0, 1 and 2 correspond to stdin, stdout, and stderr, and implement interfaces accordingly. Additionally, a process handle is opened at 3.

KTable and TTable

popcorn2 uses two different types to implement page tables. TTable is used on a per-process basis, and contains the mappings for the lower half of the address space, which is specific for each process. In contrast, KTable stores the mappings for the kernel, which are shared across every single process. On an architecture like Arm, with two separate page table registers, these map directly to the global KTable and the TTable for the current process. On architectures like x86, with only one hardware page table, the implementation is more complex and requires cooperation between KTable and each TTable instance.

Implentation details

amd64

The bootloader allocates a PML4 for the initial process, as well as preallocating 256 contiguous PDPTs. The upper half of the PML4 is mapped to point to the allocated PDPTs.

The KTable type contains the 256 PDPTs:

struct KTable {
	tables: NonNull<[Table<PDPT>; 256]>
}

On creation of a new TTable, the 256 upper entries are all initialised to point to the KTable tables. In addition, TTable enforces that modifications can only be directly made to the lower 256 entries. Since the KTable tables are mapped into every TTable, any modification to KTable will update all TTables. The CR3 register is then pointed at the correct TTable instance.