Layerless

A content-addressable, layerless container runtime. No base images. No wasted bandwidth. No redundant downloads.

---

The Problem

Docker ships containers as layers — tar.gz archives stacked on top of each other. This design optimizes for storage deduplication on a single host, but it is terrible for distribution:

• Bloated base images: A 5MB Go binary ships with a 100MB Ubuntu base image because Docker has no way to share individual files across containers.
• Cache invalidation: Change one build flag and Docker invalidates the entire layer. You re-download 100MB just to get 2MB of new bytes.
• No cross-container deduplication: Two containers sharing the same libc.so.6 store it twice on disk and download it twice over the network.
• Requires Linux VMs: On macOS and Windows, Docker runs a full Linux VM, consuming gigabytes of RAM and disk.

For developers on limited bandwidth or constrained machines, this is unacceptable.

---

The Solution

Layerless borrows the content-addressable store concept from Nix, but strips away the package manager complexity. It combines it with the OCI runtime spec (the same standard Docker uses) to create a container system that is:

• File-level deduplicated: Every file is stored once by its SHA-256 hash. Shared across all applications.
• Layerless: No tar.gz layers. No base images. Just a manifest listing exactly the files your app needs.
• Minimal bandwidth: Updates only sync the chunks you don't already have. A 5MB binary update might only transfer 200KB of changed bytes.
• Native runtime: Uses Linux namespaces directly via runc. No Docker daemon. No VM on Linux.

---

Architecture

flowchart TB
    subgraph Pack["📦 Pack Phase"]
        A[Binary] --> B[Analyze Dependencies]
        B --> C[Store Files by Hash]
        C --> D[Generate Manifest]
        D --> E[Store Manifest by Hash]
    end

    subgraph Store["🗄️ Content-Addressable Store"]
        F[(Store)]
        F --> G["blob/23/23abc... (file A)"]
        F --> H["blob/95/95def... (file B)"]
        F --> I["manifest/95/95035... (manifest)"]
    end

    subgraph Run["▶️ Run Phase"]
        J[Manifest Hash] --> K[Read Manifest]
        K --> L[Assemble Rootfs from Store]
        L --> M[Write OCI config.json]
        M --> N[Execute via runc]
    end

    E --> F
    J --> F

Loading

Core Concepts

Concept	Description
Store	A local directory where every file is stored by its SHA-256 hash, sharded into subdirectories (ab/abcdef...).
Manifest	A JSON file describing a container rootfs as a list of {path, hash, size, mode} entries. The manifest itself is stored by its own hash.
Bundle	A temporary directory containing rootfs/ (assembled from the store) and config.json (OCI runtime spec), ready for runc.
Deduplication	If two apps need the same file, it exists once in the store. Updates only fetch missing hashes.

---

How It Works

1. Pack

Analyze a binary, store its files, and produce a content-addressed manifest.

ll pack -name=hello ./myapp

sequenceDiagram
    participant User
    participant CLI as ll pack
    participant Store as Content Store
    participant Disk as ~/.local/share/layerless/store

    User->>CLI: ll pack ./myapp
    CLI->>Store: Put(binary)
    Store->>Disk: Write to tmp file
    Store->>Store: Compute SHA-256
    Store->>Disk: Rename to ab/abc123... (atomic)
    Store-->>CLI: Return hash
    CLI->>CLI: Build manifest {name, files: [{path, hash, size, mode}]}
    CLI->>Store: Put(manifest JSON)
    Store->>Disk: Store manifest by its own hash
    Store-->>CLI: Return manifest hash
    CLI-->>User: Print manifest hash

Loading

What happens under the hood:

• The binary is read, hashed, and stored in ~/.local/share/layerless/store/
• If the same binary was packed before, the store detects the hash collision and skips writing (instant deduplication)
• A manifest JSON is generated and also stored by its own hash
• The manifest hash is the only reference you need to run the container later

2. Store

The store is a simple content-addressable filesystem.

~/.local/share/layerless/store/
├── 23/
│   └── 23a4f...c3d2     ← ca-certificates.crt (shared by 10 apps)
├── 95/
│   └── 95035...a999     ← manifest: hello v1.0
│   └── 95def...b123     ← manifest: hello v1.1
├── ab/
│   └── ab12c...d456     ← libc.so.6 (shared by 5 apps)
└── f7/
    └── f78e9...a012     ← myapp binary

Properties:

• Immutable: Once a blob is written, it never changes. The hash is the name.
• Shared: Every application references the same blobs. No duplication.
• Sharded: Two-character prefix directories prevent too many files in one folder.

3. Run

Read a manifest hash, assemble a rootfs, and execute via runc.

ll run 9503568f8b4f2d76b15b0e79c5c190f023b4e7c3c55c2be1f60b621f9a8a9992

sequenceDiagram
    participant User
    participant CLI as ll run
    participant Store as Content Store
    participant Bundle as OCI Bundle
    participant Runc as runc

    User->>CLI: ll run <manifest-hash>
    CLI->>Store: Get(manifest-hash)
    Store-->>CLI: Return manifest JSON
    CLI->>CLI: Parse manifest files[]
    loop For each file in manifest
        CLI->>Store: Get(file.hash)
        Store-->>CLI: Return blob
        CLI->>Bundle: Write to rootfs/path
    end
    CLI->>Bundle: Write config.json (OCI spec)
    CLI->>Runc: runc run -b <bundle-dir> <name>
    Runc->>Runc: Create Linux namespaces
    Runc->>Bundle: chroot into rootfs
    Runc-->>User: Process output

Loading

What happens under the hood:

• The manifest is fetched from the store by hash
• Each file listed in the manifest is copied from the store into a temporary rootfs/ directory
• An OCI config.json is written pointing to the rootfs and specifying the entrypoint
• runc (the standard OCI runtime) creates the container using Linux namespaces (pid, ipc, uts, mount, network)
• The container runs with no Docker daemon, no VM, just the host kernel

---

Comparison

Feature	Docker	Nix	Layerless
Base image required	Yes (100MB+)	No	No
File-level deduplication	No (layer-level)	Yes	Yes
Bandwidth on update	Full layer re-download	Minimal (closure diff)	Minimal (missing chunks)
Runtime complexity	Daemon + VM	None (just runs binaries)	runc only
Learning curve	Medium	Steep (Nix language)	Low
OCI compatible	Yes	No	Yes
Language	Go	C++ / Perl	Go

---

Quick Start

Prerequisites

• Go 1.23+
• Linux (for runc execution; macOS can pack and assemble bundles)
• runc installed (for running containers)

Build

git clone https://github.com/yourname/layerless
cd layerless
go build -o ll ./cmd/ll

Pack a static binary

# Build a static Go binary
CGO_ENABLED=0 go build -o hello ./cmd/hello

# Pack it
./ll pack -name=hello ./hello
# manifest: 9503568f8b4f2d76b15b0e79c5c190f023b4e7c3c55c2be1f60b621f9a8a9992
# files:    1
# store:    /Users/you/.local/share/layerless/store

Run the container

# On Linux
./ll run 9503568f8b4f2d76b15b0e79c5c190f023b4e7c3c55c2be1f60b621f9a8a9992

# On macOS (assemble bundle only, print runc command)
./ll run -bundle 9503568f8b4f2d76b15b0e79c5c190f023b4e7c3c55c2be1f60b621f9a8a9992
# bundle:   /tmp/layerless-...
# entry:    /bin/hello
# (on Linux) runc run -b /tmp/layerless-... hello

Inspect the store

ls ~/.local/share/layerless/store/
# 23  95  ab  f7

---

Project Structure

layerless/
├── cmd/ll/
│   └── main.go              # CLI entry point (pack, run)
├── internal/
│   ├── store/
│   │   └── store.go         # Content-addressable blob store
│   ├── manifest/
│   │   └── manifest.go      # Manifest types and serialization
│   └── bundle/
│       └── bundle.go        # OCI bundle assembly
├── go.mod
└── README.md

---

Roadmap

• Content-addressable store with SHA-256 sharding
• Pack command for static binaries
• Manifest generation and storage
• Run command with OCI bundle assembly
• runc integration (Linux)
• Dynamic binary dependency analysis (ldd, otool -L)
• Multi-file application packing (config files, assets)
• Cross-machine sync (push/pull missing chunks)
• Content-defined chunking (desync integration)
• Rootless container execution
• SBOM generation and signing (depsec integration)
• Registry protocol for remote stores

---

Why Not Just Use...?

Docker? Docker layers are tar.gz archives. Change one byte in a layer, re-download the whole layer. No file-level dedup across images.

Nix? Nix has the right storage model but is a package manager, not a runtime. It has no simple "run this in a container" command, no OCI compatibility, and requires learning the Nix language.

Flatpak/OSTree? Desktop-focused, not designed for server workloads or OCI compatibility.

WASM? Great for sandboxed compute, but limited to WASM-compiled languages. Can't run arbitrary Linux binaries.

---

License

MIT

---

Acknowledgments

• Inspired by the Nix store model and the OCI runtime spec
• Chunking strategy informed by desync (casync reimplementation in Go)
• OCI bundle assembly follows the runc standard