Skip to content

Feature request: persistent stdin input capability #431

Description

@wschwab

Summary

I would like to run a Zero program as a persistent line-delimited JSON worker:

  1. The parent writes one JSON request followed by \n.
  2. The Zero child reads that line without waiting for EOF.
  3. Zero writes one JSON response followed by \n.
  4. The child remains alive for subsequent requests.

Zero 0.3.4 can produce output through World.out, but I could not find a corresponding stdin capability. The language documentation explicitly says that hosted programs have no stdin and receive input through std.args and files through std.fs.

I therefore believe this is a feature request, not a report that the currently documented behavior is incorrect.

Environment

  • Zero: 0.3.4
  • Build: 5b3a90a
  • Host: linux-x64
  • Backend: zero-c

Intended use case

A neutral Node process owns HTTP transport and communicates with a long-lived
Zero domain worker over JSON Lines.

Example protocol:

Request:

{"id":"request-1","method":"GET","path":"/api/health","headers":{},"body":null}

Response:

{"id":"request-1","status":200,"headers":{"content-type":"application/json"},"body":
{"status":"ok"}}

The parent must be able to leave the child's stdin open and send another request later.

This pattern would also be useful for:

  • editor and language-server workers;
  • RPC subprocesses;
  • test harnesses;
  • build-system workers;
  • plugin processes;
  • durable agent tools;
  • any request/response protocol over pipes.

What works today

A one-shot read from /dev/stdin works when the writer closes the pipe:

  pub fn main(world: World) -> Void raises {
      let fs: Fs = std.fs.host()
      var input: owned<File> = check std.fs.openOrRaise(fs, "/dev/stdin")

      var buffer: [4096]u8 = [0_u8; 4096]
      let count: usize = check std.fs.readOrRaise(&mut input, buffer)

      let line: Maybe<Span<u8>> =
          std.io.nextLine(buffer[0..count], 0)

      if line.has {
          check world.out.write(line.value)
          check world.out.write("\n")
      }
  }

Running it with a pipe whose producer exits succeeds:

  printf '%s\n' \
    '{"id":"probe-1","method":"GET","path":"/api/health","body":null}' |
    zero run

Observed output:

  {"id":"probe-1","method":"GET","path":"/api/health","body":null}

However, this only demonstrates EOF-delimited, one-shot input.

Persistent-stream probe

The following Node probe starts the Zero child, writes one complete newline-terminated request, and deliberately keeps stdin open:

  import { spawn } from "node:child_process";

  const child = spawn("./run.sh", [], {
    stdio: ["pipe", "pipe", "pipe"],
  });

  let stdout = "";
  let stderr = "";

  child.stdout.on("data", chunk => {
    stdout += chunk;
  });

  child.stderr.on("data", chunk => {
    stderr += chunk;
  });

  child.stdin.on("error", () => {});

  child.stdin.write(
    '{"id":"stream-1","method":"GET","path":"/api/health",' +
    '"headers":{},"body":null}\n'
  );

  await new Promise(resolve => setTimeout(resolve, 2000));

  console.log({
    beforeClose: stdout,
    stderr,
  });

  if (child.exitCode === null && child.signalCode === null) {
    child.kill("SIGTERM");
  }

Observed result after two seconds:

  {
    "beforeClose": "",
    "stderr": ""
  }

No response is emitted while the pipe remains open.

Other APIs tested

I also investigated these approaches:

std.io.nextLine

std.io.nextLine scans a byte span already supplied by the caller. It does not appear to read from a process input stream.

std.fs.openOrRaise("/dev/stdin") with readOrRaise

This works for one-shot input after EOF, but did not provide the required request-at-a-time behavior while the pipe remained open.

std.fs.readBytes("/dev/stdin", buffer)

This did not provide an incremental persistent-stream response in the probe.

std.fs.readBytesAt("/dev/stdin", offset, buffer)

This also did not provide an incremental persistent-stream response. It appears designed for file-oriented offset reads rather than pipes.

Expected capability

A capability-gated stdin interface analogous to World.out would be ideal. For example, conceptually:

  pub fn main(world: World) -> Void raises {
      var storage: [8192]u8 = [0_u8; 8192]
      var reader = std.io.bufferedReader(world.in, storage)

      while true {
          let line: Maybe<Span<u8>> =
              check reader.nextLine()

          if !line.has {
              break
          }

          let response: Span<u8> =
              handle_json_request(line.value)

          check world.out.write(response)
          check world.out.write("\n")
          check world.out.flush()
      }
  }

The exact API does not need to match this example. The important properties are:

  • reads block until bytes or EOF are available;
  • a complete line can be returned before EOF;
  • unread bytes remain available for the next call;
  • stdout can be flushed after each response;
  • the program remains alive for multiple request/response cycles;
  • pipe behavior is portable across supported host platforms;
  • stdin access remains explicit in the program's capability signature.

Questions

  1. Is there already a supported API for incremental reads from process stdin?
  2. If so, could an example be added to zero skills get language or zero skills get stdlib?
  3. If not, would a World.in, WorldStream, or equivalent capability fit Zero's capability model?
  4. Is std.fs intentionally restricted to regular-file semantics?
  5. Does WorldStream, which appears in the language's recognized core types, relate to this use case?
  6. Is explicit output flushing needed or planned for subprocess protocols?

Workaround

The available workaround is to start a fresh Zero process for every request, write one request, close stdin, read the response, and let the process exit.

That is functionally possible, but it changes a lightweight persistent JSONL worker into a process-per-request protocol. It adds startup overhead and makes stateful or high-frequency subprocess integrations harder to implement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions