Skip to content

VNC & Desktop Streaming

For both roles: Warmwind streams desktop content from their custom OS to the user's browser via VNC. The Linux Platform role builds this pipeline. The Backend role integrates it (WebSocket transport, session management).

Warmwind's Streaming Pipeline

sequenceDiagram
    participant App as Desktop App
    participant Sway as Sway Compositor
    participant VNC as WayVNC
    participant WS as WebSocket Proxy
    participant Browser as User's Browser

    App->>Sway: Render to wl_surface
    Sway->>VNC: wlr-screencopy (frame capture)
    VNC->>VNC: Encode (ZRLE, Tight, or H.264)
    VNC->>WS: RFB protocol (TCP)
    WS->>Browser: WebSocket frames
    Browser->>Browser: Decode + render to Canvas
    Browser->>WS: Key/mouse events
    WS->>VNC: RFB input messages
    VNC->>Sway: Virtual keyboard/pointer

VNC/RFB Protocol Basics

The Remote Framebuffer (RFB) protocol (RFC 6143) is simple:

  1. Server captures framebuffer, encodes changed regions, sends to client
  2. Client requests updates, sends keyboard/mouse events
  3. No audio, no USB, no file transfer in base protocol (unlike RDP)

Key encodings (trade-off: compression ratio vs CPU cost):

Encoding Mechanism Best for
Raw Uncompressed pixels Fast LAN
ZRLE zlib + RLE + palette, 64x64 tiles General purpose
Tight Filtered zlib + JPEG Low bandwidth
H.264 Video codec Continuous motion

WayVNC Architecture

WayVNC is purpose-built for wlroots compositors:

  • Captures frames via wlr-screencopy protocol extension
  • Tracks damage (only encodes changed pixels)
  • Rate-limits capture (doesn't overwhelm the encoder)
  • Input injection via wlr-virtual-keyboard and wlr-virtual-pointer
  • Control socket at $XDG_RUNTIME_DIR/wayvncctl (JSON IPC)

Browser Delivery

VNC speaks raw TCP. Browsers speak WebSocket. Bridge options:

Approach How Warmwind likely uses
websockify + noVNC Simple proxy + JS VNC client Possible (lightweight)
Apache Guacamole Full gateway with server-side rendering optimization Possible (more features)
Custom WebSocket relay NestJS WebSocket gateway + custom Canvas renderer Most likely (fits their NestJS stack)

Glossary

RFB (Remote Framebuffer)
The protocol underlying VNC. Defined in RFC 6143. A simple framebuffer-oriented protocol: server sends pixel rectangles, client sends input events.
websockify
A proxy that translates WebSocket connections to raw TCP. Sits between a browser (which can only do WebSocket) and a VNC server (which speaks raw TCP).
noVNC
A JavaScript VNC client that runs in the browser. Uses HTML5 Canvas for rendering. Connects via WebSocket (through websockify).
wlr-screencopy
A wlroots Wayland protocol extension. Allows a privileged client to request copies of the compositor's output buffer. WayVNC uses this to capture frames for streaming.