|
| 1 | +# Heatmap: Real-Time Order Book Visualisation |
| 2 | + |
| 3 | +[](https://www.typescriptlang.org/) |
| 4 | +[](https://www.solidjs.com/) |
| 5 | +[](https://vitejs.dev/) |
| 6 | + |
| 7 | +A high-performance, multi-threaded financial data visualisation tool for rendering real-time order book heatmaps. This project implements a local order book engine synced with the Binance BTC/USDT spot market, providing traders with deep insights into market liquidity and volume clusters. |
| 8 | + |
| 9 | +## Table of Contents |
| 10 | + |
| 11 | +- [Key Features](#key-features) |
| 12 | +- [Architecture](#architecture) |
| 13 | +- [Tech Stack](#tech-stack) |
| 14 | +- [Key Technical Decisions](#key-technical-decisions) |
| 15 | +- [Challenges & Lessons](#challenges--lessons) |
| 16 | +- [Installation & Usage](#installation--usage) |
| 17 | +- [License](#license) |
| 18 | + |
| 19 | +--- |
| 20 | + |
| 21 | +## Key Features |
| 22 | + |
| 23 | +- **Multi-Threaded Pipeline:** Utilises Web Workers to decouple data ingestion (`DataWorker`) from rendering (`RenderWorker`), ensuring the main UI thread remains responsive even during high-volatility market events. |
| 24 | +- **High-Performance Binning:** Employs a zero-allocation `HeatmapProcessor` that uses pre-allocated `TypedArrays` to process thousands of order book levels into normalised heatmap slices in sub-10ms intervals. |
| 25 | +- **Rank-Based Normalisation:** Implements a rank-transform algorithm to map power-law distributed order book volumes into a 5-step discrete intensity scale (Weather Radar style) for maximum visual clarity. |
| 26 | +- **Sub-Pixel Smooth Rendering:** Features a custom "shift-and-draw" rendering engine on an `OffscreenCanvas` with float-based coverage blending for sub-pixel accuracy. |
| 27 | +- **Full Order Book Integrity:** Maintains a local order book with 5000 levels per side, featuring snapshot reconciliation and sequence gap detection to ensure 100% data accuracy relative to the exchange. |
| 28 | +- **Adaptive UI:** Includes interactive controls for zoom, pan, auto-centring, and volume filtering, alongside dynamic price and time axes. |
| 29 | + |
| 30 | +--- |
| 31 | + |
| 32 | +## Architecture |
| 33 | + |
| 34 | +The system follows a reactive, decoupled architecture where the main thread orchestrates a direct `MessageChannel` between the data and render workers. |
| 35 | + |
| 36 | +### Directory Structure |
| 37 | + |
| 38 | +```text |
| 39 | +src/ |
| 40 | +├── components/ # Layout-level SolidJS components |
| 41 | +├── core/ # Application services (HeatmapService) and context |
| 42 | +├── engine/ # Core business logic (Binning, Normalisation, Book Core) |
| 43 | +│ ├── bookCore.ts # Order book data structures |
| 44 | +│ ├── processor.ts # Heatmap binning & rank normalisation logic |
| 45 | +│ └── palettes.ts # Colour mappings (Magma, Viridis) |
| 46 | +├── ui/ # UI components (Axes, Legend, Controls) and Hooks |
| 47 | +│ └── useHeatmapPipeline.ts # Orchestrates worker communication |
| 48 | +├── workers/ # Multi-threading logic |
| 49 | +│ ├── data.worker.ts # Binance API connector & processing |
| 50 | +│ └── render.worker.ts # OffscreenCanvas rendering engine |
| 51 | +└── index.tsx # Application entry point |
| 52 | +``` |
| 53 | + |
| 54 | +### System Flow |
| 55 | + |
| 56 | +```mermaid |
| 57 | +graph TD |
| 58 | + subgraph "External API" |
| 59 | + B_WS[Binance WebSocket] |
| 60 | + B_REST[Binance REST API] |
| 61 | + end |
| 62 | +
|
| 63 | + subgraph "DataWorker (Background Thread)" |
| 64 | + BC[BinanceConnector] |
| 65 | + HP[HeatmapProcessor] |
| 66 | + Book[Local Order Book] |
| 67 | + |
| 68 | + BC -- 1. Snapshot --> B_REST |
| 69 | + BC -- 2. Stream --> B_WS |
| 70 | + BC -- 3. Update --> Book |
| 71 | + Book -- 4. Raw Data --> HP |
| 72 | + HP -- 5. Processed Slice --> Port1((MessagePort)) |
| 73 | + end |
| 74 | +
|
| 75 | + subgraph "RenderWorker (Background Thread)" |
| 76 | + Port2((MessagePort)) |
| 77 | + HR[HeatmapRenderer] |
| 78 | + Canvas[OffscreenCanvas] |
| 79 | + History[Circular Buffer] |
| 80 | +
|
| 81 | + Port1 -- "Direct Channel (No Main Thread)" --> Port2 |
| 82 | + Port2 -- 6. Receive Slice --> History |
| 83 | + History -- 7. Render --> HR |
| 84 | + HR -- 8. Draw --> Canvas |
| 85 | + end |
| 86 | +
|
| 87 | + subgraph "Main Thread (SolidJS UI)" |
| 88 | + Hook[useHeatmapPipeline] |
| 89 | + UI[Dashboard & Controls] |
| 90 | + Overlay[Price/Time Axes] |
| 91 | +
|
| 92 | + Hook -- Control Messages --> BC |
| 93 | + Hook -- Control Messages --> HR |
| 94 | + Canvas -- Frame --> UI |
| 95 | + HR -- Viewport Stats --> Hook |
| 96 | + Hook -- Update --> UI |
| 97 | + end |
| 98 | +``` |
| 99 | + |
| 100 | +--- |
| 101 | + |
| 102 | +## Tech Stack |
| 103 | + |
| 104 | +| Category | Tools | |
| 105 | +| :--- | :--- | |
| 106 | +| **Framework** | SolidJS | |
| 107 | +| **Language** | TypeScript | |
| 108 | +| **Build Tool** | Vite | |
| 109 | +| **Rendering** | OffscreenCanvas (2D Context) | |
| 110 | +| **Concurrency** | Web Workers, MessageChannel | |
| 111 | +| **Data Source** | Binance API (WebSocket & REST) | |
| 112 | + |
| 113 | +--- |
| 114 | + |
| 115 | +## Key Technical Decisions |
| 116 | + |
| 117 | +| Decision | Logic & Reasoning | |
| 118 | +| :--- | :--- | |
| 119 | +| **Web Workers** | Decoupling ingestion and rendering prevents UI "jank" and ensures the application can handle the high message throughput of the crypto markets without blocking the event loop. | |
| 120 | +| **OffscreenCanvas** | Offloading the canvas rendering to a background thread allows the `RenderWorker` to maintain a consistent 60 FPS regardless of main-thread load. | |
| 121 | +| **TypedArrays** | Using `Float64Array` and `Int32Array` in the hot path minimizes heap allocations, reducing the overhead of Garbage Collection (GC) which is critical for low-latency financial apps. | |
| 122 | +| **Rank Normalisation** | Order book volumes often follow a power-law distribution. Linear scaling results in either oversaturated or invisible data. Rank-based scaling ensures consistent visual contrast. | |
| 123 | +| **Discrete Quantisation** | Mapping intensities to 5 discrete levels (Weather Radar style) simplifies visual scanning for traders, making it easier to identify significant support/resistance levels. | |
| 124 | + |
| 125 | +--- |
| 126 | + |
| 127 | +## Challenges & Lessons |
| 128 | + |
| 129 | +### Local Order Book Synchronisation |
| 130 | +Maintaining a perfectly synced local copy of the order book requires handling the sequence of a REST snapshot followed by real-time WebSocket updates. Implementing the Binance-specific reconciliation logic (tracking `U` and `u` update IDs) was a critical challenge that reinforced the importance of sequence validation and error-handling in high-frequency data streams. |
| 131 | + |
| 132 | +### Memory Management in Workers |
| 133 | +Transferring large amounts of data between workers can be expensive. By using a direct `MessageChannel` between the `DataWorker` and `RenderWorker`, the main thread is bypassed entirely for the high-frequency "Render Slice" messages, significantly reducing context-switching overhead. |
| 134 | + |
| 135 | +--- |
| 136 | + |
| 137 | +## Installation & Usage |
| 138 | + |
| 139 | +### Prerequisites |
| 140 | + |
| 141 | +- [Node.js](https://nodejs.org/) (v18 or higher) |
| 142 | +- [npm](https://www.npmjs.com/) or [pnpm](https://pnpm.io/) |
| 143 | + |
| 144 | +### Setup |
| 145 | + |
| 146 | +1. Clone the repository: |
| 147 | + ```bash |
| 148 | + git clone https://github.com/your-username/heatmap.git |
| 149 | + cd heatmap |
| 150 | + ``` |
| 151 | + |
| 152 | +2. Install dependencies: |
| 153 | + ```bash |
| 154 | + npm install |
| 155 | + ``` |
| 156 | + |
| 157 | +3. Start the development server: |
| 158 | + ```bash |
| 159 | + npm run dev |
| 160 | + ``` |
| 161 | + |
| 162 | +4. Build for production: |
| 163 | + ```bash |
| 164 | + npm run build |
| 165 | + ``` |
| 166 | + |
| 167 | +--- |
| 168 | + |
| 169 | +## License |
| 170 | + |
| 171 | + |
| 172 | +Copyright (c) 2026 Sajid Ahmed. **All Rights Reserved.** |
| 173 | + |
| 174 | +This repository is a **Proprietary Project**. |
| 175 | + |
| 176 | +While I am a strong supporter of Open Source Software, this specific codebase represents a significant personal investment of time and effort and is therefore provided with the following restrictions: |
| 177 | + |
| 178 | +* **Permitted:** Viewing, forking (within GitHub only), and local execution for evaluation and personal, non-commercial usage only. |
| 179 | +* **Prohibited:** Modification, redistribution, commercial use, and AI/LLM training. |
| 180 | + |
| 181 | +For the full legal terms, please see the [LICENSE](./LICENSE) file. |
0 commit comments