Skip to content

07-PERFORMANCE.md

Latest commit

 

History

History
815 lines (603 loc) · 21.4 KB

07-PERFORMANCE.md

File metadata and controls

815 lines (603 loc) · 21.4 KB
Raw

ProRT-IP WarScan: Performance Baselines and Optimization

Version: 1.0 Last Updated: October 2025

Table of Contents

  1. Performance Targets
  2. Benchmark Baselines
  3. Profiling and Measurement
  4. Optimization Techniques
  5. Platform-Specific Optimizations
  6. Performance Testing

Performance Targets

Primary Goals

Metric Target Rationale
Throughput (Stateless) 1,000,000+ pps Comparable to Masscan, enables IPv4-wide scans
Throughput (Stateful) 50,000+ pps Balance accuracy with speed for deep scans
Memory (Stateless) <100 MB Constant memory regardless of target count
Memory (Stateful) <1 GB for 1M targets Scalable to large networks
CPU Efficiency Linear scaling to 16+ cores Multi-core utilization
Latency <1 ms packet crafting Minimal overhead per packet

Comparative Benchmarks

Based on published performance data:

Tool Packets/Second Notes
Masscan 10,000,000 Stateless, single machine, 10GbE
ZMap 14,230,000 97% hit rate at 4Mpps, 63% at 14Mpps
Nmap (aggressive) ~300,000 Stateful with timing T4-T5
RustScan ~65,535 ports in 3s ~21,800 pps (stateless discovery)
Target: WarScan Stateless 1,000,000+ 10x faster than Nmap, 10% of Masscan
Target: WarScan Stateful 50,000+ 150x faster than Nmap

Benchmark Baselines

Packet Crafting Performance

Baseline: Measured on AMD Ryzen 9 5950X (16C/32T), 32GB RAM, Linux 6.1

// benches/packet_crafting.rs

TCP SYN Packet Building
  Time:     [850.23 ns 862.41 ns 875.19 ns]
  Throughput: ~1,160,000 packets/sec (single thread)

UDP Packet Building
  Time:     [620.15 ns 628.92 ns 638.47 ns]
  Throughput: ~1,590,000 packets/sec (single thread)

ICMP Echo Packet Building
  Time:     [480.37 ns 487.23 ns 494.86 ns]
  Throughput: ~2,050,000 packets/sec (single thread)

Checksum Calculation (TCP)
  Time:     [95.42 ns 96.78 ns 98.21 ns]
  Throughput: ~10,330,000 checksums/sec

Interpretation: Single-threaded packet crafting significantly exceeds target throughput. Multi-threaded scaling should achieve 1M+ pps easily.

Scanning Throughput

Test Scenario: Scan 10.0.0.0/16 (65,536 hosts), port 80, SYN scan

Stateless Mode (Target)
  Total packets:  65,536
  Duration:       ~65 ms (1,000,000 pps)
  Memory:         <50 MB
  CPU cores used: 4-8

Stateful Mode (Target)
  Total packets:  65,536 (initial) + retransmits
  Duration:       ~1.3 seconds (50,000 pps)
  Memory:         ~150 MB (connection tracking)
  CPU cores used: 4-8

Memory Benchmarks

Operation Memory Usage Notes
Base binary ~5 MB Minimal static footprint
Stateless scan (1M targets) <100 MB O(1) state via SipHash
Stateful scan (1K active conns) ~50 MB ~50KB per connection
Stateful scan (100K active conns) ~5 GB Connection state dominates
Result storage (1M entries) ~250 MB In-memory before DB write
OS fingerprint DB ~10 MB 2,000+ fingerprints loaded
Service probe DB ~5 MB 500+ probes loaded

Latency Targets

Operation Target Latency Acceptable Range
Packet crafting <1 ms 0.5-2 ms
DNS resolution <50 ms 10-100 ms (network dependent)
TCP connect scan <100 ms RTT dependent
SYN scan (single port) <10 ms 5-20 ms
Service detection (single port) <500 ms 200-1000 ms
OS fingerprinting <2 sec 1-5 sec (16 probes)

Profiling and Measurement

CPU Profiling with perf

# Build with debug symbols in release mode
RUSTFLAGS="-C debuginfo=2 -C force-frame-pointers=yes" cargo build --release

# Record performance data (requires root or perf_event_paranoid=-1)
sudo perf record --call-graph dwarf -F 997 \
    ./target/release/prtip -sS -p 1-1000 10.0.0.0/24

# Generate flamegraph
perf script | stackcollapse-perf.pl | flamegraph.pl > flame.svg

# Interactive analysis
perf report

Key Metrics to Monitor:

  • CPU cycles in packet crafting functions (<10% of total)
  • Cache misses in hot paths (<5% L1d misses)
  • Branch mispredictions (<2% of branches)
  • Lock contention (should be minimal with lock-free design)

Memory Profiling with Valgrind

# Heap profiling with massif
valgrind --tool=massif \
    --massif-out-file=massif.out \
    ./target/release/prtip -sS -p 80,443 10.0.0.0/24

# Analyze results
ms_print massif.out > massif.txt
less massif.txt

# Memory leak detection
valgrind --leak-check=full \
    --show-leak-kinds=all \
    --track-origins=yes \
    ./target/debug/prtip [args]

Expected Results:

  • Definitely lost: 0 bytes (no memory leaks)
  • Possibly lost: <1KB (from static initializers)
  • Peak heap usage: Matches expected memory targets above

Criterion.rs Benchmarks

# Run all benchmarks
cargo bench

# Run specific benchmark group
cargo bench --bench packet_crafting

# Compare against baseline
cargo bench --save-baseline before
# ... make changes ...
cargo bench --baseline before

# View HTML report
firefox target/criterion/report/index.html

Example Output:

tcp_syn_packet          time:   [850.23 ns 862.41 ns 875.19 ns]
                        change: [-2.3421% -1.1234% +0.4521%] (p = 0.18 > 0.05)
                        No change in performance detected.

udp_packet              time:   [620.15 ns 628.92 ns 638.47 ns]
                        change: [-3.1234% -2.5678% -1.9876%] (p = 0.00 < 0.05)
                        Performance has improved.

Optimization Techniques

1. Lock-Free Data Structures

Problem: Mutex contention limits scalability beyond 4-8 cores

Solution: Use crossbeam lock-free queues for task distribution

use crossbeam::queue::SegQueue;

// Replace Mutex<VecDeque<Task>>
// With:
let task_queue: Arc<SegQueue<Task>> = Arc::new(SegQueue::new());

// Workers can push/pop without locks
task_queue.push(task);
if let Some(task) = task_queue.pop() {
    // process task
}

Impact: 3-5x throughput improvement on 16+ core systems

Phase 4 Sprint 4.2 Implementation (v0.3.0+):

As of v0.3.0, the following lock-free optimizations have been implemented:

  1. SYN Scanner Connection Table (DashMap)

    • Replaced Arc<Mutex<HashMap<(Ipv4Addr, u16, u16), ConnectionState>>> with Arc<DashMap<(Ipv4Addr, u16, u16), ConnectionState>>
    • File: crates/prtip-scanner/src/syn_scanner.rs (line 69)
    • Eliminates lock contention during concurrent SYN scans
    • DashMap uses sharded locking internally for O(1) concurrent access
    • Zero performance regression, all 551 tests passing

  2. Adaptive Rate Limiter (Atomic Operations)

    • Replaced Arc<Mutex<AdaptiveState>> with atomic fields
    • File: crates/prtip-scanner/src/timing.rs (lines 221-237)
    • Key changes:
      • current_rate_mhz: AtomicU64 (millihertz for precision)
      • consecutive_timeouts: AtomicUsize
      • successful_responses: AtomicUsize
      • last_adjustment_micros: AtomicU64
    • Uses compare_exchange_weak loops for rate adjustments (AIMD algorithm)
    • RTT statistics still use Arc<Mutex<RttStats>> (complex operations require it)
    • Lock-free fast path for common operations: wait(), report_response()

Expected Performance Impact:

  • 10-30% throughput improvement on multi-core scans
  • Reduced CPU cycles in synchronization primitives (<5% target)
  • Better scaling to 10+ cores
  • Network benchmarking needed to measure real-world impact

Benchmarking Plan:

  • Requires Metasploitable2 Docker container for realistic network latency
  • Measure before/after lock contention with perf record -e lock:contention_begin
  • Compare CPU utilization across cores
  • Validate linear scaling to 10+ cores

2. SIMD for Checksum Calculation

Problem: Checksum calculation is CPU-intensive at high packet rates

Solution: Use SIMD instructions for parallel addition

#[cfg(target_arch = "x86_64")]
use std::arch::x86_64::*;

pub fn fast_checksum(data: &[u8]) -> u16 {
    unsafe {
        let mut sum = _mm_setzero_si128();

        // Process 16 bytes at a time
        for chunk in data.chunks_exact(16) {
            let bytes = _mm_loadu_si128(chunk.as_ptr() as *const __m128i);
            sum = _mm_add_epi16(sum, bytes);
        }

        // Horizontal sum and fold
        // ... (reduction logic)
    }
}

Impact: 2-3x faster checksum calculation

3. Memory Pooling for Packet Buffers

Problem: Allocating buffers per-packet causes allocator contention

Solution: Pre-allocate buffer pool, reuse buffers

use crossbeam::queue::ArrayQueue;

struct PacketBufferPool {
    buffers: ArrayQueue<Vec<u8>>,
}

impl PacketBufferPool {
    fn new(size: usize, count: usize) -> Self {
        let buffers = ArrayQueue::new(count);
        for _ in 0..count {
            buffers.push(vec![0u8; size]).ok();
        }
        Self { buffers }
    }

    fn acquire(&self) -> Option<Vec<u8>> {
        self.buffers.pop()
    }

    fn release(&self, mut buf: Vec<u8>) {
        buf.clear();
        self.buffers.push(buf).ok();
    }
}

Impact: Reduces allocation overhead by 80%+

Phase 4 Sprint 4.17 Implementation (v0.3.8+):

As of v0.3.8, ProRT-IP has implemented zero-copy packet building with thread-local buffer pools:

  1. PacketBuffer Infrastructure

    • File: crates/prtip-network/src/packet_buffer.rs (251 lines)
    • Thread-local buffer pools (4KB buffers per thread)
    • Safe closure-based API: with_buffer(|pool| { ... })
    • Automatic buffer reuse via pool.reset()
    • Zero contention between threads (thread-local storage)

  2. Zero-Copy Packet Builders

    • TcpPacketBuilder::build_with_buffer() - Returns &[u8] slice (zero-copy)
    • UdpPacketBuilder::build_with_buffer() - Returns &[u8] slice (zero-copy)
    • build_ip_packet_with_buffer() - Builds complete IPv4 + TCP/UDP packets
    • Old API preserved for backward compatibility

  3. Scanner Integration (Proof-of-Concept)

    • SYN scanner integrated (file: crates/prtip-scanner/src/syn_scanner.rs)
    • Pattern: Wrap packet building in with_buffer() closure
    • Remaining scanners: UDP, stealth, decoy, OS probe (~3.5 hours scoped)

Example Usage:

use prtip_network::{TcpPacketBuilder, TcpFlags, packet_buffer::with_buffer};
use std::net::Ipv4Addr;

with_buffer(|pool| {
    let packet = TcpPacketBuilder::new()
        .source_ip(Ipv4Addr::new(10, 0, 0, 1))
        .dest_ip(Ipv4Addr::new(10, 0, 0, 2))
        .source_port(12345)
        .dest_port(80)
        .flags(TcpFlags::SYN)
        .build_ip_packet_with_buffer(pool)?;

    // Use packet slice (e.g., send via raw socket)
    send_packet(packet)?;

    pool.reset();  // Reuse buffer for next packet
    Ok::<(), Box<dyn std::error::Error>>(())
})?;

Performance Impact:

Metric Old API (allocates) Zero-Copy Improvement
Per-packet time 68.3 ns 58.8 ns 15% faster
Allocations 3-7 per packet 0 per packet 100% reduction
CPU cycles (1K packets) 209K 180K 29K saved
Throughput (theoretical) 14.6M pps 17.0M pps +2.4M pps

Real-World Impact at 1M pps:

  • Allocations eliminated: 3-7 million per second → 0
  • Memory pressure: Zero heap fragmentation
  • Predictability: Zero allocator contention
  • Scalability: Benefits increase at higher packet rates

Benchmarking:

  • Criterion.rs benchmarks in benches/packet_crafting.rs
  • Statistical validation: 50-100 samples, p < 0.05 confidence
  • Run benchmarks: cargo bench --bench packet_crafting

Migration Guide:

Remaining scanners can adopt zero-copy by following the SYN scanner pattern:

  1. Add use prtip_network::packet_buffer::with_buffer;
  2. Wrap packet building in with_buffer(|pool| { ... }) closure
  3. Replace .build() with .build_ip_packet_with_buffer(pool)
  4. Add pool.reset() after packet sent
  5. Return Ok::<_, Error>(()) from closure

Estimated migration time: 30-90 minutes per scanner (total ~3.5 hours for all remaining scanners).

4. Batched System Calls

Problem: System call overhead dominates at high packet rates

Solution: Use sendmmsg/recvmmsg to batch operations (Linux)

use libc::{sendmmsg, recvmmsg, mmsghdr, iovec};

pub fn send_packet_batch(fd: RawFd, packets: &[Vec<u8>]) -> Result<usize> {
    let mut msgvec: Vec<mmsghdr> = packets.iter().map(|pkt| {
        let mut msg: mmsghdr = unsafe { std::mem::zeroed() };
        let mut iov: iovec = iovec {
            iov_base: pkt.as_ptr() as *mut _,
            iov_len: pkt.len(),
        };
        msg.msg_hdr.msg_iov = &mut iov;
        msg.msg_hdr.msg_iovlen = 1;
        msg
    }).collect();

    let sent = unsafe {
        sendmmsg(fd, msgvec.as_mut_ptr(), msgvec.len() as u32, 0)
    };

    if sent < 0 {
        Err(io::Error::last_os_error())
    } else {
        Ok(sent as usize)
    }
}

Impact: 5-10x reduction in syscall overhead

5. NUMA-Aware Thread Placement

Problem: Cross-NUMA memory access penalties (10-30% slowdown)

Solution: Pin threads to NUMA nodes matching network interfaces

use libc::{cpu_set_t, sched_setaffinity, CPU_SET, CPU_ZERO};

pub fn pin_thread_to_core(core: usize) -> Result<()> {
    unsafe {
        let mut cpuset: cpu_set_t = std::mem::zeroed();
        CPU_ZERO(&mut cpuset);
        CPU_SET(core, &mut cpuset);

        let result = sched_setaffinity(
            0, // current thread
            std::mem::size_of::<cpu_set_t>(),
            &cpuset,
        );

        if result == 0 {
            Ok(())
        } else {
            Err(io::Error::last_os_error())
        }
    }
}

// Usage in worker pool
for (i, worker) in workers.iter().enumerate() {
    let core = numa_node_cores[i % numa_nodes];
    worker.spawn(move || {
        pin_thread_to_core(core).unwrap();
        // ... worker logic
    });
}

Impact: 10-30% improvement on multi-socket systems

6. Adaptive Batching

Problem: Fixed batch sizes suboptimal for varying network conditions

Solution: Dynamically adjust batch size based on success rate

struct AdaptiveBatcher {
    current_batch_size: usize,
    min_batch: usize,
    max_batch: usize,
    success_rate: f64,
}

impl AdaptiveBatcher {
    fn adjust(&mut self, successes: usize, total: usize) {
        self.success_rate = successes as f64 / total as f64;

        if self.success_rate > 0.95 {
            // Increase batch size (less overhead)
            self.current_batch_size = (self.current_batch_size * 110 / 100)
                .min(self.max_batch);
        } else if self.success_rate < 0.80 {
            // Decrease batch size (better responsiveness)
            self.current_batch_size = (self.current_batch_size * 90 / 100)
                .max(self.min_batch);
        }
    }

    fn batch_size(&self) -> usize {
        self.current_batch_size
    }
}

Impact: 15-25% improvement in variable network conditions

Platform-Specific Optimizations

Linux

AF_PACKET with PACKET_MMAP

Zero-copy packet capture using memory-mapped ring buffers:

use libc::{AF_PACKET, SOCK_RAW, setsockopt, SOL_PACKET, PACKET_MMAP};

// Create ring buffer for RX
let req = tpacket_req {
    tp_block_size: 4096,
    tp_frame_size: 2048,
    tp_block_nr: 256,
    tp_frame_nr: 512,
};

unsafe {
    setsockopt(
        fd,
        SOL_PACKET,
        PACKET_MMAP,
        &req as *const _ as *const c_void,
        std::mem::size_of::<tpacket_req>() as u32,
    );
}

// mmap the ring buffer
let buffer = unsafe {
    libc::mmap(
        std::ptr::null_mut(),
        req.tp_block_size * req.tp_block_nr,
        libc::PROT_READ | libc::PROT_WRITE,
        libc::MAP_SHARED,
        fd,
        0,
    )
};

Impact: 30-50% reduction in CPU usage for packet capture

eBPF/XDP for Ultimate Performance

For 10M+ pps, leverage XDP (eXpress Data Path):

// xdp_filter.c - simple example

SEC("xdp")
int xdp_scan_filter(struct xdp_md *ctx) {
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;

    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return XDP_DROP;

    if (eth->h_proto != htons(ETH_P_IP))
        return XDP_PASS;

    struct iphdr *ip = data + sizeof(*eth);
    if ((void *)(ip + 1) > data_end)
        return XDP_DROP;

    // Only accept packets to our scanner (reduces userspace overhead)
    if (ip->daddr == htonl(SCANNER_IP)) {
        return XDP_PASS;
    }

    return XDP_DROP;
}

Impact: 24M+ pps per core with hardware offload

Windows

Npcap Optimization

// Use SendPacketEx for better performance
#[cfg(target_os = "windows")]
pub fn send_packets_windows(handle: *mut pcap_t, packets: &[Vec<u8>]) -> Result<()> {
    use npcap_sys::*;

    unsafe {
        for packet in packets {
            // Use SendPacketEx instead of SendPacket for better performance
            let result = pcap_sendpacket(
                handle,
                packet.as_ptr(),
                packet.len() as i32,
            );

            if result != 0 {
                return Err(Error::PacketSendFailed);
            }
        }
    }

    Ok(())
}

Impact: 20-30% improvement over standard SendPacket

macOS

BPF Buffer Sizing

use libc::{ioctl, BIOCSBLEN};

pub fn optimize_bpf_buffer(fd: RawFd) -> Result<()> {
    // Increase buffer size for better batching
    let bufsize: i32 = 1024 * 1024; // 1MB

    unsafe {
        if ioctl(fd, BIOCSBLEN, &bufsize) < 0 {
            return Err(io::Error::last_os_error());
        }
    }

    Ok(())
}

Impact: Reduces packet loss at high rates

Performance Testing

Throughput Test Suite

#!/bin/bash
# scripts/perf_test.sh

echo "=== ProRT-IP WarScan Performance Test Suite ==="

# Test 1: Single port, many hosts
echo "Test 1: Scanning 10.0.0.0/16 port 80..."
time ./target/release/prtip -sS -p 80 --max-rate 100000 10.0.0.0/16

# Test 2: Many ports, single host
echo "Test 2: Scanning 127.0.0.1 all ports..."
time ./target/release/prtip -sS -p- 127.0.0.1

# Test 3: Stateless vs Stateful comparison
echo "Test 3: Stateless scan..."
time ./target/release/prtip --stateless -p 80 10.0.0.0/24

echo "Test 3: Stateful scan (same targets)..."
time ./target/release/prtip -sS -p 80 10.0.0.0/24

# Test 4: Memory usage monitoring
echo "Test 4: Memory usage (1M targets)..."
/usr/bin/time -v ./target/release/prtip --stateless -p 80,443 0.0.0.0/0 \
    | grep "Maximum resident set size"

Load Testing

// tests/load_test.rs

#[test]
fn load_test_sustained_throughput() {
    let target_pps = 100_000;
    let duration = Duration::from_secs(60); // 1 minute sustained

    let scanner = Scanner::new(ScanConfig {
        max_rate: target_pps,
        ..Default::default()
    }).unwrap();

    let start = Instant::now();
    let mut packets_sent = 0;

    while start.elapsed() < duration {
        packets_sent += scanner.send_batch().unwrap();
    }

    let actual_pps = packets_sent / duration.as_secs() as usize;

    // Allow 5% variance
    assert!(actual_pps >= target_pps * 95 / 100);
    assert!(actual_pps <= target_pps * 105 / 100);
}

Regression Detection

# .github/workflows/performance.yml

name: Performance Regression Check

on: [pull_request]

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0  # Need history for comparison

      - name: Run benchmarks (baseline)
        run: |
          git checkout main
          cargo bench --bench packet_crafting -- --save-baseline main

      - name: Run benchmarks (PR)
        run: |
          git checkout ${{ github.head_ref }}
          cargo bench --bench packet_crafting -- --baseline main

      - name: Check for regression
        run: |
          # Fail if any benchmark regresses >5%
          cargo bench --bench packet_crafting -- --baseline main \
            | grep "change:.*-.*%" && exit 1 || exit 0

Performance Troubleshooting

Symptom: Low throughput (<10K pps)

Possible Causes:

  1. Running without root/capabilities (falling back to connect scan)
  2. Network interface limit (check with ethtool)
  3. CPU bottleneck (check with htop)

Debug:

# Check privileges
getcap ./target/release/prtip

# Check NIC speed
ethtool eth0 | grep Speed

# Profile to find bottleneck
perf top

Symptom: High CPU usage (>80% on all cores)

Possible Causes:

  1. Inefficient packet parsing
  2. Lock contention
  3. Allocation overhead

Debug:

# Profile CPU usage
perf record -g ./target/release/prtip [args]
perf report

# Look for:
# - High time in __pthread_mutex_lock
# - High time in malloc/free
# - Hot loops in packet parsing

Symptom: Memory growth over time

Possible Causes:

  1. Connection state not being cleaned up
  2. Result buffer not flushing
  3. Memory leak

Debug:

# Check for leaks
valgrind --leak-check=full ./target/debug/prtip [args]

# Monitor memory over time
watch -n 1 'ps aux | grep prtip'

Next Steps