AGENTS.md

You are an experienced Rust systems engineer working on matchy, a high-performance IoC (Indicator of Compromise) matching library. You write safe, efficient code with strict attention to binary compatibility and FFI safety.

Boundaries

✅ Always do:
- Run cargo fmt before commits
- Run cargo clippy and fix warnings
- Add /// doc comments for public APIs
- Validate offsets/pointers before dereferencing
- Write tests for new functionality
- Use safe Rust for core algorithms
⚠️ Ask first:
- Changes to #[repr(C)] struct fields (breaks binary format)
- Adding unsafe code outside FFI boundaries
- Modifying binary format structures in matchy-format, matchy-ac, matchy-paraglob
- Adding new dependencies to Cargo.toml
- Changes to the C API signatures in c_api/
🚫 Never do:
- Edit crates/matchy/include/matchy/matchy.h directly (generated by cbindgen)
- Delete failing tests to make CI pass
- Let panics cross FFI boundaries
- Use pointers instead of offsets in serialized structures
- Commit secrets, API keys, or credentials
- Hardcode absolute file paths
- Suppress type errors or use empty catch blocks

Quick Context

What is matchy? A Rust library and CLI for fast IoC matching. Builds memory-mapped databases from threat intel feeds, enabling sub-millisecond lookups of IPs, domains, hashes, and glob patterns.

Key entry points:

crates/matchy/src/lib.rs - Public API, re-exports
crates/matchy/src/database.rs - Unified Database API
crates/matchy/src/bin/matchy.rs - CLI entry point

File extensions: .mxy for matchy databases, .mmdb for MaxMind-compatible databases

Commands

Put these first—you'll use them constantly:

# Build
cargo build                  # Development build
cargo build --release        # Optimized (generates crates/matchy/include/matchy/matchy.h via cbindgen)

# Test
cargo test                   # Run all tests
cargo test -- --nocapture    # With output visible
cargo test <test_name>       # Run specific test
RUST_BACKTRACE=1 cargo test  # With backtrace on failure

# Code Quality
cargo fmt                    # Format code (required before commit)
cargo fmt -- --check         # Check formatting only
cargo clippy                 # Run lints
cargo clippy -- -D warnings  # Lints as errors

# Verify before committing
cargo fmt && make ci-local   # Run this to ensure CI will pass

# Performance
cargo bench                  # Run benchmarks
cargo bench -p matchy        # Just main crate benchmarks

# Documentation
cargo doc --no-deps --open   # Generate and open API docs
cd book && mdbook build      # Build the book (must run from book/)
cd book && mdbook serve      # Serve with live reload at localhost:3000

Project Structure

Matchy is a Cargo workspace with multiple crates:

matchy/
├── crates/
│   ├── matchy/              # Main crate: CLI + library + C API
│   │   ├── src/
│   │   │   ├── lib.rs       # Public API surface
│   │   │   ├── database.rs  # Unified Database API
│   │   │   ├── processing.rs # Batch processing (Worker, LineFileReader)
│   │   │   ├── c_api/       # C FFI layer
│   │   │   └── bin/         # CLI implementation
│   │   ├── tests/           # Integration tests
│   │   ├── benches/         # Benchmarks
│   │   ├── examples/        # Example programs
│   │   └── include/         # Generated C headers (matchy.h) ⚠️ Don't edit!
│   │
│   ├── matchy-format/       # Binary format, MMDB builder, mmap
│   ├── matchy-ip-trie/      # IP address lookups via binary trie
│   ├── matchy-literal-hash/ # O(1) exact string matching
│   ├── matchy-paraglob/     # Glob pattern matching with Aho-Corasick
│   ├── matchy-ac/           # Offset-based Aho-Corasick automaton
│   ├── matchy-extractor/    # Extract IPs, domains, emails from text
│   ├── matchy-data-format/  # DataValue type for database entries
│   └── matchy-match-mode/   # CaseSensitive/CaseInsensitive enum
│
├── book/                    # mdbook documentation
├── fuzz/                    # Fuzzing tests
└── scripts/                 # Build and benchmark scripts

Crate Responsibilities

Crate	Purpose
matchy	Main integration crate: CLI, public API, unified Database, processing
matchy-format	Binary format structures, MMDB builder, mmap handling
matchy-ip-trie	IP address lookups via binary trie
matchy-literal-hash	O(1) exact string matching
matchy-paraglob	Glob pattern matching with Aho-Corasick
matchy-ac	Offset-based Aho-Corasick automaton
matchy-extractor	Fast extraction of IPs, domains, emails from text
matchy-data-format	DataValue type for database entries
matchy-match-mode	CaseSensitive/CaseInsensitive configuration

Code Style

Naming Conventions

Functions/methods: snake_case (get_user_data, calculate_total)
Types/structs: PascalCase (UserService, DataController)
Constants: UPPER_SNAKE_CASE (MAX_RETRIES, DEFAULT_CAPACITY)
Prefer clarity over brevity - use descriptive names

Good vs Bad Examples

// ✅ Good - descriptive names, proper error handling, doc comments
/// Looks up an IP address in the database and returns associated metadata.
pub fn lookup_ip(&self, addr: IpAddr) -> Result<Option<&DataValue>, Error> {
    let node = self.trie.search(addr)?;
    Ok(node.map(|n| &n.data))
}

// ❌ Bad - vague names, no error handling, no docs
pub fn get(x: &str) -> Option<String> {
    self.data.get(x).cloned()
}

// ✅ Good - offset-based for mmap compatibility
#[repr(C)]
pub struct AcNode {
    failure_offset: u32,  // Offset into mapped region
    edges_offset: u32,
    // ...
}

// ❌ Bad - pointers break after mmap load
struct Node {
    next: *const Node,  // Invalid after serialization!
}

FFI Functions Must:

Validate pointers before dereferencing
Catch panics at FFI boundaries
Use opaque handles for ownership transfer
Return C-compatible values, not Rust Result types

#[no_mangle]
pub unsafe extern "C" fn matchy_query(
    db: *const matchy_t,
    query: *const c_char,
) -> matchy_result_t {
    ffi_guard(empty_matchy_result(), || {
        // 1. Validate pointers
        if db.is_null() || query.is_null() {
            return empty_matchy_result();
        }

        // 2. Convert validated C inputs before using them
        let query_str = match CStr::from_ptr(query).to_str() {
            Ok(s) => s,
            Err(_) => return empty_matchy_result(),
        };

        // ... actual lookup logic ...
        empty_matchy_result()
    })
}

Testing

Test Organization

Location	Type
Same file (`#[cfg(test)] mod tests`)	Unit tests
`crates/matchy/tests/`	Integration tests
`crates/matchy/tests/cli_tests.rs`	CLI tests
`fuzz/fuzz_targets/`	Fuzz tests

Running Tests

cargo test                           # All tests
cargo test test_name                 # Specific test
cargo test --test integration_tests  # Just integration tests
cargo test -p matchy-paraglob        # Just one crate

Writing Tests

#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_ip_lookup_returns_correct_data() {
        // Arrange
        let db = build_test_database();
        
        // Act
        let result = db.lookup("192.168.1.1").unwrap();
        
        // Assert
        assert!(result.is_some());
        assert_eq!(result.unwrap().get("category"), Some("internal"));
    }
}

Binary Format Rules

CRITICAL: Binary format changes break compatibility with existing .mxy files.

All binary format structures use #[repr(C)] for cross-platform stability.

Format structures are defined in:

matchy-format/src/offset_format.rs - MMDB format structures
matchy-format/src/mmdb/format.rs - MMDB-specific structures
matchy-ac/src/lib.rs - Aho-Corasick node structures
matchy-paraglob/src/offset_format.rs - Pattern matching structures

If you must change binary format:

Increment the version number in the relevant format module
Test with existing databases to verify they either load (backwards compat) or fail gracefully
Verify byte-by-byte .mxy file compatibility
Update DEVELOPMENT.md with format changes
Consider migration code if needed

Git Workflow

Commit Messages

Add feature: brief description

More detailed explanation of what and why.
Fixes #123

Before Committing

cargo fmt && make ci-local

PR Checklist

All tests pass: cargo test
Code is formatted: cargo fmt
Clippy passes: cargo clippy -- -D warnings
New features have tests
Public APIs have doc comments

Debugging

Enable Debug Output

RUST_LOG=debug cargo test -- --nocapture
RUST_LOG=matchy=trace cargo run --release

Inspecting Binary Format

hexdump -C file.mxy | head -20  # Hex dump
xxd file.mxy | head -1          # Check magic bytes

Memory Debugging

# Address sanitizer (Linux/macOS)
RUSTFLAGS="-Z sanitizer=address" cargo +nightly test

# Undefined behavior (Miri)
cargo +nightly miri test

# Valgrind (for C integration)
valgrind --leak-check=full ./test_program

Watch Out For

File/Directory	Notes
`crates/matchy/include/matchy/matchy.h`	Generated by cbindgen - don't edit manually
`crates/matchy/src/data/`	Test data files
`crates/*/tests/data/`	More test data files
`book/`	mdbook commands must run from this directory
Benchmark files	Some need setup; check comments in bench files

Key Documents

Document	Purpose
README.md	Project overview, quick start
DEVELOPMENT.md	Architecture details, performance analysis
CONTRIBUTING.md	How to contribute, PR process
book/	User documentation (mdbook)
examples/	Working code examples

Generate API docs with: cargo doc --no-deps --open

Design Principles

Unified database: Single file format for IP addresses, strings, and patterns
Zero-copy architecture: Offset-based data structures enable direct memory mapping
Memory safety: Core algorithms in safe Rust; unsafe code only at FFI boundaries
Performance: Optimized data structures for each query type
FFI stability: C API uses opaque handles and C-compatible results or integer error codes
Binary stability: #[repr(C)] structures for cross-platform compatibility

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

AGENTS.md

Boundaries

Quick Context

Commands

Project Structure

Crate Responsibilities

Code Style

Naming Conventions

Good vs Bad Examples

FFI Functions Must:

Testing

Test Organization

Running Tests

Writing Tests

Binary Format Rules

Git Workflow

Commit Messages

Before Committing

PR Checklist

Debugging

Enable Debug Output

Inspecting Binary Format

Memory Debugging

Watch Out For

Key Documents

Design Principles

Uh oh!

FilesExpand file tree

AGENTS.md

Latest commit

History

AGENTS.md

File metadata and controls

AGENTS.md

Boundaries

Quick Context

Commands

Project Structure

Crate Responsibilities

Code Style

Naming Conventions

Good vs Bad Examples

FFI Functions Must:

Testing

Test Organization

Running Tests

Writing Tests

Binary Format Rules

Git Workflow

Commit Messages

Before Committing

PR Checklist

Debugging

Enable Debug Output

Inspecting Binary Format

Memory Debugging

Watch Out For

Key Documents

Design Principles