CartoCloud

Cloud infrastructure diagram generation and visualization tools.

Features

• PlantUML Diagram Builder: Generate AWS infrastructure diagrams using a code-based API with AWS stdlib components
• Infrastructure Documentation: Generate comprehensive markdown documentation from CloudMapper and Cartography data
• Multi-Format Support: Output to PlantUML and Mermaid diagram formats

Documentation Generator

The generate_docs.py script generates comprehensive AWS infrastructure documentation from CloudMapper and Cartography data using Jinja2 templates.

Quick Start

# Generate all documentation for an account
python src/generate_docs.py --account <account_name>

# Generate specific documentation types
python src/generate_docs.py --account <account_name> --type organizations --type networks

# Generate for a specific region
python src/generate_docs.py --account <account_name> --region eu-central-1

# Generate only PlantUML diagrams
python src/generate_docs.py --account <account_name> --format plantuml

# Skip Neo4j integration (CloudMapper data only)
python src/generate_docs.py --account <account_name> --no-neo4j

Documentation Types

Type	Description	Output
organizations	AWS Organization hierarchy with master/tenant accounts	organizations.md, diagrams
regions	Regions with custom-created resources and counts	regions.md
networks	VPCs with deployed resources and connectivity	networks.md, per-VPC diagrams
instances	EC2 instances with detailed connectivity info	instances.md, per-instance diagrams
vpc_overview	Per-VPC resource overview (subnets, RDS, LBs, IGW)	vpc_overview_{vpc}.md, diagrams
ecs_clusters	ECS clusters with services, containers, load balancers	ecs_{cluster}.md, diagrams
highlevel_4p1	One high-level 4+1 deployment doc per Environment (VPC, IGW, LB, ASG, EC2, RDS)	{env}-{project}-architecture.{md,puml,mmd} — see High-level 4+1

CLI Arguments

Argument	Required	Description
--account	Yes	Account name in account-data/ directory
--account-dir	No	Base directory for account data (default: account-data)
--type	No	Documentation type(s): organizations, regions, networks, instances, vpc_overview, ecs_clusters, highlevel_4p1, all (default: all). highlevel_4p1 is opt-in (not part of all).
--region	No	AWS region for instances (default: us-east-1)
--output-dir	No	Output directory (default: inventory_documentation)
--neo4j-uri	No	Neo4j connection URI (default: bolt://localhost:7687)
--no-neo4j	No	Skip Neo4j, use CloudMapper only
--format	No	Diagram format: plantuml, mermaid, both (default: both)
--environment	No	highlevel_4p1: render a single Environment tag value (bypasses the environments config)
--project	No	highlevel_4p1: Project tag value (required with --environment if config is unset)
--check-render	No	highlevel_4p1: after generation, render each .puml/.mmd via plantuml/mmdc and report OK/FAIL

Output Structure

inventory_documentation/{account}/
├── organizations.md          # Organization overview
├── organizations/
│   ├── hierarchy.puml      # PlantUML diagram
│   └── hierarchy.mmd       # Mermaid diagram
├── regions.md               # Region summaries
├── networks.md              # VPC documentation
├── networks/
│   └── {vpc_id}/
│       ├── diagram.puml    # VPC PlantUML diagram
│       └── diagram.mmd     # VPC Mermaid diagram
├── instances.md             # Instance documentation
└── instances/
    └── {instance_id}/
        ├── diagram.puml   # Instance connectivity diagram
        └── diagram.mmd    # Instance connectivity diagram

High-level 4+1 Architecture Diagrams (highlevel_4p1)

The highlevel_4p1 renderer produces one high-level 4+1 deployment/physical architecture document per Environment — a clean, DevOps-oriented picture of an environment's top-level infrastructure and how it connects. Each document carries dual diagrams: PlantUML (primary) + Mermaid (mirror).

In scope (only these are depicted): VPC, Internet Gateway, Load Balancer, Auto Scaling Group (with its member EC2 instances), standalone EC2 instances (not in any ASG), and RDS Database. Low-level objects (subnets, security groups, route tables, ECS, etc.) are deliberately omitted to keep the view uncluttered.

highlevel_4p1 requires the Cartography Neo4j graph (it cannot run with --no-neo4j) — the Environment/Project tags and the relationships live there.

Configuring Environments & Projects

Resources are selected by their AWS Environment and Project tags, read from the Cartography graph (tags are AWSTag nodes attached via :TAGGED). Declare which environments/projects to render per account in config.json — environments are nested under each account entry (not shared across accounts):

{
  "accounts": [
    { "id": "283115397952", "name": "premai", "default": true, "profile": "",
      "neo4j": { "uri": "bolt://localhost:7688", "database": "neo4j", "username": "neo4j", "password": "" },
      "environments": [
        { "name": "premapp-prod-eu", "region": "eu-west-1", "project": "premapp" },
        { "name": "premapp-dev",     "region": "eu-west-1", "project": "premapp" }
      ]
    }
  ]
}

Field	Meaning
name	The Environment tag value (also the <environment> part of the output filename)
project	The Project tag value (also the <project> part of the filename). Selection requires both Environment and Project to match on a resource
region	AWS region scoping the graph query

account is implicit from the parent accounts[] entry — no per-entry account field is needed. The environment belongs to the account that contains it.

Adding a new environment or project is a config-only change — no code edit. Two entries may share a project (or an environment name) as long as the (name, project) pair is unique.

Tagging notes (Cartography specifics): EC2 instances carry a Project tag but no Environment tag — they are recovered via their Auto Scaling Group name (e.g. premapp-prod-eu-runtime-hosts). AutoScalingGroup nodes are themselves untagged and matched by name prefix. RDS may be stored under the Database or RDSInstance label; Load Balancers under AWSLoadBalancerV2 or LoadBalancerV2 — the renderer unions both. Environments with partial tagging (e.g. premapp-dev has no tagged VPC/IGW) still render the objects that are present rather than failing.

Generating the documents

# All environments declared in config.json (one document set per Environment)
python src/generate_docs.py --account premai --type highlevel_4p1

# A single environment on demand (bypasses config)
python src/generate_docs.py --account premai --type highlevel_4p1 \
    --environment premapp-prod-eu --project premapp

# Interactive: pick account + environment with fzf (Esc on environment = render all configured)
make generate_highlevel

Each run prints a per-environment console run-summary (objects found / rendered / skipped / invalid).

Rendering & validating the diagrams

Sources are written under inventory_documentation/{account}/highlevel_4p1/ as {environment}-{project}-architecture.{md,puml,mmd}. Every diagram is syntax-validated before it is written (invalid artifacts are skipped and reported, never emitted). To render to SVG/PNG and/or authoritatively validate:

# Render all generated diagrams to SVG/PNG (PlantUML + Mermaid)
make render-diagrams

# Authoritatively validate every .puml/.mmd via plantuml/mmdc (exit code = validity)
make validate-diagrams

# Validate as part of generation (renders each emitted diagram, reports OK/FAIL)
python src/generate_docs.py --account premai --type highlevel_4p1 \
    --environment premapp-prod-eu --project premapp --check-render

Requires plantuml (Java) and mmdc (Node @mermaid-js/mermaid-cli) on PATH.

Data Sources

CloudMapper

Primary data source for AWS inventory. Expects data in account-data/{account}/{region}/ structure:

• ec2-describe-vpcs.json - VPCs
• ec2-describe-subnets.json - Subnets
• ec2-describe-instances.json - EC2 instances
• ec2-describe-security-groups.json - Security groups
• organizations-describe-organization.json - Organization details
• organizations-list-accounts.json - Account hierarchy

Cartography (Optional)

Neo4j-based graph database for enhanced relationship discovery:

• Load balancer to instance associations
• IAM role relationships
• Other cross-resource dependencies

Project Structure

src/
├── cartocloud/
│   └── builders/
│       └── plantuml/      # PlantUML diagram builder
├── inventory_shared/
│   ├── diagrams/           # Diagram generator modules
│   │   ├── organization.py
│   │   ├── vpc.py
│   │   └── instance.py
│   ├── template_context.py # Jinja2 context builders
│   ├── models.py           # Data models
│   ├── cloudmapper_loader.py # CloudMapper data loader
│   └── cartography_loader.py # Cartography integration
├── generate_docs.py         # Main CLI entry point
└── templates/               # Jinja2 templates
    ├── base/               # Base templates
    │   ├── layout.md
    │   └── macros.md
    └── pages/              # Page templates
        ├── organizations.md.j2
        ├── networks.md.j2
        ├── instances.md.j2
        └── regions.md.j2

Development

Setup

# Install dependencies (using poetry)
poetry install

Running Tests

# Generate documentation for test account
python src/generate_docs.py --account demo --type all --output-dir ./test_output

Template Customization

Templates are located in templates/ and use Jinja2 syntax. To customize:

1. Edit the template files in templates/pages/ or templates/base/
2. Context variables are provided by functions in src/inventory_shared/template_context.py
3. Diagram generators are in src/inventory_shared/diagrams/

Requirements

• Python 3.12+
• Jinja2
• (Optional) Neo4j for enhanced relationship discovery

License

[Add your license information here]