NL2HLTL

a quick-start guide for the three-stage pipeline of this repository:

1. Stage 1: NL -> HTT (Task Decomposition)
2. Stage 2: HTT -> HLTL (Proposition & Temporal Compilation)
3. Stage 3: HLTL -> STAP (AI2THOR Planning & Execution)

---

1. Environment Setup

1. Activate Python Environment

conda activate nl2hltl

2. OpenAI Configuration

• Default model settings are located in configs/pipeline_ai2thor.json.
• Supports access via port 10808 proxy (the proxy_url is reserved in the config).
• The API key is read by default from the key file in the repository root.

---

2. Execution Entry Point

The main entry script is: run_repro.py

Common Commands:

1. Full Pipeline Run (Stage 1 -> Stage 2 -> Stage 3)

python run_repro.py --config configs/pipeline_ai2thor.json

2. Limit Task Quantity (For quick testing)

python run_repro.py --config configs/pipeline_ai2thor.json --limit 5

3. Disable LLM (Uses heuristic-based Stage 1)

python run_repro.py --config configs/pipeline_ai2thor.json --disable-llm

---

3. Stage-wise Execution & Cache Reuse

The pipeline supports starting from specific stages and reusing historical results to avoid re-running Stage 1 or 2.

Parameters:

• --start-stage {1,2,3}: Start from the specified stage.
• --reuse-cached: Reuse existing Stage 1/Stage 2 data in outputs/repro_run3/<task>/result.json.
• --stage3-timeout <sec>: Timeout limit for a single Stage 3 attempt.

Examples:

1. Start from Stage 3 Only (Reuse existing Stage 1/2)

python run_repro.py --config configs/pipeline_ai2thor.json --start-stage 3 --reuse-cached

2. Start from Stage 2 (Keep Stage 1, re-compile Stage 2 + Stage 3)

python run_repro.py --config configs/pipeline_ai2thor.json --start-stage 2 --reuse-cached

---

4. Output Directory Structure

Default output root: outputs/repro_run3

Each task directory (e.g., FloorPlan303.tasks_1.compose_4.fintune_1) contains:

• result.json: Full structured results for all three stages.
• result.md: Human-readable stage-by-stage report.
• ai2thor_constraints.json: Scene constraints for the current task.
• ai2thor_object_lookup.json: Bidirectional index for symbol/objectId/name.
• ai2thor_containment.json: Relationships between receptacles and children.

Batch Summary Files:

• summary.json / summary.md
• failure_analysis.json / failure_analysis.md

---

5. Interpreting Results

Stage 1

• Check if the task_tree hierarchy is logical.
• Check ltl_pattern_check_passed and pattern_violations.

Stage 2

• Leaf tasks should only use two types of propositions:
  • Object proposition: object_symbol
  • Action proposition: action000object_symbol
• Verify checks.syntax_ok and checks.affordance_ok.

Stage 3

• On Success: Note stage3.success, stage3.cost, stage3.completion_time, and stage3.plan (step-by-step locations and actions).
• On Failure: Check stage3.message, stage3.planning_retries, and the statistical breakdown in failure_analysis.md.

---

6. AI2THOR Proposition Conventions

Reference: env_catalog/ai2thor_action_reference.md

Conventions:

• Navigation tasks use object propositions (e.g., shelf1).
• Action tasks use action propositions (e.g., pickup000creditcard1).
• The 000 string is strictly used as a separator between the action and the object.

---

7. Project Structure Overview

• run_repro.py: Main pipeline entry.
• nl2hltl_pipeline/stage1_decompose.py: Stage 1 logic.
• nl2hltl_pipeline/stage2_nl2ltl.py: Stage 2 logic.
• nl2hltl_pipeline/stage3_stap_ai2thor.py: Stage 3 logic.
• Hierarchical-LTL-STAP/: The core STAP planner.
• env_catalog/: Scene objects, affordances, and action mappings.

---

📚 Citation

If you find this work useful in your research, please consider citing:

@article{xu2025nl2hltl2plan,
  title     = {NL2HLTL2Plan: Scaling up Natural Language Understanding for Multi-Robots
               Through Hierarchical Temporal Logic Task Specifications},
  author    = {Xu, Shaojun and Luo, Xusheng and Huang, Yutong and Leng, Letian
               and Liu, Ruixuan and Liu, Changliu},
  journal   = {IEEE Robotics and Automation Letters},
  year      = {2025},
  publisher = {IEEE}
}