vertical-partition-system-plugin/README.md

# Vertical Partition Streaming

> **This is not an infinite open-world partition system.**
> **This is a bounded vertical route partition system for a hand-crafted spiral climbing map** (Only Up / Only Climb style).

It takes an *already finished* level, scans the actors inside a volume, segments them
into vertical **Z cells** (optionally **XY sub-cells**), bakes a **descriptor**, generates
**offline HLOD proxies**, and streams the route at runtime with a vertical-aware policy
(full / preload / HLOD / unloaded bands, hysteresis, velocity prediction, fall-risk-below),
plus World-Partition-style debug visualization.

It is **not** World Partition, it is **not** Level Instances, and it does **no** runtime
mesh merging. Full cells use classic persistent-level + streaming-sublevel loading (or, in
the default safe mode, visibility toggling of the existing actors).

---

## 1. How it understands the task

| Requirement | How it's met |
|---|---|
| Bounded vertical route, spiral, handcrafted | `AVerticalPartitionVolume` (a box) defines the playable region; cells are slices of Z. |
| Convert an existing map | `UVerticalPartitionBuilder` scans the level in place — nothing is authored from scratch. |
| World-Partition *by meaning*, not infinite XY | Z is the primary axis; XY sub-cells are optional refinement. |
| Full / HLOD / preload / unload ranges | `FVerticalPartitionSettings` + `UVerticalPartitionStatics::GetDesiredStateForCell`. |
| Offline HLOD, no runtime merge | `MeshMergeUtilities` at build time → per-cell proxy `UStaticMesh`. |
| Two conversion modes | `NonDestructive` (ships first, safe) and `DestructiveCommit` (V2). |
| WP-style debug | `UVerticalPartitionDebugComponent` (3D boxes + overlay) + `vp.*` console. |

---

## 2. Architecture

```
EDITOR (VerticalPartitionEditor module)            RUNTIME (VerticalPartition module)
┌───────────────────────────┐                      ┌────────────────────────────────┐
│ AVerticalPartitionVolume   │  CallInEditor btns   │ AVerticalPartitionManager       │
│  • box bounds + settings   │──┐  via bridge        │  • finds tracked pawn           │
│  • Descriptor ref          │  │                    │  • builds streaming context     │
└───────────────────────────┘  │                    │  • drives subsystem @ interval  │
            ▲                   ▼                    │  • hosts DebugComponent         │
            │        FVerticalPartitionEditorBridge  └───────────────┬────────────────┘
            │                   │                                     │ UpdateStreaming()
┌───────────┴────────────┐     ▼                    ┌────────────────▼────────────────┐
│ UVerticalPartitionBuilder│  Analyze/Build/...      │ UVerticalPartitionRuntimeSubsystem│
│  • GatherActors          │────────────────────────▶│  • per-cell state machine        │
│  • BuildCells            │  writes                  │  • streaming-level OR visibility  │
│  • GenerateHLODProxy     │     ┌──────────────────┐ │  • proxy actors, async budget     │
│  • CreateBackup/Commit   │────▶│ UVerticalPartition │◀│  • FVerticalRuntimeStats          │
└──────────────────────────┘     │ Descriptor (asset) │ └───────────────────────────────────┘
                                  │  • cells + bounds  │
UVerticalPartitionCommandlet ────▶│  • settings snapshot│   UVerticalPartitionStatics (pure math,
 (offline -run=VerticalPartition) │  • last report     │    shared by editor + runtime)
                                  └──────────────────┘
```

**Module split** keeps the heavy editor APIs (mesh merge, level surgery) out of packaged
builds. The volume lives in the runtime module so it can be streamed/placed at runtime, and
its editor buttons reach the builder through `FVerticalPartitionEditorBridge` (a `TFunction`
the editor module installs on startup).

---

## 3. Editor conversion pipeline

1. Open the finished map.
2. Drop an **`AVerticalPartitionVolume`** around the whole playable spiral; scale its box.
3. Tune **Settings** on the volume (segmentation, ranges, HLOD, large-actor policy, exclude classes, mode).
4. Press the detail-panel buttons (or `vp.*`):
   - **Analyze** → report only (no writes): counts, cells, largest cell, warnings.
   - **Build** → backup → scan → classify → `VPD_<Map>` descriptor → HLOD proxies → save → spawn a manager.
   - **Validate / Preview Chunks / Rebuild HLOD / Rebuild Changed / Clear Generated / Commit / Restore Backup / Dump Report**.
5. Press **Play** — the manager registers the descriptor with the subsystem and streaming starts.
6. Toggle debug (`vp.Debug 1`).

**Generated content** lives under `Settings.GeneratedPackageRoot` (default `/Game/VPGenerated/<Map>/`):
`VPD_<Map>` (descriptor) and `Proxies/HLOD_Cell_Zxxxx` (proxy meshes). Backups go to
`Saved/VerticalPartition/Backups/`.

### Conversion modes
- **NonDestructive (default, ships first):** the persistent level is untouched. The descriptor
  records each cell's source actors by soft path; at runtime far cells are **hidden**
  (`SetActorHiddenInGame` + collision/tick off) and their HLOD proxy shown. Zero risk, fully
  reversible, validates the whole pipeline. *No memory savings* (actors stay resident).
- **DestructiveCommit (V2):** actors are baked into per-cell streaming sub-level `.umap`
  packages and removed from the persistent level → true memory streaming. A backup is taken
  first. The exact baking steps are documented in `CommitConversion()` and below; the runtime
  already supports this backend (a cell with a valid `FullCellPackage` switches to streaming).

> **Recommended MVP: NonDestructive.** It's safe, gives draw-call savings + correct streaming
> behavior, and lets you tune ranges before committing to destructive level surgery.

---

## 4. Runtime streaming pipeline

`AVerticalPartitionManager::Tick` (throttled to `UpdateInterval`) →
builds `FVerticalStreamingContext` (player Z cell, velocity Z, camera pitch, teleport flag) →
`UVerticalPartitionRuntimeSubsystem::UpdateStreaming`:

1. **PollStreaming** each cell (refresh state from in-flight async ops).
2. **Desired state** per cell = `GetDesiredStateForCell` (or forced / show-only).
3. **Budget pass** — nearest `MaxFullCells` stay Full, nearest `MaxVisibleHLODCells` stay HLOD, the rest demote.
4. **Apply**, nearest-first, within `MaxAsyncLoadsPerFrame` / `MaxAsyncUnloadsPerFrame`:
   - `LoadCellFullAsync(visible)` — streaming backend: `SetShouldBeLoaded/Visible`.
   - `ShowCellHLOD` — spawn/show the proxy `AStaticMeshActor`.
   - `UnloadCell` — hide actors / unload streaming level / hide proxy.
5. **Rebuild stats**, record update time, flag hitches.

`bSnap` (teleport / checkpoint / first frame) bypasses the per-frame budget for instant convergence.

---

## 5. Streaming ranges (all editable per-volume)

Bands are measured in **cells from the player cell**. Example with `Full=±1`, `Preload above=3`,
`HLOD=±... ` and player in **Z10**:

```
Z15  ── Unloaded ───────────────
Z14  ─┐
Z13  ─┤ HLOD (proxy visible)
Z12  ─┘ (+ Preload: full data resident but hidden)
Z11  ─┐
Z10  ─┤ FULL  ← player
Z9   ─┘
Z8   ─┐ HLOD
Z7   ─┘
Z6   ── Unloaded ───────────────
```

Key knobs (see `FVerticalPartitionSettings`):
`FullLoadCells{Below,Above}`, `PreloadCells{Below,Above}`, `HLODLoadCells{Below,Above}`,
`UnloadCells{Below,Above}`, `HysteresisDistanceZ`, `HysteresisCells`, `UpdateInterval`,
`MaxAsyncLoads/UnloadsPerFrame`, `MaxVisibleHLODCells`, `MaxFullCells`,
`bPrioritizeAbovePlayer`, `bKeepBelowPlayerLoadedOnFallRisk`, `bUseCameraDirectionPriority`,
`bUsePlayerVelocityPrediction`, `VelocityPredictionSeconds`.

`GetDesiredStateForCell` accounts for: bands, **sticky hysteresis** (a loaded cell at the edge
stays loaded for `HysteresisCells` extra), **velocity prediction** (shifts the effective player
cell), **camera direction**, **fall-risk-below** (keeps below-player cells loaded to the unload
edge), forced cells and show-only mode. Teleport is handled by the manager raising `bSnap`.

---

## 6. Data structures (`VerticalPartitionTypes.h`)

`FVerticalCellId{X,Y,Z}` · `EVerticalCellState{Unloaded,LoadingHLOD,HLOD,PreloadingFull,LoadingFull,Full,Unloading,Error}` ·
`EVerticalCellDesiredState{Unloaded,HLOD,Full}` · `EVerticalConversionMode{NonDestructive,DestructiveCommit}` ·
`ELargeActorPolicy{KeepInPersistentLevel,AssignToDominantCell,DuplicateAsHLODOnly,SplitNotSupportedWarning,ExcludeFromPartition}` ·
`FVerticalPartitionSettings` (segmentation + streaming + conversion + HLOD + debug) ·
`FVerticalCellDescriptor` · `FVerticalPartitionWarning` / `FVerticalPartitionError` ·
`FVerticalPartitionBuildReport` · `FVerticalRuntimeStats`.

---

## 7. Classes

| Class | Module | Role |
|---|---|---|
| `AVerticalPartitionVolume` | Runtime | Bounds + settings + descriptor ref + editor buttons. |
| `UVerticalPartitionDeveloperSettings` | Runtime | Project-default settings + global debug (Project Settings → Plugins). |
| `UVerticalPartitionStatics` | Runtime | Pure cell math + the desired-state policy. |
| `UVerticalPartitionDescriptor` | Runtime | The build artifact (cells, bounds, settings snapshot, report). |
| `UVerticalPartitionRuntimeSubsystem` | Runtime | The streaming state machine + stats. |
| `AVerticalPartitionManager` | Runtime | Per-level coordinator; drives the subsystem; hosts debug. |
| `UVerticalPartitionDebugComponent` | Runtime | 3D cell boxes + on-screen overlay. |
| `UVerticalPartitionBuilder` | Editor | Analyze / Build / RebuildHLOD / Validate / Commit / Backup. |
| `UVerticalPartitionCommandlet` | Editor | `-run=VerticalPartition -Map=...` offline build. |

---

## 8. HLOD / proxy strategy

- **Offline only.** Built during Build / Rebuild HLOD via `IMeshMergeUtilities::MergeComponentsToStaticMesh`
  over each cell's **static** mesh components → one proxy `UStaticMesh` per cell, saved under `Proxies/`.
- The runtime spawns one hidden `AStaticMeshActor` per cell from that mesh and toggles its visibility
  for the HLOD band; Full load hides the proxy, so there's no double-draw.
- **Exclusions (never proxied):** dynamic / movable components, physics actors, gameplay actors,
  moving platforms, lights, audio. Proxy collision is off.
- **Graceful degrade:** if a cell has no static geometry or merge fails, the cell still streams —
  the far representation just hides instead of showing a proxy, and a warning is recorded.

**Method (`Settings.HLODMethod`):**
- **MergedMesh** (default) — `MergeComponentsToStaticMesh`, one merged mesh with original
  materials; ISM/HISM instances are auto-flattened into the single object.
- **SimplifiedProxy** — `CreateProxyMesh` (proxy LOD: simplified geometry + one baked atlas
  material, the cheapest "true HLOD", same tech as Epic's HLOD *Simplified Mesh*). Requires the
  **ProxyLODPlugin** enabled; auto-falls-back to MergedMesh with a warning if absent.

### Level Instances & Packed Level Actors (universal)
- **Packed Level Actor** — its baked `ISM/HISM` live as owned components, caught by
  `GetComponents<UStaticMeshComponent>` and merged (instances flattened to one object).
- **Level Instance** — `CollectStreamLeafActors` recursively flattens the LI (runtime API
  `ULevelInstanceSubsystem::ForEachActorInLevelInstance`, gated on `IsLoaded`) into its static
  child actors; nested LIs recurse; nested PLAs stay whole. Its static geometry is baked per
  cell; its static children are hidden/shown at runtime; **lights / gameplay inside the LI stay
  persistent**. Scan skips LI content (`GetOwningLevelInstance`) so nothing is double-counted.
  An **unloaded** LI is skipped with a warning — load it in the editor and rebuild.
  Toggle with `Settings.bStreamLevelInstances` (default on).

---

## 9. Debug / validation

**Console:** `vp.Debug`, `vp.DebugDraw`, `vp.DebugOverlay` (CVars); commands
`vp.Analyze`, `vp.BuildFromSelectedVolume`, `vp.Validate`, `vp.Rebuild`, `vp.RebuildHLOD`,
`vp.ForceCellFull/HLOD <Z>`, `vp.UnforceCell <Z>`, `vp.UnforceAll`, `vp.ShowOnlyCell <Z>`,
`vp.TeleportToCell <Z>`, `vp.SetFullRange <b> <a>`, `vp.SetHLODRange <b> <a>`,
`vp.SetCellHeight <cm>`, `vp.DumpState`, `vp.DumpCells`, `vp.DumpWarnings`, `vp.ProfileStreaming`.

**3D draw colors:** Green=Full · Blue=HLOD · Yellow=Loading/Transition · Red=Error · Gray=Unloaded · Purple=Forced/locked.
**Overlay:** player Z cell, full/HLOD/unloaded counts, loading queue, async ops, est. memory, update ms, hitches.
**Editor preview:** *Preview Chunks* draws the Z slices in the editor viewport without entering PIE.

**Validation warnings** (analyze/build): crosses-multiple-cells, too-large, physics,
blueprint-tick, dynamic-mobility, audio/light/nav, replicated/not-stream-safe, cell-too-many-actors,
HLOD-generation-failed, package-save-failed, descriptor-out-of-date.

---

## 10. Edge cases & how they're handled

| Case | Handling |
|---|---|
| Actor crosses Z cells | Assigned to dominant (centre) cell + `ActorCrossesMultipleCells` warning. |
| Origin in one cell, bounds in another | `bUseActorBoundsInsteadOfOrigin` picks the rule; assignment uses bounds centre. |
| Attached actors split across cells | Grouped per cell in the descriptor; cross-cell attaches are a documented warning to review. |
| Blueprint tick / physics / interactable / movable | Flagged; `LargeActorPolicy`/exclude classes keep them in the persistent level. |
| SaveGame / replicated actors | `ActorNotStreamSafe` warning — keep in persistent level (NonDestructive leaves them resident). |
| Hard ref into another cell | `Dependencies` array on the descriptor (dependency graph is a V2 expansion). |
| Decor visible far / silhouette | That's the HLOD proxy's job. |
| Light in one cell lighting another | Lights flagged; recommended kept in persistent level or Full cell, not proxied. |
| Level Instance / Packed Level Actor | Universal: PLA's HISM and LI's flattened static children baked per-cell HLOD + streamed; LI lights/gameplay stay persistent. `bStreamLevelInstances`. |
| HISM / ISM (incl. inside PLA) | Merge auto-expands instances into one merged proxy object. |
| Unloaded Level Instance at build | Skipped with a warning — load it in the editor and rebuild. |
| Navmesh on a vertical map | Out of scope of streaming; keep navmesh in the persistent level. |
| Player falls 10 cells / flies up fast | Velocity prediction + `bKeepBelowPlayerLoadedOnFallRisk` + `bSnap` on big jumps. |
| Teleport / checkpoint | Manager detects the jump (>3 cells) → `bSnap` immediate convergence. |
| Multiplayer | Streaming is per-world around the local tracked pawn (client-side); replicated actors flagged. Server-authoritative union streaming is V2. |
| PIE vs packaged | Editor module is `Type:Editor` (excluded from packaged); runtime is self-contained. |
| Package load failure | Cell → `Error` state (red), logged. |
| HLOD gen failure | Warning; cell still streams (hides far). |
| Source map changed after build | `Validate` flags `DescriptorOutOfDate` via content hash. |
| Rebuild only changed cells | `RebuildChangedCells` (currently full rebuild; incremental diff is V2). |

---

## 11. MVP roadmap

**MVP (implemented):** one volume · Z-only segmentation · `CellHeight` · Analyze · Build
(NonDestructive) · descriptor asset · runtime manager + subsystem · full load/unload via
visibility · offline HLOD proxy for static-mesh actors · Z-slice debug draw · stats overlay ·
warnings for large/dynamic/blueprint/physics actors · full `vp.*` console.

**V2 (scaffolded / documented):** XY sub-cells (settings + math in place) · DestructiveCommit
baking (`CommitConversion` stub + documented steps) · incremental rebuild (dependency graph) ·
full HLOD LOD chains · commandlet hardening · multiplayer union streaming · velocity/camera
preload tuning · memory profiler integration (Insights) · editor tool *window* (today: detail
buttons + console).

---

## 12. Build & enable

New C++ **modules** were added, so a Live-Coding pass is not enough — regenerate project files
and do a full rebuild with the editor closed, then enable the plugin (it's `EnabledByDefault`),
or add to `IHY.uproject`:

```json
{ "Name": "VerticalPartition", "Enabled": true }
```

Files live under `Plugins/VerticalPartition/Source/{VerticalPartition,VerticalPartitionEditor}/`.