dorf/INVESTIGATION_HANDOFF.md

Investigation Handoff: Dorf Job Queue System

Current State

System Overview

This is a dwarf (dorf) simulation with a job queue system, think dwarf fortress in a chunked world a la minecraft. We are early in development and are currently debugging the job queue system. The expected DEMO flow is:

1. ONE FellTree job should always exist (when none is in progress)
2. When a tree falls, X logs spawn, each creating a HaulCargo job
3. All available dorfs (not busy felling) should take HaulCargo jobs based on the jobs priorities
4. No dorfs should be idle while haul jobs exist to assign

Latest Test Results (output15.log)

• Progress: One tree successfully chopped, one haul completed!
• Improvement: The is_standable cargo fix and ambulatory.target clearing fix allowed dorfs to path to cargo
• Remaining Issue: Only 2 of 5 dorfs participated; others froze with stale targets from previous Idle task

Fixes Applied This Session

1. is_standable for Cargo Tiles (src/world/tiles/tilemap.rs line 311-319)

   • Added: if self.cargo_tiles.contains_key(&world_pos) { return true; }
   • Cargo (logs, rocks, ores) is now standable - dorfs can walk on cargo to pick it up

2. Clear ambulatory.target on Job Assignment (src/entities/tasks/job_pathfinding.rs line 305)

   • Added: ambulatory.target = None; when assigning new task
   • Fixes "Lost Signal" bug where old Idle task targets persisted

Known Issues After Fixes

1. Some dorfs still freeze - 3 of 5 dorfs had stale targets and weren't moving
2. Z-level spawn delay - Dorfs fall ~4 seconds before becoming standable (expected)
3. Distances don't decrease - Some dorfs show arrival checks with unchanging dist_sq

Root Cause Analysis

Issue 1: "Lost Signal" Bug (FIXED)

When job_pathfinding.rs assigns a new task to a dorf, it clears queue and sets state to Pending, but was NOT clearing ambulatory.target. The old target from the Idle task persisted, causing pathfinding to route to the wrong location.

Fix: Clear ambulatory.target = None in job assignment.

Issue 2: Cargo Not Standable (FIXED)

The is_standable() function checked floor/fixture bitsets but ignored the cargo_tiles HashMap. Logs couldn't be stood on for pickup.

Fix: Check cargo_tiles.contains_key() first in is_standable().

Issue 3: Target Not Being Set (POSSIBLE)

When ambulatory.target = None, the executor should set target from approach in HaulStep::MovingToCargo. But if pathfinding clears the target before executor sets it, dorfs freeze.

Potential fix: Ensure executor sets target before depending on it.

Key Files Modified

src/world/tiles/tilemap.rs           - Added cargo check to is_standable()
src/entities/tasks/job_pathfinding.rs - Clear ambulatory.target on assign

Key Files to Investigate

Job Systems

src/entities/tasks/job_pathfinding.rs  - Job assignment, candidate filtering
src/entities/tasks/job_queue.rs         - Job queue state management  
src/entities/tasks/executor.rs         - Task execution (ChopTree, HaulCargo)
src/entities/tasks/demo.rs              - FellTree job creation logic

Pathfinding & Movement

src/entities/shared_systems/pathfinding.rs  - Entity movement, is_standable checks
src/world/tiles/tilemap.rs                   - is_standable(), find_nearest_free_cargo_tile()

Debug Logs Present

// job_pathfinding.rs - Assignment success
info!("[PATHFIND] SUCCESS: Assigned job {:?} to dorf {:?} with approach {:?}", job_id, dorf_entity, approach_target);

// executor.rs - HaulCargo arrival check
info!("[HAUL] {:?} arrival check: approach={:?} dist_sq={} arrive_sq={} transform={:?}", entity, approach, dist_sq, arrive_sq, transform.truncate());

// executor.rs - ChopTree state
info!("[EXECUTOR] ChopTree {:?}: trunk_pos={:?} approach={:?} target={:?}", entity, trunk_pos, approach, ambulatory.target);

Key Patterns to Watch

Arrival Distance Check

• arrive_sq = (ITILE_SIZE * 1.5)^2 = 576.0 (square pixels)
• dist_sq must be <= 576 to transition to PickingUp
• If dist_sq doesn't change between frames, dorf isn't moving

Stale Target Pattern

approach=IVec3(A, B, C) target=Vec3(X, Y, Z)  where (X,Y,Z) != (A,B,C)

This shows the target doesn't match the approach - old target persists.

Remaining Investigation

Why Some Dorfs Freeze

1. Check if target is set from approach in executor when target.is_none():

   • executor.rs lines 502-516 sets target from approach
   • If pathfinding clears target between frames, executor needs to re-set it

2. Check movement system - Is current_path being populated?

   • pathfinding.rs line 659: returns if current_path.is_none()
   • Need to see if paths are being calculated

3. Check if dorfs are falling - falling clears target and path:

   • pathfinding.rs lines 654-656: clears target when not standable

Why Different Distances Between Dorfs

Looking at output15.log output15:

• dorf 8891v0: dist_sq=2007296.0 → dist_sq=2049280.0 (distances INCREASING!)
• dorf 8890v0: dist_sq=848896.0 (constant - not moving)
• dorf 8888v0: dist_sq=312320.0 (constant - not moving)
• dorf 8889v0: dist_sq=726016.0 (constant - not moving)

The distances are NOT DECREASING, meaning dorfs are not pathing/moving toward cargo.

Possible Quick Fixes to Try

1. Force target recalculation in executor when arrival check fails:

if dist_sq > arrive_sq && ambulatory.target.is_none() {
    // Force re-path
    ambulatory.current_path = None;
}

2. Check path validity before arrival distance check

3. Log current_path to see if paths are being generated

Commands to Run

# Check target vs approach mismatch
grep -E "approach=.*target=" output15.log | head -20

# Check if paths are being generated
grep -E "current_path|path_index" output15.log | head -20

# Check distances over time
grep -E "arrival check.*dist_sq" output15.log | grep "8891v0" | head -10

Next Steps

1. Investigate why dist_sq doesn't decrease - dorfs aren't moving toward cargo
2. Check if current_path is populated - paths may not be calculated
3. Verify pathfinding generates valid paths during falling/in-air
4. Check why some dorfs get correct targets and others don't