Crafting atlas citizen attribution: exporter rows misalign and carry numeric ids #5

Closed
opened 2026-06-12 13:44:19 +00:00 by coilysiren · 3 comments
Owner

The crafting atlas's by-citizen dimension is unusable as shipped, found while building the SPA /crafting page (issue #2 wave 0).

Two stacked problems in the action exporter CSVs (/api/v1/exporter/actions?actionName=...):

  1. Row misalignment - some rows carry an extra tool column the header doesn't declare (e.g. HarvestOrHunt rows gain "HandsItem" before Position), shifting every later field. Header-indexed parsing then reads a position triple like 254,86,313 as the Citizen, and a position as Count (float() fails, count -> 0).
  2. Citizen is a numeric user id even in aligned rows (e.g. 129312), and no name mapping is fetched, so a correctly parsed by-citizen table would still render bare ids.

Live evidence (cycle 13, 2026-06-12): byCitizen had 1569/1569 garbage-shaped keys, topped by "254,86,313" with a 1.28M count against totalEvents of 14,251.

Interim state after the wave-0 commit: by-item and by-station fold only sane keys (numeric/position-shaped keys skipped), by-citizen is disabled with a payload warning pointing here, flows (station -> item) unaffected.

Proper fix needs both: alignment-tolerant row parsing (detect and absorb the optional tool column), and an id -> display-name join from whatever surface the server exposes (specieslist-style user export, or the jobs mod could grow one).

The crafting atlas's by-citizen dimension is unusable as shipped, found while building the SPA /crafting page (issue #2 wave 0). Two stacked problems in the action exporter CSVs (`/api/v1/exporter/actions?actionName=...`): 1. **Row misalignment** - some rows carry an extra tool column the header doesn't declare (e.g. HarvestOrHunt rows gain `"HandsItem"` before Position), shifting every later field. Header-indexed parsing then reads a position triple like `254,86,313` as the Citizen, and a position as Count (float() fails, count -> 0). 2. **Citizen is a numeric user id** even in aligned rows (e.g. `129312`), and no name mapping is fetched, so a correctly parsed by-citizen table would still render bare ids. Live evidence (cycle 13, 2026-06-12): byCitizen had 1569/1569 garbage-shaped keys, topped by `"254,86,313"` with a 1.28M count against totalEvents of 14,251. Interim state after the wave-0 commit: by-item and by-station fold only sane keys (numeric/position-shaped keys skipped), by-citizen is disabled with a payload warning pointing here, flows (station -> item) unaffected. Proper fix needs both: alignment-tolerant row parsing (detect and absorb the optional tool column), and an id -> display-name join from whatever surface the server exposes (specieslist-style user export, or the jobs mod could grow one).
Author
Owner

Name-join lead found while scoping the trades ledger (#6): /api/v1/users (admin-keyed) returns Name / PlayFabId / SteamId for all 114 settlers - but no numeric in-game id, and the exporter CSVs key parties by that numeric id (e.g. 129558). So the join is one link short: either another surface exposes numeric id alongside name, or the jobs mod grows an id field in its /api/v1/skills payload (it iterates UserManager.Users, which carries the id).

Name-join lead found while scoping the trades ledger (#6): `/api/v1/users` (admin-keyed) returns `Name / PlayFabId / SteamId` for all 114 settlers - but no numeric in-game id, and the exporter CSVs key parties by that numeric id (e.g. 129558). So the join is one link short: either another surface exposes numeric id alongside name, or the jobs mod grows an id field in its /api/v1/skills payload (it iterates UserManager.Users, which carries the id).
Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-eco-app-5 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T02:39:40Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

<!-- ward-agent-reservation --> 🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-eco-app-5` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T02:39:40Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. <!-- ward-agent-signature --> — Claude (she/her), via `ward agent`
Author
Owner

WARD-OUTCOME: done - realignment-tolerant row parsing + citizen id→name join via new jobs-mod /api/v1/citizens, by-citizen re-enabled end to end (SPA "Top crafters").

Retrospective: the Python half went smoothly and is well-covered - the realign scorer, the join, and the "Citizen #" fallback all have tests. The one thing that actually fought back was the realignment tie-break: scoring candidate insertion points only against numeric/position columns left p=0 and the true p scoring identically (every typed column sat after the insertion), so a shifted row silently pushed Species into a numeric slot. Adding name-shape validation for the item/species columns (plus a latest-insertion-point tie-break) fixed it cleanly.

Where I'm less sure: the C# side is unverifiable in-container (no dotnet, no Eco SDK, no live server), so CitizensApiController is code-review-only. It's modeled exactly on the existing SkillsApiController and should compile, but the real confidence gap is whether user.Id is the same numeric the action exporter emits for Citizen - I couldn't confirm that against live data. If it isn't, the join returns empty and everything degrades to Citizen #<id> labels (no crash, no garbage - just no names).

Follow-up worth filing: verify against a live cycle that user.Id matches the exporter's Citizen id; if it diverges, the CitizenDto likely needs a different/extra identifier field. Also the realign path now also covers Buyer/Seller/ShopOwner (economy actions) by shape but that's untested until #6-era economy CSVs flow through it.

WARD-OUTCOME: done - realignment-tolerant row parsing + citizen id→name join via new jobs-mod /api/v1/citizens, by-citizen re-enabled end to end (SPA "Top crafters"). Retrospective: the Python half went smoothly and is well-covered - the realign scorer, the join, and the "Citizen #<id>" fallback all have tests. The one thing that actually fought back was the realignment tie-break: scoring candidate insertion points only against numeric/position columns left `p=0` and the true `p` scoring identically (every typed column sat after the insertion), so a shifted row silently pushed Species into a numeric slot. Adding name-shape validation for the item/species columns (plus a latest-insertion-point tie-break) fixed it cleanly. Where I'm less sure: the C# side is unverifiable in-container (no dotnet, no Eco SDK, no live server), so `CitizensApiController` is code-review-only. It's modeled exactly on the existing `SkillsApiController` and should compile, but the real confidence gap is whether `user.Id` is the *same* numeric the action exporter emits for `Citizen` - I couldn't confirm that against live data. If it isn't, the join returns empty and everything degrades to `Citizen #<id>` labels (no crash, no garbage - just no names). Follow-up worth filing: verify against a live cycle that `user.Id` matches the exporter's Citizen id; if it diverges, the `CitizenDto` likely needs a different/extra identifier field. Also the realign path now also covers Buyer/Seller/ShopOwner (economy actions) by shape but that's untested until #6-era economy CSVs flow through it.
Sign in to join this conversation.
No description provided.