Crafting atlas / item pivot misreads ItemCraftedAction Count as items crafted

coilyco-ops commented

2026-07-21 03:45:56 +00:00

Member

Found while investigating "why does Theo have 32 stump latrines" against the #128 snapshot.

Ground truth vs app claim. The eco-crafts Discord feed (DiscordLink per-work-order messages) shows Theo de Thael crafted 1 Stump Latrine at 2026-07-04T00:32:25Z and 1 more at 00:41:46Z - two singles. The app's item pivot (/preview/item.json?item=StumpLatrineItem) reports one craft event, quantity 32.0, and the crafting atlas ranks crafters by these numbers (CptBlkSparrow "40", Theo "32").

The CSV row is aligned, not shifted. This is not the #5 extra-tool-column shift. Theo's raw ItemCraftedAction row reads "910,61,478","WorkbenchItem",131722,"StumpLatrineItem",false,32.0,9160 - every column lands where the header says. Game-time 9160s maps exactly onto the 00:32Z Discord message (cycle start ~22:00Z July 3), so it is the same event.

Two implications:

Count is not "items crafted." Crafting 2 latrines produced Count=32 (16 per latrine - plausibly ingredient units, labor ticks, or calories). Other Theo rows (Campfire 189, Adobe 145, CharredCorn 96) are similarly non-quantity-shaped. Everything that sums this column as craft quantity is wrong: the item pivot's craftQuantity, the crafting atlas top-crafter rankings, and the jobs page "Most valuable to craft" board's demand math where it consumes craft volumes.
Rows are aggregated, not per-event. Theo's 00:41Z second latrine craft has no CSV row of its own - the exporter appears to roll events up (per item+station+citizen window), so "events" counts are also understated.

Fix directions to evaluate: work out the real Count semantics from the Eco source (the survey's checkout path ~/projects/StrangeLoopGames/Eco is not present on this host), relabel the metric in the UI ("crafting effort" rather than "items"), or derive true quantities from a better source. The eco-crafts Discord channel itself is a candidate ground-truth feed - the bot tokens in SSM can read it (note: Discord's Cloudflare requires a proper User-Agent; default python-urllib gets 403 error 1010), and it carries exact per-work-order quantities back to cycle start.

Found while investigating "why does Theo have 32 stump latrines" against the #128 snapshot. **Ground truth vs app claim.** The `eco-crafts` Discord feed (DiscordLink per-work-order messages) shows Theo de Thael crafted 1 Stump Latrine at 2026-07-04T00:32:25Z and 1 more at 00:41:46Z - two singles. The app's item pivot (`/preview/item.json?item=StumpLatrineItem`) reports one craft event, quantity 32.0, and the crafting atlas ranks crafters by these numbers (CptBlkSparrow "40", Theo "32"). **The CSV row is aligned, not shifted.** This is not the #5 extra-tool-column shift. Theo's raw `ItemCraftedAction` row reads `"910,61,478","WorkbenchItem",131722,"StumpLatrineItem",false,32.0,9160` - every column lands where the header says. Game-time 9160s maps exactly onto the 00:32Z Discord message (cycle start ~22:00Z July 3), so it is the same event. **Two implications:** 1. `Count` is not "items crafted." Crafting 2 latrines produced Count=32 (16 per latrine - plausibly ingredient units, labor ticks, or calories). Other Theo rows (Campfire 189, Adobe 145, CharredCorn 96) are similarly non-quantity-shaped. Everything that sums this column as craft quantity is wrong: the item pivot's `craftQuantity`, the crafting atlas top-crafter rankings, and the jobs page "Most valuable to craft" board's demand math where it consumes craft volumes. 2. Rows are aggregated, not per-event. Theo's 00:41Z second latrine craft has no CSV row of its own - the exporter appears to roll events up (per item+station+citizen window), so "events" counts are also understated. **Fix directions to evaluate:** work out the real Count semantics from the Eco source (the survey's checkout path `~/projects/StrangeLoopGames/Eco` is not present on this host), relabel the metric in the UI ("crafting effort" rather than "items"), or derive true quantities from a better source. The `eco-crafts` Discord channel itself is a candidate ground-truth feed - the bot tokens in SSM can read it (note: Discord's Cloudflare requires a proper User-Agent; default python-urllib gets 403 error 1010), and it carries exact per-work-order quantities back to cycle start.

coilyco-ops commented

2026-07-21 03:55:00 +00:00

Author

Member

Scope updates from Kai:

Drop the eco-crafts-as-data-source fix direction. DiscordLink is slated for removal except chat sync, so no app surface should take a dependency on the DiscordLink feed channels. The Discord evidence in this issue stays valid as one-time ground truth, but not as a pipeline.
Backfill is not the fix. Even with the channel history imported, the live exporter reads would keep producing the same wrong numbers - the defect is in interpreting Count as items crafted, so the fix has to land in the reader (correct semantics from the Eco source, or relabel the metric).

Discord-side papercuts from this investigation moved to sirens-discord-ops# - nothing SSM-side needed fixing for this issue (the suspected stale params were a ciphertext misread, since corrected).

Scope updates from Kai: * **Drop the eco-crafts-as-data-source fix direction.** DiscordLink is slated for removal except chat sync, so no app surface should take a dependency on the DiscordLink feed channels. The Discord evidence in this issue stays valid as one-time ground truth, but not as a pipeline. * **Backfill is not the fix.** Even with the channel history imported, the live exporter reads would keep producing the same wrong numbers - the defect is in interpreting `Count` as items crafted, so the fix has to land in the reader (correct semantics from the Eco source, or relabel the metric). Discord-side papercuts from this investigation moved to sirens-discord-ops#<see repo> - nothing SSM-side needed fixing for this issue (the suspected stale params were a ciphertext misread, since corrected).

coilyco-ops commented

2026-07-21 03:55:10 +00:00

Author

Member

The papercut issue landed as sirens-discord-ops#28.

The papercut issue landed as [sirens-discord-ops#28](https://forgejo.coilysiren.me/coilyco-gaming/sirens-discord-ops/issues/28).

coilyco-ops commented

2026-07-21 04:15:02 +00:00

Author

Member

Fixed in 5ed40dd on main. Full semantics, pinned from the Eco source (via the GitHub repo - the local checkout is gone from this host):

ItemCraftedAction fires once per completed crafting iteration (WorkOrder.CompleteIteration), not per item or work order.
It extends AggregatableAction, whose Count defaults to "count of grouped actions". Eco.Stats.StatsAggregator merges records older than KeepDetailsHours per grouping key per hour: Count becomes the merged-event total, Time becomes the max, and every non-key column keeps the FIRST merged record's values.
With essential-fields grouping the only key is Citizen, so a Count>1 row's item/station/location are one arbitrary event's labels. "Theo crafted 32 stump latrines" was 32 iterations of anything in the 00:00-01:00Z hour, 2 of them latrines.
Verified in the cycle-14 snapshot: rows before day 16 are 99% Count>1 (mean 192), the last day is 96% Count==1.

The reader fix: Count>1 craft rows are excluded from by_crafted / by_station / flows and from the item pivot's craft leg, and surfaced as new rollupEvents / rollupIterations fields plus a warning. The citizen leaderboard now weighs crafts by Count, so it stays correct across all history (Citizen is the grouping key). Frontend relabels craft quantities as iterations and shows the rollup note. Against the live snapshot: 879,194 iterations across 4,632 rollups now correctly excluded from item attribution.

Sibling defect in the trades ledger (CurrencyTrade is also aggregatable - 79% of snapshot rows are rollups, so old Buyer/Seller/item labels are representative-only) filed separately.

Fixed in 5ed40dd on main. Full semantics, pinned from the Eco source (via the GitHub repo - the local checkout is gone from this host): * `ItemCraftedAction` fires once per completed crafting **iteration** (`WorkOrder.CompleteIteration`), not per item or work order. * It extends `AggregatableAction`, whose `Count` defaults to "count of grouped actions". `Eco.Stats.StatsAggregator` merges records older than `KeepDetailsHours` per grouping key per **hour**: Count becomes the merged-event total, Time becomes the max, and every non-key column keeps the FIRST merged record's values. * With essential-fields grouping the only key is `Citizen`, so a Count>1 row's item/station/location are one arbitrary event's labels. "Theo crafted 32 stump latrines" was 32 iterations of anything in the 00:00-01:00Z hour, 2 of them latrines. * Verified in the cycle-14 snapshot: rows before day 16 are 99% Count>1 (mean 192), the last day is 96% Count==1. The reader fix: Count>1 craft rows are excluded from `by_crafted` / `by_station` / `flows` and from the item pivot's craft leg, and surfaced as new `rollupEvents` / `rollupIterations` fields plus a warning. The citizen leaderboard now weighs crafts by Count, so it stays correct across all history (Citizen is the grouping key). Frontend relabels craft quantities as iterations and shows the rollup note. Against the live snapshot: 879,194 iterations across 4,632 rollups now correctly excluded from item attribution. Sibling defect in the trades ledger (CurrencyTrade is also aggregatable - 79% of snapshot rows are rollups, so old Buyer/Seller/item labels are representative-only) filed separately.

coilyco-ops referenced this issue

2026-07-21 04:15:18 +00:00

Trades ledger misreads aggregated CurrencyTrade rows (same rollup defect as #131) #132

coilyco-ops closed this issue

2026-07-21 04:15:24 +00:00

coilyco-ops referenced this issue

2026-07-21 04:15:30 +00:00

frontend vitest: PagePassword tests fail locally - localStorage undefined in test env #133

coilyco-ops commented

2026-07-21 04:20:56 +00:00

Author

Member

Refinement in a300601: full exclusion made rollup-only items vanish from /items entirely (Kai caught stump latrines disappearing). Since the aggregator's representative labels come from the FIRST merged event, one iteration of the label item is genuinely proven per rollup row - so every craft row now contributes exactly 1 confirmed iteration to the item/station/flow boards (floors, never merged totals), the remaining count-1 land in rollupIterations, and the pivot clamps rollup rows to quantity 1. The UI marks the numbers as floors ("N+ iterations"). StumpLatrineItem now shows its two confirmed crafts: Theo de Thael 1+ (day 0.11, the real 00:32Z craft) and CptBlkSparrow 1+ (day 1.16).

Refinement in a300601: full exclusion made rollup-only items vanish from `/items` entirely (Kai caught stump latrines disappearing). Since the aggregator's representative labels come from the FIRST merged event, one iteration of the label item is genuinely proven per rollup row - so every craft row now contributes exactly 1 confirmed iteration to the item/station/flow boards (floors, never merged totals), the remaining count-1 land in `rollupIterations`, and the pivot clamps rollup rows to quantity 1. The UI marks the numbers as floors ("N+ iterations"). StumpLatrineItem now shows its two confirmed crafts: Theo de Thael 1+ (day 0.11, the real 00:32Z craft) and CptBlkSparrow 1+ (day 1.16).

Rows
Columns

Crafting atlas / item pivot misreads ItemCraftedAction Count as items crafted #131