Pull-everything survey: 76 row-level action exporters cataloged, ~14 consumed #7

Closed
opened 2026-06-12 14:06:42 +00:00 by coilysiren · 3 comments
Owner

While answering "is trades data only a counter?" the agent enumerated /datasets/flatlist: 205 datasets, 76 of them row-level action exporters (IsAction: true, served as CSV via /api/v1/exporter/actions?actionName=...). The MCP tools currently consume only ~14 of them as aggregate /datasets/get series plus the 4 crafting actions.

Standing direction (now an AGENTS.md rule): pull everything remotely interesting - code is cheap, the server is colocated, CPU is fine. This issue is the umbrella tracker; each dataset that gets pulled and turns out messy gets its own cleanup issue.

High-interest candidates from the catalog, roughly grouped:

  • Commerce - CurrencyTrade (#6), BarterTrade, TransferMoney, PayWages, PayTax, PayRentOrMoveInFee, ReceiveGovernmentFunds, MintCurrency, CreateCurrency, PropertyTransfer.
  • Civics - Vote, DidntVote, StartElection, JoinOrLeaveElection, WonElection, LostElection, DemographicChange, ResidencyChanged, BecomeCitizen, LeaveCitizenship, SettlementFounded, PlaceNewSettlementFoundation, StartHomestead.
  • Social - ChatSent (a full chat log!), ReputationTransfer, Play, FirstLogin.
  • Progression - GainProfession, GainSpecialty, LoseSpecialty, SpecialtyLevelUp, CharacterLevelUp, CompleteClass, EnrollAction - overlaps the jobs mod's view, could enrich /jobs with history.
  • Work - CreateWorkOrder, LaborWorkOrderAction, PostedWorkParty, JoinedWorkParty, WorkedForWorkParty, CompletedWorkParty, PostedContract, JoinedContract, CompletedContract, FailedContract.
  • World - PolluteAir, ObjectExplosion, ConstructOrDeconstruct, PlaceOrPickUpObject, MoveWorldObject, PlantSeeds, PlowField, FertilizeAction, TampRoad, DropOrPickupGarbage.

Shared cleanup dependencies: id-to-name join (#5), undecoded enums, per-action column-shape quirks (misalignment, #5), Time-to-day conversion.

Full 76-name catalog captured 2026-06-12 from the live cycle-13 server.

While answering "is trades data only a counter?" the agent enumerated `/datasets/flatlist`: **205 datasets, 76 of them row-level action exporters** (`IsAction: true`, served as CSV via `/api/v1/exporter/actions?actionName=...`). The MCP tools currently consume only ~14 of them as aggregate `/datasets/get` series plus the 4 crafting actions. Standing direction (now an AGENTS.md rule): pull everything remotely interesting - code is cheap, the server is colocated, CPU is fine. This issue is the umbrella tracker; each dataset that gets pulled and turns out messy gets its own cleanup issue. High-interest candidates from the catalog, roughly grouped: * **Commerce** - CurrencyTrade (#6), BarterTrade, TransferMoney, PayWages, PayTax, PayRentOrMoveInFee, ReceiveGovernmentFunds, MintCurrency, CreateCurrency, PropertyTransfer. * **Civics** - Vote, DidntVote, StartElection, JoinOrLeaveElection, WonElection, LostElection, DemographicChange, ResidencyChanged, BecomeCitizen, LeaveCitizenship, SettlementFounded, PlaceNewSettlementFoundation, StartHomestead. * **Social** - ChatSent (a full chat log!), ReputationTransfer, Play, FirstLogin. * **Progression** - GainProfession, GainSpecialty, LoseSpecialty, SpecialtyLevelUp, CharacterLevelUp, CompleteClass, EnrollAction - overlaps the jobs mod's view, could enrich /jobs with history. * **Work** - CreateWorkOrder, LaborWorkOrderAction, PostedWorkParty, JoinedWorkParty, WorkedForWorkParty, CompletedWorkParty, PostedContract, JoinedContract, CompletedContract, FailedContract. * **World** - PolluteAir, ObjectExplosion, ConstructOrDeconstruct, PlaceOrPickUpObject, MoveWorldObject, PlantSeeds, PlowField, FertilizeAction, TampRoad, DropOrPickupGarbage. Shared cleanup dependencies: id-to-name join (#5), undecoded enums, per-action column-shape quirks (misalignment, #5), Time-to-day conversion. Full 76-name catalog captured 2026-06-12 from the live cycle-13 server.
Author
Owner

Full populated-dataset survey committed: docs/dataset-survey-cycle13.md. 192 of 205 datasets are live: 68 action exporters (row-level CSVs) + 124 daily series. Headline volumes: PayTax 6,909 rows, TransferMoney 5,979, ReceiveGovernmentFunds 5,436, ClaimOrUnclaimProperty 4,825, plus 94 species population series. Kai reviews the list and picks pulls; Eco source access at ~/projects/StrangeLoopGames/Eco is now available for probing beyond the exporter surface.

Full populated-dataset survey committed: [docs/dataset-survey-cycle13.md](https://forgejo.coilysiren.me/coilyco-gaming/eco-app/src/branch/main/docs/dataset-survey-cycle13.md). 192 of 205 datasets are live: 68 action exporters (row-level CSVs) + 124 daily series. Headline volumes: PayTax 6,909 rows, TransferMoney 5,979, ReceiveGovernmentFunds 5,436, ClaimOrUnclaimProperty 4,825, plus 94 species population series. Kai reviews the list and picks pulls; Eco source access at ~/projects/StrangeLoopGames/Eco is now available for probing beyond the exporter surface.
Owner

🔎 ward agent advisor

ward agent advisor --driver claude ran a one-shot standard research pass on this question:

76 row-level action exporters are cataloged and ~14 consumed. Given the open feature tickets (#12 economic health dashboard, #17 crafting activity atlas, #20 biodiversity drift, #6 trades ledger), which unconsumed exporters unlock which tickets, and what is the highest-leverage consumption order? Post a ranked mapping so each future feature run starts with its exporter list already decided.


Grounded in the live catalog (docs/datasets/, cycle-13) and the actual consumers in src/eco_mcp_app/ (server.py ECONOMY_DATASETS, crafting.py CRAFT_ACTION_TYPES). Here is the ranked mapping.


What "consumed" actually means today

Two distinct consumption modes are already in the tree, and the distinction drives the whole plan:

  • Row-level CSV (/api/v1/exporter/actions) - only 4 actions, all in crafting.py: ItemCraftedAction, HarvestOrHunt, ChopTree, DigOrMine.
  • Aggregate daily counter (/datasets/get) - the 14 in ECONOMY_DATASETS: the loan quartet, PayWages, PayRentOrMoveInFee, PostedContract, CompletedContract, FailedContract, PropertyTransfer, ReputationTransfer, TransferMoney, PayTax, ReceiveGovernmentFunds.

Key consequence: those 14 are consumed as counters only. Their row-level exporters (who paid whom, how much, when) are still unconsumed. For #12 the highest-value move is not "pull new actions" but "pull the row-level form of actions we already count." That reframes the leverage math below.

Per-ticket exporter lists (the decided lists)

#6 trades ledger - the named, narrowest, highest-signal target.

  • CurrencyTrade (the ledger's spine; BoughtOrSold enum 32/33 is the #6 decode blocker)
  • BarterTrade (non-currency exchange; completes "all trades," not just money trades)
  • Reuse counter: TradesInLast7Days series (latest ~1.8M currency) as the sanity cross-check that row sums reconcile.

#12 economic health dashboard - upgrade counters to distributions, then add the money-supply side.

  • Row-level of already-counted flows: PayTax (6,909 rows), TransferMoney (5,979), ReceiveGovernmentFunds (5,436), PayWages, PayRentOrMoveInFee - these turn today's single-number KPIs into per-citizen distributions (tax burden, wage dispersion, a real Gini).
  • New money-supply actions: MintCurrency, CreateCurrency (inflation / new-currency events the counter set has no view of).
  • CurrencyTrade / BarterTrade shared with #6 (velocity + market depth).
  • ClaimOrUnclaimProperty (4,825 rows) + PropertyTransfer row-level for the wealth-in-land dimension.

#17 crafting activity atlas - the 4 core actions ship; enrich toward the full production pipeline.

  • Work orchestration: CreateWorkOrder, LaborWorkOrderAction, PostedWorkParty, JoinedWorkParty, WorkedForWorkParty, CompletedWorkParty.
  • Contracts row-level: PostedContract, CompletedContract, FailedContract (currently counters only in #12's set - shared).
  • Skill provenance (ties who-can-craft to what-got-crafted): GainProfession, GainSpecialty, SpecialtyLevelUp, CompleteClass, EnrollAction - also the natural /jobs history enrichment.

#20 biodiversity drift - correlate the 94 flora/fauna population series (already live via /datasets/get) against the player actions that move them.

  • Extraction: HarvestOrHunt (2,471) and ChopTree (1,076) - already pulled by crafting, re-frame the same rows biodiversity-first.
  • Regeneration / land use: PlantSeeds (690), PlowField (640), FertilizeAction (5), CreateTreeDebris (4,326).
  • Pollution pressure: PolluteAir (2,109) row-level, cross-referenced with the climate CO2 series.

Highest-leverage consumption order

Ranked by (tickets unlocked) x (signal density) / (cleanup cost), and sequenced so shared cleanup work is paid once and reused.

1. CurrencyTrade + BarterTrade - do first.
Unlocks #6 outright and is a core input to #12 (double-duty). It is the ticket with a name attached, the data is high-volume and high-signal, and it forces the two cleanup dependencies (#6 enum decode, #5 id-to-name join) that everything downstream also needs. Pay that tax here where a shipping feature justifies it.

2. Row-level upgrade of the counted-14 for #12 - PayTax, TransferMoney, ReceiveGovernmentFunds, PayWages, PayRentOrMoveInFee, then MintCurrency / CreateCurrency.
Second because it reuses the #5 join stood up in step 1 and turns the existing economy dashboard from counters into distributions - the biggest quality jump per row pulled, on a surface that already exists. No new enum work (money amounts, not coded fields).

3. Biodiversity action set for #20 - PlantSeeds, PlowField, FertilizeAction, CreateTreeDebris, PolluteAir, plus re-framed HarvestOrHunt / ChopTree.
Third because it is the cheapest cleanup profile (mostly Time + quantity, no enums, less blocked by #5 since the payoff is action-vs-population correlation, not per-citizen identity) and the 94 population series are already consumable - so this is "join new action rows to data we already have," a fast win once the two economic tickets have absorbed the shared plumbing.

4. Work / production + progression for #17 - CreateWorkOrder, LaborWorkOrderAction, the work-party family, contracts row-level, then the skill/progression actions.
Last of the four because it enriches an already-shipping feature (lowest marginal user value), carries the heaviest column-shape risk (work orders and labor have the most per-action schema quirks flagged in #5), and the progression subset overlaps /jobs - worth splitting into its own run once the atlas core is stable.

Shared cleanup prerequisites (do once, in step 1)

  • #5 id-to-name join - blocks any per-citizen view (#6 ledger names, #12 distributions, #17 leaderboards). Stand it up in step 1; steps 2 and 4 inherit it. #20 can proceed without it.
  • #6 enum decode - CurrencyTrade.BoughtOrSold 32/33 and any sibling coded fields; step 1 territory.
  • Time-to-day conversion - universal (seconds-since-cycle-start), trivial, factor into the shared exporter helper so every later pull gets it free.
  • Column misalignment / undeclared tool column (#5) - the defensive header-keyed parse belongs in the shared CSV reader, not per-feature; step 1 is where it earns its place.

Net: steps 1-2 front-load essentially all the reusable cleanup, so #20 and #17 runs start with the plumbing already paid and just wire their exporter lists.

Out of scope note: ChatSent (full chat log), Vote/StartElection/civics, and Play/FirstLogin social actions map to none of these four tickets. They are worth their own feature tickets before consumption - do not fold them into these runs.


Researched and posted automatically by ward agent advisor --driver claude (ward#179). This is one-shot research, not a carried change - verify before acting on it.

— Claude (she/her), via ward agent

### 🔎 ward agent advisor `ward agent advisor --driver claude` ran a one-shot **standard** research pass on this question: > 76 row-level action exporters are cataloged and ~14 consumed. Given the open feature tickets (#12 economic health dashboard, #17 crafting activity atlas, #20 biodiversity drift, #6 trades ledger), which unconsumed exporters unlock which tickets, and what is the highest-leverage consumption order? Post a ranked mapping so each future feature run starts with its exporter list already decided. --- Grounded in the live catalog (`docs/datasets/`, cycle-13) and the actual consumers in `src/eco_mcp_app/` (`server.py` `ECONOMY_DATASETS`, `crafting.py` `CRAFT_ACTION_TYPES`). Here is the ranked mapping. --- ## What "consumed" actually means today Two distinct consumption modes are already in the tree, and the distinction drives the whole plan: * **Row-level CSV** (`/api/v1/exporter/actions`) - only **4** actions, all in `crafting.py`: `ItemCraftedAction`, `HarvestOrHunt`, `ChopTree`, `DigOrMine`. * **Aggregate daily counter** (`/datasets/get`) - the **14** in `ECONOMY_DATASETS`: the loan quartet, `PayWages`, `PayRentOrMoveInFee`, `PostedContract`, `CompletedContract`, `FailedContract`, `PropertyTransfer`, `ReputationTransfer`, `TransferMoney`, `PayTax`, `ReceiveGovernmentFunds`. Key consequence: those 14 are consumed **as counters only**. Their row-level exporters (who paid whom, how much, when) are still unconsumed. For #12 the highest-value move is not "pull new actions" but "pull the row-level form of actions we already count." That reframes the leverage math below. ## Per-ticket exporter lists (the decided lists) **#6 trades ledger** - the named, narrowest, highest-signal target. * `CurrencyTrade` (the ledger's spine; `BoughtOrSold` enum 32/33 is the #6 decode blocker) * `BarterTrade` (non-currency exchange; completes "all trades," not just money trades) * Reuse counter: `TradesInLast7Days` series (latest ~1.8M currency) as the sanity cross-check that row sums reconcile. **#12 economic health dashboard** - upgrade counters to distributions, then add the money-supply side. * Row-level of already-counted flows: `PayTax` (6,909 rows), `TransferMoney` (5,979), `ReceiveGovernmentFunds` (5,436), `PayWages`, `PayRentOrMoveInFee` - these turn today's single-number KPIs into per-citizen distributions (tax burden, wage dispersion, a real Gini). * New money-supply actions: `MintCurrency`, `CreateCurrency` (inflation / new-currency events the counter set has no view of). * `CurrencyTrade` / `BarterTrade` shared with #6 (velocity + market depth). * `ClaimOrUnclaimProperty` (4,825 rows) + `PropertyTransfer` row-level for the wealth-in-land dimension. **#17 crafting activity atlas** - the 4 core actions ship; enrich toward the full production pipeline. * Work orchestration: `CreateWorkOrder`, `LaborWorkOrderAction`, `PostedWorkParty`, `JoinedWorkParty`, `WorkedForWorkParty`, `CompletedWorkParty`. * Contracts row-level: `PostedContract`, `CompletedContract`, `FailedContract` (currently counters only in #12's set - shared). * Skill provenance (ties who-can-craft to what-got-crafted): `GainProfession`, `GainSpecialty`, `SpecialtyLevelUp`, `CompleteClass`, `EnrollAction` - also the natural `/jobs` history enrichment. **#20 biodiversity drift** - correlate the 94 flora/fauna population series (already live via `/datasets/get`) against the player actions that move them. * Extraction: `HarvestOrHunt` (2,471) and `ChopTree` (1,076) - already pulled by crafting, re-frame the same rows biodiversity-first. * Regeneration / land use: `PlantSeeds` (690), `PlowField` (640), `FertilizeAction` (5), `CreateTreeDebris` (4,326). * Pollution pressure: `PolluteAir` (2,109) row-level, cross-referenced with the climate CO2 series. ## Highest-leverage consumption order Ranked by (tickets unlocked) x (signal density) / (cleanup cost), and sequenced so shared cleanup work is paid once and reused. **1. `CurrencyTrade` + `BarterTrade`** - do first. Unlocks #6 outright and is a core input to #12 (double-duty). It is the ticket with a name attached, the data is high-volume and high-signal, and it forces the two cleanup dependencies (#6 enum decode, #5 id-to-name join) that everything downstream also needs. Pay that tax here where a shipping feature justifies it. **2. Row-level upgrade of the counted-14 for #12** - `PayTax`, `TransferMoney`, `ReceiveGovernmentFunds`, `PayWages`, `PayRentOrMoveInFee`, then `MintCurrency` / `CreateCurrency`. Second because it reuses the #5 join stood up in step 1 and turns the existing economy dashboard from counters into distributions - the biggest quality jump per row pulled, on a surface that already exists. No new enum work (money amounts, not coded fields). **3. Biodiversity action set for #20** - `PlantSeeds`, `PlowField`, `FertilizeAction`, `CreateTreeDebris`, `PolluteAir`, plus re-framed `HarvestOrHunt` / `ChopTree`. Third because it is the cheapest cleanup profile (mostly `Time` + quantity, no enums, less blocked by #5 since the payoff is action-vs-population correlation, not per-citizen identity) and the 94 population series are already consumable - so this is "join new action rows to data we already have," a fast win once the two economic tickets have absorbed the shared plumbing. **4. Work / production + progression for #17** - `CreateWorkOrder`, `LaborWorkOrderAction`, the work-party family, contracts row-level, then the skill/progression actions. Last of the four because it enriches an **already-shipping** feature (lowest marginal user value), carries the heaviest column-shape risk (work orders and labor have the most per-action schema quirks flagged in #5), and the progression subset overlaps `/jobs` - worth splitting into its own run once the atlas core is stable. ## Shared cleanup prerequisites (do once, in step 1) * **#5 id-to-name join** - blocks any per-citizen view (#6 ledger names, #12 distributions, #17 leaderboards). Stand it up in step 1; steps 2 and 4 inherit it. #20 can proceed without it. * **#6 enum decode** - `CurrencyTrade.BoughtOrSold` 32/33 and any sibling coded fields; step 1 territory. * **`Time`-to-day conversion** - universal (seconds-since-cycle-start), trivial, factor into the shared exporter helper so every later pull gets it free. * **Column misalignment / undeclared tool column (#5)** - the defensive header-keyed parse belongs in the shared CSV reader, not per-feature; step 1 is where it earns its place. Net: steps 1-2 front-load essentially all the reusable cleanup, so #20 and #17 runs start with the plumbing already paid and just wire their exporter lists. **Out of scope note:** `ChatSent` (full chat log), `Vote`/`StartElection`/civics, and `Play`/`FirstLogin` social actions map to **none** of these four tickets. They are worth their own feature tickets before consumption - do not fold them into these runs. --- Researched and posted automatically by `ward agent advisor --driver claude` (ward#179). This is one-shot research, not a carried change - verify before acting on it. <!-- ward-agent-reply --> <!-- ward-agent-signature --> — Claude (she/her), via `ward agent`
Owner

The survey and its prioritization are complete, so this no longer needs a standing Kai selection checkpoint.

  • The full populated-dataset inventory is committed under docs/datasets/.
  • The ranked follow-up comment chose commerce first, then economy distributions, biodiversity, and work/progression.
  • The resulting feature and shared-cleanup issues are closed: #5, #6, #12, #17, #20, #49 through #55, #61, and #64.
  • The repository now has consumers for each of those selected slices. AGENTS.md still requires any newly interesting messy dataset to receive its own cleanup issue.

Keeping #7 open would turn a completed survey into an indefinite product backlog. New dataset pulls should be bounded source issues. Closing the completed survey.

The survey and its prioritization are complete, so this no longer needs a standing Kai selection checkpoint. * The full populated-dataset inventory is committed under `docs/datasets/`. * The ranked follow-up comment chose commerce first, then economy distributions, biodiversity, and work/progression. * The resulting feature and shared-cleanup issues are closed: #5, #6, #12, #17, #20, #49 through #55, #61, and #64. * The repository now has consumers for each of those selected slices. AGENTS.md still requires any newly interesting messy dataset to receive its own cleanup issue. Keeping #7 open would turn a completed survey into an indefinite product backlog. New dataset pulls should be bounded source issues. Closing the completed survey.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-gaming/eco-app#7
No description provided.