Recipe data ingest (keystone): seed /preview/recipes.json from Eco Gnome's eco_gnome_data.json - follow-up A from #98 [UNBLOCKED by #105] #100

Closed
opened 2026-07-07 06:54:04 +00:00 by coilysiren · 3 comments
Owner

Keystone of the recipe/pricing roadmap on #98: hydrate recipe / skill / item definitions into eco-app so every recipe-shaped surface has a bill-of-materials to read. This is follow-up A, refined by the director correction on #98.

What and why

eco-app today has the market/demand half of the economy (market.py, trades.py, logistics.py, stores.py) but no recipe definitions anywhere - no ingredients-in / product-out, no station, no required skill, no labor/calorie cost. crafting.py is a production-event leaderboard, not a bill-of-materials. That gap blocks the recipes page (B), the cost engine (C), the value-per-profession grouping (D), and the pricing page (E).

Eco Gnome is itself a recipe exporter, so ingest its output rather than build a bespoke Eco RecipeManager dumper. eco-gnome-mod (https://github.com/Eco-Gnome/eco-gnome-mod) is a server-side DataExporter that dumps a server's modded recipes, skills, and items, and the eco-gnome image bundles ecocraft/eco_gnome_data.json (vanilla recipe/item data). See docs/calculator.md and #40.

Scope

  • A Python consumer (recipes.py) mirroring the crafting.py / trades.py streaming + cache pattern, plus a /preview/recipes.json surface, producing the DTO spec'd on #98: Recipe { product, ingredients[], byproducts[], station, skill{name,level}, laborCost, craftMinutes, tableTierRequired, variants } and a RecipeIndex.
  • Source the data from Eco Gnome's export (the mod's dump and/or the bundled eco_gnome_data.json), not a new mod, unless the probe (see below) finds that path unavailable.

Gate - do NOT dispatch this until the probe answers

The actual data availability is unconfirmed: the current Eco Gnome integration is only a homepage link-out (Home.tsx), and the self-host + DataExporter path (#40) is blocked on kai-server mod-ops (ward#585 / eco-ops#30). The companion advisor probe issue determines exactly what recipe data is obtainable today and by what path. Wait for that finding, then this ticket's real work is landing the confirmed ingest path. Unblocks B / C / D / E.

Keystone of the recipe/pricing roadmap on #98: hydrate recipe / skill / item definitions into eco-app so every recipe-shaped surface has a bill-of-materials to read. This is follow-up **A**, refined by the director correction on #98. ## What and why eco-app today has the market/demand half of the economy (`market.py`, `trades.py`, `logistics.py`, `stores.py`) but **no recipe definitions** anywhere - no ingredients-in / product-out, no station, no required skill, no labor/calorie cost. `crafting.py` is a production-event leaderboard, not a bill-of-materials. That gap blocks the recipes page (B), the cost engine (C), the value-per-profession grouping (D), and the pricing page (E). **Eco Gnome is itself a recipe exporter, so ingest its output rather than build a bespoke Eco `RecipeManager` dumper.** `eco-gnome-mod` (https://github.com/Eco-Gnome/eco-gnome-mod) is a server-side DataExporter that dumps a server's modded recipes, skills, and items, and the eco-gnome image bundles `ecocraft/eco_gnome_data.json` (vanilla recipe/item data). See `docs/calculator.md` and #40. ## Scope - A Python consumer (`recipes.py`) mirroring the `crafting.py` / `trades.py` streaming + cache pattern, plus a `/preview/recipes.json` surface, producing the DTO spec'd on #98: `Recipe { product, ingredients[], byproducts[], station, skill{name,level}, laborCost, craftMinutes, tableTierRequired, variants }` and a `RecipeIndex`. - Source the data from Eco Gnome's export (the mod's dump and/or the bundled `eco_gnome_data.json`), not a new mod, unless the probe (see below) finds that path unavailable. ## Gate - do NOT dispatch this until the probe answers The **actual data availability is unconfirmed**: the current Eco Gnome integration is only a homepage link-out (`Home.tsx`), and the self-host + DataExporter path (#40) is blocked on kai-server mod-ops (ward#585 / eco-ops#30). The companion **advisor probe issue** determines exactly what recipe data is obtainable today and by what path. Wait for that finding, then this ticket's real work is landing the confirmed ingest path. Unblocks B / C / D / E.
coilysiren changed title from Recipe data ingest (keystone): hydrate recipe/skill/item definitions from Eco Gnome's export into /preview/recipes.json - follow-up A from #98 [gated: awaits the eco-gnome data probe] to Recipe data ingest (keystone): seed /preview/recipes.json from Eco Gnome's eco_gnome_data.json - follow-up A from #98 [UNBLOCKED by #105] 2026-07-07 06:59:40 +00:00
Author
Owner

Unblocked by the #105 probe - this is startable now. The probe's decisive finding: the bundled ecocraft/eco_gnome_data.json (eco-gnome-website, MIT, ~6 MB, fetchable raw from GitHub) is a complete recipe graph (1,453 recipes / 1,526 items / 43 skills / 142 tags) in the identical schema the DataExporter mod emits. So A can build against it now, and the eventual modded export is a drop-in replacement of the same file - the parser needs no rework.

Ingest contract (target this schema)

Top level: { Version, Skills[], Items[], Tags[], Recipes[] }.

  • Recipe: Name, FamilyName (variant family, IsDefault flags the canonical), Ingredients[] / Products[] each { ItemOrTag, Quantity:{BaseValue, Modifiers[]} }, CraftingTable (station), RequiredSkill + RequiredSkillLevel, Labor (BaseValue = calories - labor and calories are the same field in Eco), CraftMinutes (time), IsBlueprint, IsDefault. Modifiers carry skill/module/talent effects for accurate pricing.
  • Skill: Name, MaxLevel, LaborReducePercent[], Talents[]. Item: lean (Name, FuelCalories). Tag: Name, AssociatedItems[] (resolves ItemOrTag inputs to concrete items).
  • One gap: no first-class table-tier field - infer from CraftingTable + RequiredSkillLevel.

Decisions (named defaults - shout if you want otherwise)

  • Source: vendor a pinned copy of eco_gnome_data.json in-repo (reproducible builds, no build-time network, CSP-clean), rather than fetch-at-build. Keep the Eco-Gnome MIT attribution already recorded in the deploy ATTRIBUTION.md.
  • Vanilla-only for now: the seed omits Sirens' modded recipes/items until the modded export lands (ward#585 / eco-ops#30). A ships the pipeline against vanilla, Phase 2 swaps the file. Label the surface honestly as vanilla-derived until then.
  • Not the self-hosted DB: eco-gnome.coilysiren.me is live but exposes prices-only over REST, its Postgres is internal, and it holds no Sirens data. Not a recipe source.

Doc-drift to fold into this PR (not a separate issue)

docs/calculator.md / the deploy README still say the eco-gnome self-host is 'remaining to go live'. The probe confirmed eco-gnome.coilysiren.me is already up and serving (Kestrel / .NET 9). Correct that line in this PR.

**Unblocked by the #105 probe - this is startable now.** The probe's decisive finding: the bundled `ecocraft/eco_gnome_data.json` (eco-gnome-website, MIT, ~6 MB, fetchable raw from GitHub) is a complete recipe graph (1,453 recipes / 1,526 items / 43 skills / 142 tags) in the **identical schema the DataExporter mod emits**. So A can build against it now, and the eventual modded export is a drop-in replacement of the same file - the parser needs no rework. ## Ingest contract (target this schema) Top level: `{ Version, Skills[], Items[], Tags[], Recipes[] }`. - **Recipe**: `Name`, `FamilyName` (variant family, `IsDefault` flags the canonical), `Ingredients[]` / `Products[]` each `{ ItemOrTag, Quantity:{BaseValue, Modifiers[]} }`, `CraftingTable` (station), `RequiredSkill` + `RequiredSkillLevel`, `Labor` (BaseValue = calories - labor and calories are the same field in Eco), `CraftMinutes` (time), `IsBlueprint`, `IsDefault`. Modifiers carry skill/module/talent effects for accurate pricing. - **Skill**: `Name`, `MaxLevel`, `LaborReducePercent[]`, `Talents[]`. **Item**: lean (`Name`, `FuelCalories`). **Tag**: `Name`, `AssociatedItems[]` (resolves `ItemOrTag` inputs to concrete items). - One gap: **no first-class table-tier field** - infer from `CraftingTable` + `RequiredSkillLevel`. ## Decisions (named defaults - shout if you want otherwise) - **Source: vendor a pinned copy of `eco_gnome_data.json` in-repo** (reproducible builds, no build-time network, CSP-clean), rather than fetch-at-build. Keep the Eco-Gnome MIT attribution already recorded in the deploy `ATTRIBUTION.md`. - **Vanilla-only for now**: the seed omits Sirens' modded recipes/items until the modded export lands (ward#585 / eco-ops#30). A ships the pipeline against vanilla, Phase 2 swaps the file. Label the surface honestly as vanilla-derived until then. - **Not** the self-hosted DB: `eco-gnome.coilysiren.me` is live but exposes prices-only over REST, its Postgres is internal, and it holds no Sirens data. Not a recipe source. ## Doc-drift to fold into this PR (not a separate issue) `docs/calculator.md` / the deploy README still say the eco-gnome self-host is 'remaining to go live'. The probe confirmed `eco-gnome.coilysiren.me` is **already up and serving** (Kestrel / .NET 9). Correct that line in this PR.
Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-eco-app-100 on host KAI-DESKTOP-TOWER is carrying this issue (reserved 2026-07-07T06:59:57Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)
  • Resolved: coilyco-gaming/eco-app#100 · branch issue-100 · driver claude · workflow direct-main
  • Run: engineer-claude-eco-app-100 · ward v0.419.0 · dispatched 2026-07-07T06:59:57Z
  • Comment thread: 1 included in the pre-flight read, 0 stripped (ward's own automated comments).

Issue body as seeded:

Keystone of the recipe/pricing roadmap on #98: hydrate recipe / skill / item definitions into eco-app so every recipe-shaped surface has a bill-of-materials to read. This is follow-up **A**, refined by the director correction on #98.

## What and why

eco-app today has the market/demand half of the economy (`market.py`, `trades.py`, `logistics.py`, `stores.py`) but **no recipe definitions** anywhere - no ingredients-in / product-out, no station, no required skill, no labor/calorie cost. `crafting.py` is a production-event leaderboard, not a bill-of-materials. That gap blocks the recipes page (B), the cost engine (C), the value-per-profession grouping (D), and the pricing page (E).

**Eco Gnome is itself a recipe exporter, so ingest its output rather than build a bespoke Eco `RecipeManager` dumper.** `eco-gnome-mod` (https://github.com/Eco-Gnome/eco-gnome-mod) is a server-side DataExporter that dumps a server's modded recipes, skills, and items, and the eco-gnome image bundles `ecocraft/eco_gnome_data.json` (vanilla recipe/item data). See `docs/calculator.md` and #40.

## Scope

- A Python consumer (`recipes.py`) mirroring the `crafting.py` / `trades.py` streaming + cache pattern, plus a `/preview/recipes.json` surface, producing the DTO spec'd on #98: `Recipe { product, ingredients[], byproducts[], station, skill{name,level}, laborCost, craftMinutes, tableTierRequired, variants }` and a `RecipeIndex`.
- Source the data from Eco Gnome's export (the mod's dump and/or the bundled `eco_gnome_data.json`), not a new mod, unless the probe (see below) finds that path unavailable.

## Gate - do NOT dispatch this until the probe answers

The **actual data availability is unconfirmed**: the current Eco Gnome integration is only a homepage link-out (`Home.tsx`), and the self-host + DataExporter path (#40) is blocked on kai-server mod-ops (ward#585 / eco-ops#30). The companion **advisor probe issue** determines exactly what recipe data is obtainable today and by what path. Wait 

… (truncated to 2000 chars; full body is on this issue)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.419.0).

— Claude (she/her), via ward agent

<!-- ward-agent-reservation --> 🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-eco-app-100` on host `KAI-DESKTOP-TOWER` is carrying this issue (reserved 2026-07-07T06:59:57Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh** — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-gaming/eco-app#100` · branch `issue-100` · driver `claude` · workflow `direct-main` - **Run:** `engineer-claude-eco-app-100` · ward `v0.419.0` · dispatched `2026-07-07T06:59:57Z` - **Comment thread:** 1 included in the pre-flight read, 0 stripped (ward's own automated comments). - included: @coilysiren (2026-07-07T06:59:40Z) **Issue body as seeded:** ``` Keystone of the recipe/pricing roadmap on #98: hydrate recipe / skill / item definitions into eco-app so every recipe-shaped surface has a bill-of-materials to read. This is follow-up **A**, refined by the director correction on #98. ## What and why eco-app today has the market/demand half of the economy (`market.py`, `trades.py`, `logistics.py`, `stores.py`) but **no recipe definitions** anywhere - no ingredients-in / product-out, no station, no required skill, no labor/calorie cost. `crafting.py` is a production-event leaderboard, not a bill-of-materials. That gap blocks the recipes page (B), the cost engine (C), the value-per-profession grouping (D), and the pricing page (E). **Eco Gnome is itself a recipe exporter, so ingest its output rather than build a bespoke Eco `RecipeManager` dumper.** `eco-gnome-mod` (https://github.com/Eco-Gnome/eco-gnome-mod) is a server-side DataExporter that dumps a server's modded recipes, skills, and items, and the eco-gnome image bundles `ecocraft/eco_gnome_data.json` (vanilla recipe/item data). See `docs/calculator.md` and #40. ## Scope - A Python consumer (`recipes.py`) mirroring the `crafting.py` / `trades.py` streaming + cache pattern, plus a `/preview/recipes.json` surface, producing the DTO spec'd on #98: `Recipe { product, ingredients[], byproducts[], station, skill{name,level}, laborCost, craftMinutes, tableTierRequired, variants }` and a `RecipeIndex`. - Source the data from Eco Gnome's export (the mod's dump and/or the bundled `eco_gnome_data.json`), not a new mod, unless the probe (see below) finds that path unavailable. ## Gate - do NOT dispatch this until the probe answers The **actual data availability is unconfirmed**: the current Eco Gnome integration is only a homepage link-out (`Home.tsx`), and the self-host + DataExporter path (#40) is blocked on kai-server mod-ops (ward#585 / eco-ops#30). The companion **advisor probe issue** determines exactly what recipe data is obtainable today and by what path. Wait ``` … (truncated to 2000 chars; full body is on this issue) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.419.0). </details> <!-- ward-agent-signature --> — Claude (she/her), via `ward agent`
Author
Owner

WARD-OUTCOME: done - /preview/recipes.json now serves the vendored Eco Gnome recipe graph (1,453 recipes) as the #98 Recipe DTO, landed on main.

This one went smoothly, mostly because the #105 probe did the hard part: it named the exact file, schema, and the vendor-don't-fetch decision, so there was no real fork left to agonize over. The Eco Gnome export is cleaner than the CSV exporters the other consumers wrestle with - every quantity is a tidy {BaseValue, Modifiers[]}, no undeclared-column realignment, no id->name joins. So recipes.py ended up shorter than crafting.py despite mirroring its DTO+to_dict shape, and the streaming/SQLite-cache machinery would have been dead weight against an immutable local file, so I swapped it for a one-line in-process memo and said why in the docstring.

What fought back, mildly: the source file is ~6 MB of 23-locale name blocks. Committing that felt wrong, so I trimmed LocalizedName to en-US only (~2.2 MB) while leaving every structural key intact - schema-identical to a full DataExporter dump, so the Phase-2 modded swap still needs zero parser changes. Also hit the doc-layout hook rejecting a .md attribution file under data/; moved the MIT notice to a plain .txt next to the data.

Confidence is high: 10 new tests (minimal-export unit coverage + a real-graph assertion pinning the 1453/43/142 counts + the live route), full suite 501 green, lint/mypy/precommit clean, and I exercised the route end-to-end against the real bundle.

Rough edges worth a follow-up: (1) tableTierRequired is honestly null - the export carries no table tier, so the cost engine (C) will have to derive it. (2) The en-US trim is a deliberate call I made solo; if anyone wants full localization back it's a one-line regen from upstream. (3) This is vanilla data - server-accurate numbers still wait on the DataExporter self-host (ward#585 / eco-ops#30). None of those block B-E from starting.

WARD-OUTCOME: done - /preview/recipes.json now serves the vendored Eco Gnome recipe graph (1,453 recipes) as the #98 Recipe DTO, landed on main. This one went smoothly, mostly because the #105 probe did the hard part: it named the exact file, schema, and the vendor-don't-fetch decision, so there was no real fork left to agonize over. The Eco Gnome export is cleaner than the CSV exporters the other consumers wrestle with - every quantity is a tidy `{BaseValue, Modifiers[]}`, no undeclared-column realignment, no id->name joins. So `recipes.py` ended up *shorter* than `crafting.py` despite mirroring its DTO+to_dict shape, and the streaming/SQLite-cache machinery would have been dead weight against an immutable local file, so I swapped it for a one-line in-process memo and said why in the docstring. What fought back, mildly: the source file is ~6 MB of 23-locale name blocks. Committing that felt wrong, so I trimmed `LocalizedName` to en-US only (~2.2 MB) while leaving every structural key intact - schema-identical to a full DataExporter dump, so the Phase-2 modded swap still needs zero parser changes. Also hit the doc-layout hook rejecting a `.md` attribution file under `data/`; moved the MIT notice to a plain `.txt` next to the data. Confidence is high: 10 new tests (minimal-export unit coverage + a real-graph assertion pinning the 1453/43/142 counts + the live route), full suite 501 green, lint/mypy/precommit clean, and I exercised the route end-to-end against the real bundle. Rough edges worth a follow-up: (1) `tableTierRequired` is honestly `null` - the export carries no table tier, so the cost engine (C) will have to derive it. (2) The `en-US` trim is a deliberate call I made solo; if anyone wants full localization back it's a one-line regen from upstream. (3) This is vanilla data - server-accurate numbers still wait on the DataExporter self-host (ward#585 / eco-ops#30). None of those block B-E from starting.
Sign in to join this conversation.
No description provided.