Repository navigation
test(storage): ingest growth must trip the footprint finding (#4099) - #4127
Conversation
Drives real LCM payload ingest into a temporary profile and asserts the census measures it and doctor reports OverBudgetStore past the bound (#4099).
|
There was a problem hiding this comment.
Devin Review found 3 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| let read = reads.remove(configured_store); | ||
| if read.is_none() && footprint_keys.contains(configured_store.as_str()) { | ||
| continue; |
There was a problem hiding this comment.
🟡 Zero footprint budgets bypass validation
When store_soft_budgets_bytes sets a footprint key to zero, soft_budget_findings skips its validation. The footprint accepts that zero bound, producing a healthy finding for empty entries or a degraded finding for any bytes.
Learn more
Owner budgets are validated through store_soft_budget, which rejects zero limits. Footprint keys skip that call, while census_profile_footprint uses override values without checking them. Consequently the same invalid budget gets a normal size verdict instead of an invalid-configuration result.
Example: Set store_soft_budgets_bytes["lcm-payloads"] = 0 and ingest a 10-byte payload. Doctor emits a degraded 10-of-0 footprint finding; without payload files it emits a healthy 0-of-0 finding. Neither identifies the invalid budget.
Recommended fix: Validate configured budget values before skipping footprint-owned keys, and ensure profile and any long footprint keys can be validated without forcing them through a filename-only store-key constraint.
Was this helpful? React with 👍 or 👎 to provide feedback.
| let persisted = self | ||
| .table_growth_state_path | ||
| .as_deref() | ||
| .zip(path.to_str()) | ||
| .and_then(|(state_path, key)| load_table_growth_states(state_path).remove(key)) | ||
| .unwrap_or_default(); |
There was a problem hiding this comment.
🟡 Replacement stores inherit stale growth baselines
When a database is reset at the same path, register_port restores its old table-growth state. The persisted state uses only the path as its key, so Doctor shows stale measurements and compares unrelated tables.
Learn more
The persistent table-growth file maps database path strings to watermarks and last readings. A newly mounted store uses register_port to reload the old value; persist_table_growth_states only removes a record when its database path no longer exists at persistence time. An in-place reset or database replacement can preserve the path throughout, so it inherits unrelated historical growth.
Example: Measure messages at 80 MiB, reset the project graph database at the same path to an empty new store, then remount. Until maintenance runs, Doctor still reports the 80 MiB measurement; subsequent measurements compare against 80 MiB instead of establishing a new baseline.
Recommended fix: Persist a store generation or durable database identity with the watermarks and verify it during registration; discard persisted state when that identity differs. Also invalidate state on explicit store reset.
Was this helpful? React with 👍 or 👎 to provide feedback.
| let primary = if let Some(end) = components.iter().position(|name| is_wal(name)) { | ||
| (components[..=end].join("/"), ProfileFamilyV1::Wal) | ||
| } else if let Some(stem) = first | ||
| .find(GRAPH_CONTAINER_SUFFIX) | ||
| .map(|at| &first[..at + GRAPH_CONTAINER_SUFFIX.len()]) | ||
| { | ||
| // A top-level container and its sidecars (`.verified`, `.spill`). | ||
| (stem.to_owned(), ProfileFamilyV1::GraphContainer) | ||
| } else if first == "global.db" || first.starts_with("global.db-") { | ||
| ("global.db".to_owned(), ProfileFamilyV1::ProfileRegistry) | ||
| } else if first.starts_with("user-sessions.db") { |
There was a problem hiding this comment.
🟡 Database WALs evade their family bounds
When global.db-wal or user-sessions.db-wal grows, footprint_keys assigns it a separate WAL key. Neither byte count reaches its database family's bound, so that family's finding can remain healthy despite an oversized database family.
Learn more
The classifier checks the WAL suffix before the profile database names. This splits a SQLite database from its WAL even though StoreKindBytesV1 and database_family_bytes treat database sidecars as part of the database family. The WAL gets its own bound, but the family's verdict no longer measures the entire database family.
Example: Suppose global.db occupies 3 GiB and global.db-wal occupies 2 GiB. The registry's 4 GiB bound reads 3 GiB and passes, though its SQLite family occupies 5 GiB.
Recommended fix: Count WAL sidecars in the corresponding profile database family as well as their WAL-specific overlay, mirroring how project stores count both store bytes and a WAL overlay. Keep the profile total summed from files only.
Was this helpful? React with 👍 or 👎 to provide feedback.
Part of #4099.
#4123 adds the profile footprint census and its doctor findings, with unit tests over synthetic directories. Nothing yet proved that real growth reaches the finding. In #4099, LCM payload files and the profile session store grew past 100 GB before anyone saw it.
This adds a
session_suiteregression test that drives the real ingest path. It writes six externalized LCM payloads through the host admission runtime into a temporary profile, never the operator's~/.tracedecay. Then it asserts the following:lcm-payloadsentry equals the bytes on disk and the profile session store grew.OverBudgetStoreas degraded, and the statement names the measured size and the bound.If growth stops reaching the census, or the census stops reaching doctor, this test fails.
Verification
bazel test //crates/tracedecay:session_suitepasses, includingprofile_footprint::session_ingest_growth_trips_the_payload_footprint_finding.footprint_findingsfed zero bytes instead of the measured entry size, the test fails.rustfmt_check_2024fortracedecay.