Skip to content

fix(deriver): reduce ephemeral explicit observations#445

Open
Vein05 wants to merge 2 commits intoplastic-labs:mainfrom
Vein05:fix/444-deriver-ephemeral-conclusions
Open

fix(deriver): reduce ephemeral explicit observations#445
Vein05 wants to merge 2 commits intoplastic-labs:mainfrom
Vein05:fix/444-deriver-ephemeral-conclusions

Conversation

@Vein05
Copy link
Copy Markdown

@Vein05 Vein05 commented Mar 24, 2026

Summary

  • narrow the minimal deriver prompt toward durable, cross-session-useful observations
  • explicitly exclude one-off tasks, transient workflow updates, and short-lived session logistics from explicit observation extraction
  • add a focused prompt regression test covering the new durable vs ephemeral guidance

Closes #444

Validation

  • uv run ruff check src/deriver/prompts.py tests/deriver/test_prompts.py tests/deriver/test_representation_crud.py
  • uv run pytest --noconftest -n0 tests/deriver/test_prompts.py tests/deriver/test_representation_crud.py -q
  • uv run pytest -n0 tests/deriver -q

All tests passed.

Notes

  • this change is intentionally prompt-first and does not add heuristic filtering to the deriver pipeline, as that would require bigger changes
  • the codebase currently doesn't include a dedicated benchmark for ephemeral-observation suppression vs durable-memory retention, so validation here is minimal.

Summary by CodeRabbit

  • Bug Fixes

    • Tightened observation filtering to prioritize durable facts (standing instructions, long-lived preferences, relationships, commitments, ongoing situations) and exclude transient items (one-off tasks, temporary workflow steps, ephemeral process updates).
    • Added explicit handling for declared enduring preferences (e.g., “short version first”).
  • Tests

    • Added test coverage verifying durable-observation bias, explicit skip examples for transient items, and respect for declared preferences.
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 24, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b21fc271-e18f-45a6-8a8a-1698f5ac07b2

📥 Commits

Reviewing files that changed from the base of the PR and between 4c60eb1 and 7e4cf01.

📒 Files selected for processing (1)
  • tests/deriver/test_prompts.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/deriver/test_prompts.py

Walkthrough

Modified the minimal deriver prompt to extract only durable, long-lived observations (standing instructions, long-lived preferences, relationships, commitments, ongoing situations) and to exclude one-off tasks, temporary workflow steps, and ephemeral session updates unless explicitly stated as enduring. Added corresponding test validating the prompt text.

Changes

Cohort / File(s) Summary
Prompt Refinement
src/deriver/prompts.py
Tightened minimal_deriver_prompt instructions to prioritize durable facts and skip transient/session-ephemeral items; updated examples to include a durable-preference example and SKIP examples for transient actions.
Test Coverage
tests/deriver/test_prompts.py
New test that dynamically imports the prompts module and asserts minimal_deriver_prompt(peer_id="alice", ...) contains directives favoring durable observations, explicit SKIP entries for transient phrases, and an EXPLICIT echo of the input request.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 I hop the prompts with careful paws,

keeping what lasts and hiding the claws.
Short first, I whisper, let habits stay—
fleeting tasks, you skip away.
✨🧺

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(deriver): reduce ephemeral explicit observations' directly summarizes the main change: narrowing the minimal deriver prompt to exclude ephemeral/transient observations and favor durable ones.
Linked Issues check ✅ Passed The changes implement the core objective from #444 by modifying the minimal_deriver_prompt to explicitly exclude session-ephemeral events (temporary tasks, workflow steps, session logistics) and prioritize durable observations. The test validates this behavior with specific assertions on prompt content.
Out of Scope Changes check ✅ Passed All changes are scoped to the deriver prompt logic and its test coverage. No out-of-scope modifications to unrelated systems, filtering mechanisms, or other modules were introduced.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/deriver/test_prompts.py (1)

1-12: Replace dynamic module loading with standard imports for simplicity.

The importlib pattern here adds unnecessary complexity. Since the project already imports from src.* throughout the test suite, a standard import would be simpler and more maintainable:

from src.deriver.prompts import minimal_deriver_prompt

Additionally, the bare assert on line 8 will produce a cryptic ImportError or AssertionError at test collection time rather than a clear test failure if something goes wrong.

♻️ Proposed simplification
-import importlib.util
-from pathlib import Path
-
-_PROMPTS_PATH = Path(__file__).resolve().parents[2] / "src" / "deriver" / "prompts.py"
-_PROMPTS_SPEC = importlib.util.spec_from_file_location(
-    "deriver_prompts", _PROMPTS_PATH
-)
-assert _PROMPTS_SPEC is not None and _PROMPTS_SPEC.loader is not None
-_PROMPTS_MODULE = importlib.util.module_from_spec(_PROMPTS_SPEC)
-_PROMPTS_SPEC.loader.exec_module(_PROMPTS_MODULE)
-
-minimal_deriver_prompt = _PROMPTS_MODULE.minimal_deriver_prompt
+from src.deriver.prompts import minimal_deriver_prompt
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/deriver/test_prompts.py` around lines 1 - 12, Replace the dynamic
import block (_PROMPTS_PATH, _PROMPTS_SPEC, _PROMPTS_MODULE and the assert +
exec_module calls) with a normal import statement that directly imports
minimal_deriver_prompt from src.deriver.prompts (e.g. from src.deriver.prompts
import minimal_deriver_prompt); remove the manual spec/loader assertions and
exec_module usage so import errors surface as normal ImportError during test
collection instead of relying on the bare assert.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/deriver/test_prompts.py`:
- Around line 1-12: Replace the dynamic import block (_PROMPTS_PATH,
_PROMPTS_SPEC, _PROMPTS_MODULE and the assert + exec_module calls) with a normal
import statement that directly imports minimal_deriver_prompt from
src.deriver.prompts (e.g. from src.deriver.prompts import
minimal_deriver_prompt); remove the manual spec/loader assertions and
exec_module usage so import errors surface as normal ImportError during test
collection instead of relying on the bare assert.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 46b7290d-f7c0-499e-90d7-586dbd0271b8

📥 Commits

Reviewing files that changed from the base of the PR and between b3c1917 and 4c60eb1.

📒 Files selected for processing (2)
  • src/deriver/prompts.py
  • tests/deriver/test_prompts.py
@Vein05
Copy link
Copy Markdown
Author

Vein05 commented Mar 24, 2026

@VVoruganti what would you suggest here?

I'm getting a

ValueError: Missing client for Summary: google

when i try to import from src it through src.deriver.__init__, which turns into app/client bootstrap and fails. Do we need to load the entire client for this simple test. If needed, I can work around that!

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant