FN-6357: document external integration evidence format
Document the labeled provenance evidence layout expected by spec validation. - Add an AGENTS.md example for required external integration evidence fields. - Expand contributing guidance with accepted labels, URL expectations, and checksum rules. - Add a regression test that keeps the documented example aligned with the evidence gate. Files changed: AGENTS.md | 14 ++++++++ docs/contributing.md | 28 ++++++++++++++++ .../src/__tests__/docs-evidence-example.test.ts | 37 ++++++++++++++++++++++ 3 files changed, 79 insertions(+) Fusion-Task-Id: FN-6357 Fusion-Task-Lineage: 788260ed-d5cf-4592-b117-40af5c45e0a1
This commit is contained in:
14
AGENTS.md
14
AGENTS.md
@@ -19,6 +19,20 @@ Any task integrating a third-party tool (CLI, daemon, downloadable binary, insta
|
||||
|
||||
Missing evidence is a blocking REVISE. Never invent release URLs, binary names, or hashes.
|
||||
|
||||
Example evidence section shape:
|
||||
|
||||
```markdown
|
||||
## External Integration Evidence
|
||||
|
||||
- Canonical upstream repo URL: https://github.com/max-sixty/worktrunk
|
||||
- Docs / homepage URL: https://worktrunk.dev/
|
||||
- Release / download URL: https://github.com/max-sixty/worktrunk/releases/latest/download/wt-linux-x64.tar.gz
|
||||
- Binary / CLI name: `wt`
|
||||
- Checksum: `sha256-<digest>` (or `upstream-pending-verification` until the checksum is pinned)
|
||||
```
|
||||
|
||||
See `docs/contributing.md` for the fuller spec-authoring guidance and accepted labeled layout variants.
|
||||
|
||||
### Finalizing Changes
|
||||
|
||||
When a change affects published `@runfusion/fusion`, add a changeset (example: `.changeset/<name>.md` with `"@runfusion/fusion": patch`).
|
||||
|
||||
@@ -112,6 +112,34 @@ Fusion tests must run against disposable test data, never live local state:
|
||||
|
||||
If you add or change test entrypoints, keep this isolation guard path intact and ensure guard + test execution share the same disposable HOME so changed/full/cached paths stay consistent.
|
||||
|
||||
## Spec authoring: provenance evidence for outside tooling
|
||||
|
||||
Any task that wires in an outside command-line program, daemon, separately-fetched program, or package-managed dependency must include provenance evidence in its `PROMPT.md`. The deterministic spec-validation gate (`detectExternalIntegrationEvidenceGaps`) REVISEs specs that mention this kind of outside tooling without enough provenance to audit where it comes from and what command or artifact is expected.
|
||||
|
||||
Use a dedicated `## External Integration Evidence` or `## External-Integration Evidence` section when possible. The gate accepts semantically labeled bullets; labels may include or omit a trailing `URL`/`name`, may use `/` or `:` separators (for example `Docs / homepage URL:` or `Docs/homepage:`), and URLs may be bare or backtick-wrapped.
|
||||
|
||||
Include all five evidence fields:
|
||||
|
||||
1. Canonical upstream repo URL — a GitHub URL with distinct owner/repo; duplicate owner/owner placeholders are rejected.
|
||||
2. Docs / homepage URL — a distinct non-GitHub, non-artifact URL.
|
||||
3. Release / download URL — a GitHub `…/releases/…` URL, a generic `…download…` URL, an npm `registry.npmjs.org/<pkg>/-/<name>-<ver>.tgz` URL, or any `.tgz`/`.tar.gz` artifact URL.
|
||||
4. Binary / CLI name — the command name in backticks, such as `` `wt` ``.
|
||||
5. Checksum — a `sha256`/`sha512` digest, a pinned-manifest token, or the literal `upstream-pending-verification` marker. The marker is accepted for the checksum field only; never use it in place of source, docs, or artifact URLs.
|
||||
|
||||
Never fabricate source URLs, command names, release locations, or checksums. Cite real provenance, or use `upstream-pending-verification` only for the checksum field while the digest is being pinned.
|
||||
|
||||
<!-- evidence-example:start -->
|
||||
```markdown
|
||||
## External Integration Evidence
|
||||
|
||||
- Canonical upstream repo URL: https://github.com/max-sixty/worktrunk
|
||||
- Docs / homepage URL: https://worktrunk.dev/
|
||||
- Release / download URL: https://github.com/max-sixty/worktrunk/releases/latest/download/wt-linux-x64.tar.gz
|
||||
- Binary / CLI name: `wt`
|
||||
- Checksum: `sha256-<digest>` (or `upstream-pending-verification` until the checksum is pinned)
|
||||
```
|
||||
<!-- evidence-example:end -->
|
||||
|
||||
## Quality Gate Checklist
|
||||
|
||||
Before submitting changes, verify:
|
||||
|
||||
37
packages/engine/src/__tests__/docs-evidence-example.test.ts
Normal file
37
packages/engine/src/__tests__/docs-evidence-example.test.ts
Normal file
@@ -0,0 +1,37 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { detectExternalIntegrationEvidenceGaps } from "../spec-validation/external-integration-evidence.js";
|
||||
|
||||
const workspaceRoot = resolve(import.meta.dirname, "../../../..");
|
||||
const contributingPath = resolve(workspaceRoot, "docs", "contributing.md");
|
||||
|
||||
function extractEvidenceExample(): string {
|
||||
const contributing = readFileSync(contributingPath, "utf8");
|
||||
const match = contributing.match(
|
||||
/<!-- evidence-example:start -->([\s\S]*?)<!-- evidence-example:end -->/,
|
||||
);
|
||||
expect(match?.[1]).toBeDefined();
|
||||
|
||||
const fenced = match?.[1]?.trim() ?? "";
|
||||
const fenceMatch = fenced.match(/^```markdown\r?\n([\s\S]*?)\r?\n```$/);
|
||||
expect(fenceMatch?.[1]).toBeDefined();
|
||||
return fenceMatch?.[1] ?? "";
|
||||
}
|
||||
|
||||
describe("documented external integration evidence example", () => {
|
||||
it("satisfies the spec-validation gate", () => {
|
||||
const example = extractEvidenceExample();
|
||||
|
||||
expect(detectExternalIntegrationEvidenceGaps({ promptContent: example })).toEqual([]);
|
||||
});
|
||||
|
||||
it("fails the gate when checksum evidence is removed", () => {
|
||||
const example = extractEvidenceExample();
|
||||
const withoutChecksum = example.replace(/^- Checksum:.*$/m, "- Checksum:");
|
||||
|
||||
const findings = detectExternalIntegrationEvidenceGaps({ promptContent: withoutChecksum });
|
||||
expect(findings.length).toBeGreaterThan(0);
|
||||
expect(findings[0]?.missing).toContain("checksum-or-source-of-truth-evidence");
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user