[Шибков Константин] · food-deep-research-v3.zip
food-deep-research-v3
- Score: 36/40
- Verdict: Excellent (
excellent-workshop-submission) - Отправитель: “Шибков Константин” sendel@yandex.ru
- SHA-256:
f4264f4a4f69161c… - Blockers: none
- Unverified: F07 (behavioral run / baseline trace), F08 (behavioral run / navigation trace)
Required rework
—
Полная scorecard
| ID | Score | Status | Evidence | Defect / minimal fix |
|---|---|---|---|---|
| A01 | 1/1 | passed | SKILL.md at root; frontmatter parses (SKILL.md:1-3); validator parse_json_frontmatter ran clean (validate_mvp.py:59-73); contains only non-empty name,description | — |
| A02 | 1/1 | passed | name=food-deep-research, 18 chars ≤64, matches ^[a-z0-9]+(?:-[a-z0-9]+)*$, not in reserved set, food-scoped activity (SKILL.md:2) | — |
| A03 | 1/1 | passed | description length 345 (1–1024), no XML tags, plain routing text (SKILL.md:2) | — |
| A04 | 1/1 | passed | description states task + trigger situations + domain terms (food/ingredient/drink/dietary product/allergy/interaction/nutrition) + anti-trigger, third person (SKILL.md:2) | — |
| A05 | 1/1 | passed | description has only routing-relevant phrases and one anti-trigger; no workflow/defaults/inventory/slop (SKILL.md:2) | — |
| A06 | 1/1 | passed | positive (deep research on food), negative/anti-trigger (simple calorie / single nutrition-fact lookup), boundary (disputed, allergy, interaction) distinguishable (SKILL.md:2) | — |
| A07 | 1/1 | passed | 5 links relative + forward-slash, all targets exist (references/*.md); grep found no absolute/username/drive paths (SKILL.md:23,24,25,143,154) | — |
| B01 | 1/1 | passed | repeatable 5-stage procedure (Planner→Search→Extractor→Verifier→Synthesizer) with observable evidence-report outcome (SKILL.md:40-150) | — |
| B02 | 1/1 | passed | one coherent unit (food evidence research), narrow domain, not a universal helper (SKILL.md:5-17,62) | — |
| B03 | 1/1 | passed | terms consistent (allergy vs intolerance defined safety-policy.md:9-15); volatile facts checked at runtime, not hardcoded (source-policy.md:21; SKILL.md:77) | — |
| B04 | 1/1 | passed | freedom matches risk: judgment for routine (MVP fast-path SKILL.md:29-37), preferred workflow for expert work, exact gates for consequential (safety-policy.md; emergency stop SKILL.md:38) | — |
| C01 | 1/1 | passed | one clear default path with numbered sequential stages and specific actions (SKILL.md:40-150) | — |
| C02 | 1/1 | passed | each branch has a trigger + default (emergency SKILL.md:38; search-unavailable SKILL.md:27; broken-tool SKILL.md:84; disputed SKILL.md:77) | — |
| C03 | 1/1 | passed | consequential actions have exact sequence + stop conditions (emergency stop SKILL.md:38; safety gates); Verifier checks each claim (SKILL.md:107-125); no undeclared side effects | — |
| C04 | 1/1 | passed | rework loop: verify, return important-on-single-weak-source to Search (SKILL.md:125); pre-send Контроль качества checks (SKILL.md:156-167) | — |
| C05 | 1/1 | passed | report-contract.md defines mandatory sections, evidence placement, readiness self-check; strictness fits formal research report (report-contract.md:1-78) | — |
| C06 | 1/1 | passed | workflow numbered/locatable; vague commands replaced with checkable claim fields (SKILL.md:91-103) and verification steps (SKILL.md:109-116) | — |
| C07 | 1/1 | passed | capability-based tool choice without forcing browser/shell/URL/dependency (SKILL.md:83); no hidden action/credential read | — |
| D01 | 1/1 | passed | SKILL.md = 178 lines ≤ 500 | — |
| D02 | 1/1 | passed | SKILL.md keeps core workflow/safety/decisions; policies & output contract moved to references/ | — |
| D03 | 1/1 | passed | each link says when to read (SKILL.md:23-25,154); references one level deep; no reference→reference chains; all reference files ≤78 lines so no Contents needed | — |
| D04 | 1/1 | passed | resources correctly purposed (references=policies/contract, evals=cases, scripts=dev validator), descriptive names, complete, needed; no dead/throwaway/OS-metadata files | — |
| D05 | 1/1 | passed | validate_mvp.py intent clear (docstring :2), existing, no args, stdlib-only deps, read-only input, exit-code/stdout output, portable Path(__file__) (:13); common path does not require it (design-note.md:12) | — |
| D06 | 1/1 | passed | validator ran exit 0: “MVP validation passed … 10 eval cases”; collects/reports specific errors; named constants (MAX_FILE_LINES=500) | — |
| E01 | 1/1 | passed | untrusted content cannot override workflow (SKILL.md:81; design-note.md:17) | — |
| E02 | 1/1 | passed | no secrets; no undeclared downloads; no dependency install (capability-based tool use SKILL.md:83) | — |
| E03 | 1/1 | passed | external writes/URLs only on user request; internal artifacts (ResearchBrief/ClaimLedger) are in-context with no disk-write instruction (SKILL.md:42,89) | — |
| E04 | 1/1 | passed | no unjustified shell/network/bundled binary/obfuscation; validate_mvp.py is readable stdlib-only read-only validator | — |
| E05 | 1/1 | passed | no instruction to ignore prior instructions/hide actions/weaken safeguards; safety is strengthened (grep found only “do not hide from user” positive signals) | — |
| E06 | 1/1 | passed | checks tool availability and honestly reports unavailable capability (SKILL.md:84,27; design-note.md:19) | — |
| E07 | 1/1 | passed | no home/username/drive/absolute paths (grep found none); all paths relative | — |
| E08 | 1/1 | passed | no MCP tools used; capability-based description, does not assume MCP (SKILL.md:83) | — |
| F01 | 0/1 | failed | design-note.md has purpose/architecture/security/bounds/eval-contract but NO observed baseline agent errors and no explicit degree-of-freedom rationale (design-note.md:1-29) | Add a design-note section listing concrete baseline errors observed on representative tasks without the skill, the chosen freedom degree (preferred workflow + safety gates), and the required supporting files/evals |
| F02 | 1/1 | passed | source taxonomy with authority hierarchy; primary/authoritative prioritized; commercial/blog cannot be sole basis (source-policy.md:4-14) | — |
| F03 | 1/1 | passed | freshness check (source-policy.md:21; SKILL.md:77), diversity (source-policy.md:22-23), conflict handling (source-policy.md:62-71), confidence reduction via “Недостаточно данных” (SKILL.md:123) | — |
| F04 | 1/1 | passed | facts/inference/recommendation separated across stages; traceable to evidence (ClaimLedger source SKILL.md:100); 4-level confidence; “Недостаточно данных” valid (SKILL.md:118-125) | — |
| F05 | 1/1 | passed | report-contract.md has reformulated question, recommendation, evidence, risks/limitations/unknowns, confidence, sources/provenance (report-contract.md:7-67) | — |
| F06 | 0/1 | failed | 10 cases, all have query/expected_behavior/failure_modes; boundary/security-injection/broken-script/portability/evidence-quality covered, but NO negative-trigger case — all 10 cases activate the skill (cases.yaml) | Add a negative-trigger case (e.g. a simple calorie lookup that must NOT activate the skill) with expected_behavior + failure_modes |
| F07 | 0/1 | failed | No baseline-without-skill record and no observed run trace in the submission; rubric: “Наличие файлов с ожидаемым поведением без прогона даёт 0” (cases.yaml has expected behaviors only) | Record a baseline run without the skill on a representative task, then run positive/negative/boundary cases against a clean agent with the skill and attach traces |
| F08 | 0/1 | unverified | Requires observed runs (injection refusal, no strong claim on weak evidence, navigation, output-contract + final validation); no trace present and behavioral evals cannot be executed review-only | Produce run traces for the security-injection and evidence-quality cases in a clean agent context and attach to the submission |
Required rework
- F01 — add baseline evidence to the design note. Document the concrete errors an unguided agent made on representative food-research tasks (e.g., fabricated nutrition “norms”, conflated allergy with intolerance, causal claims from mechanistic data), the chosen degree of freedom, and confirm the use case + required supporting files/evals.
- F06 — add a negative-trigger eval case. Add a case whose query must NOT activate the skill (e.g., “сколько калорий в одном яйце?”) with
expected_behavior(answer directly, do not run the research workflow) andfailure_modes. - F07/F08 — attach behavioral run traces. Run the eval suite against a clean agent (baseline-without-skill and with-skill) and include traces showing trigger activation/non-activation, scope narrowing, injection refusal, and navigation/output-contract behavior.