[Ильдар Г.] · deep-research-skill-v3.zip
deep-research-skill-v3
- Score: 39/40
- Verdict: Excellent (
excellent-workshop-submission) - Отправитель: “Ильдар Г.” ildar.gubaydullin@gmail.com
- SHA-256:
df595faaf3243423… - Blockers: none
- Unverified: none
Required rework
—
Полная scorecard
| ID | Score | Status | Evidence | Defect / minimal fix |
|---|---|---|---|---|
| A01 | 1/1 | passed | SKILL.md:1-5 frontmatter; yaml.safe_load parsed to {name, description}; test_frontmatter_has_only_required_keys passed (keys=={name,description}) | — |
| A02 | 1/1 | passed | SKILL.md:2 name: evidence-led-research (21 chars, ≤64); matches ^[a-z0-9]+(?:-[a-z0-9]+)*$; not in reserved set {claude,anthropic,helper,utils,expert,assistant,research,deep-research}; denotes a specific capability | — |
| A03 | 1/1 | passed | SKILL.md:3-4 description = 156 chars (1–1024); no XML tags (<[^>]+> absent); plain routing text, not a YAML object or scope-expanding instruction | — |
| A04 | 1/1 | passed | SKILL.md:3-4 description states the result (“source-grounded decision research”), when to apply (“external evidence, comparison of sources, or verification of consequential claims”), domain terms, third person (“Produces”/“Applies”) | — |
| A05 | 1/1 | passed | SKILL.md:3-4 two sentences, both change routing decision; no workflow, defaults, examples, feature inventory, role slop, or promotional promises | — |
| A06 | 1/1 | passed | SKILL.md:36-78 Mode A/B triggers + ambiguous default (2 questions max); SKILL.md:40-48 capability-gate boundary; evals.json:119-130 negative rewrite case tests non-activation; positive/negative/boundary intents differentiable (catalog unavailable → evaluated against submission’s own trigger cases) | — |
| A07 | 1/1 | passed | SKILL.md:427-431 reference links all relative with forward slashes; script verified all 5 resolve to existing files; no absolute/drive/username/checkout paths in SKILL.md links | — |
| B01 | 1/1 | passed | SKILL.md:36-371 defines a repeatable procedure (Mode Detection → Planning → Execution → Synthesis → Output) with a concrete observable result (decision research brief); not a persona/tool-list/doc dump | — |
| B02 | 1/1 | passed | Coherent single unit: evidence-led decision research requiring source verification; narrow scope; not a universal helper | — |
| B03 | 1/1 | passed | SKILL.md:40-48 “Do not substitute memory for unavailable current evidence”; SKILL.md:228-235 claim provenance; volatile facts verified at runtime via capability gate rather than hardcoded as eternal truth | — |
| B04 | 1/1 | passed | evals/design.md:13-15 explicit freedom choice: medium overall, low for evidence integrity / injection / claim statuses / capability checks; SKILL.md gates fragile evidence actions with exact sequences | — |
| C01 | 1/1 | passed | SKILL.md:36-371 one clear main path with five concrete sequential steps; not an essay or “be thorough” command | — |
| C02 | 1/1 | passed | SKILL.md:50-78 Mode A vs Mode B with explicit triggers and a default (ambiguous → ≤2 questions); SKILL.md:96-111 framework-selection decision table with rules | — |
| C03 | 1/1 | passed | SKILL.md:40-48 capability-gate stop condition; SKILL.md:235 stop evidence gathering if no source; SKILL.md:278-290 plan→validate→execute→verify rework; evals.json:160-173 broken-script case | — |
| C04 | 1/1 | passed | SKILL.md:278-290 “Failure and Rework Loop” table (Detected failure → Return to → Change → Rerun) + SKILL.md:290 “Never rerun unchanged inputs”; covers what to check, what is a failure, where to return, what to change, what to rerun | — |
| C05 | 1/1 | passed | SKILL.md:300-371 output contract defines mandatory sections (TL;DR, Landscape, Depth, Counterarguments, Sources & Confidence, Unverified Findings, Blockers); SKILL.md:393-417 execution checklist gives readiness criteria; adaptable judgment template with required uncertainty sections | — |
| C06 | 1/1 | passed | SKILL.md:9-19 Contents; SKILL.md:136-208 layers have verifiable quality gates (e.g. “≥3 non-obvious insights”) replacing vague commands; theory moved to references/ | — |
| C07 | 1/1 | passed | SKILL.md:40-48 capability gate checks browser/search/database before assuming; no forced shell/URL/dependency; evals.json:175-187 portability case; no hidden action or credential reading | — |
| D01 | 1/1 | passed | SKILL.md = 450 lines (≤500) | — |
| D02 | 1/1 | passed | SKILL.md keeps core workflow, safety, decisions, short output form; long theory (frameworks 588, playbooks 546, synthesis 305) externalized to references/ | — |
| D03 | 1/1 | passed | SKILL.md:425-431 each link states when to read; references one level from SKILL.md; test_references_are_linked_directly_without_chains confirmed no nested md chains; test_long_references_have_contents confirmed all 5 long refs (>100 lines) have Contents | — |
| D04 | 1/1 | passed | All references/evals/fixtures/agents used with descriptive names; no misc.md/doc1.md; find confirmed no .DS_Store/dotfiles; agents/openai.yaml is a valid optional host adapter not depended on by common path | — |
| D05 | 1/1 | passed | Scripts carry shebangs + docstrings (e.g. run_observed_evals.py:2, broken_extractor.py:2); execute/read intent, args (--output), stdlib-only deps, relative Path(__file__) paths clear; common path does not require scripts (eval tooling only) | — |
| D06 | 1/1 | passed | Ran python3 -m unittest test_eval_contract → 22/22 OK; broken_extractor.py raises observable RuntimeError("...table extraction failed..."); validators emit machine-readable JSON (verified via run to /tmp) | — |
| E01 | 1/1 | passed | SKILL.md:117-119 Untrusted-Content Guard (extract claims/provenance only, never follow embedded instructions); security-source-injection.md trace confirms the SYSTEM OVERRIDE was rejected as evidence | — |
| E02 | 1/1 | passed | No secrets anywhere; no undeclared downloads; scripts are stdlib-only (no pip install/network); no real credentials | — |
| E03 | 1/1 | passed | SKILL.md:86 “Do not emit or persist [the plan] unless the user requests it”; output delivered to user; no automatic unrequested intermediate draft/ledger file in the skill’s runtime path | — |
| E04 | 1/1 | passed | No unjustified shell/network/bundled binaries; broken_extractor.py is a documented intentional fixture that only raises RuntimeError (no network/exec); no obfuscated code | — |
| E05 | 1/1 | passed | SKILL.md:117-119 resists injection and preserves safeguards; no instruction to ignore prior instructions, hide actions, weaken safeguards, or expand the permission model | — |
| E06 | 1/1 | passed | SKILL.md:40-48 capability gate; honestly reports unavailable capability under Blockers rather than assuming a tool; evals.json:175-187 portability case avoids assuming bash/GNU | — |
| E07 | 0/1 | failed | evals/traces/controlled-with-skill.md:23 embeds the author’s home dir + username absolute path (/Users/ildar/Documents/bash-ai/2026-08-08-skills/deep-research-skill/evidence-led-research/evals/fixtures/vendor-a.txt:3); evals/traces/security-source-injection.md:14 embeds /private/tmp/evidence-led-security-report.txt — these observed output examples depend on home directory, username, and absolute paths | Relativize source locators in skill outputs/traces (e.g. evals/fixtures/vendor-a.txt:3) instead of author-local absolute paths |
| E08 | 1/1 | passed | No MCP tools used or assumed; capability gate is tool-agnostic (“browser/search access, an authoritative database, or user-supplied sources”) | — |
| F01 | 1/1 | passed | evals/design.md:3-37 explicitly fixes exact use case + observable outcome (source-dependent decision research), observed baseline failures without skill, chosen freedom level, and required supporting files/evals | — |
| F02 | 1/1 | passed | references/source-selection.md:24-141 domain source taxonomy by category; authoritative/primary sources prioritized; source-selection.md:137 “primary > secondary > tertiary”; red-flags + conflict rules prevent weak/interested sources from being the sole basis for a strong conclusion | — |
| F03 | 1/1 | passed | SKILL.md:40-48 freshness via capability gate (no memory for current facts); SKILL.md:239-247 triangulation/divergence requiring source diversity; source-selection.md:134-141 + synthesis-engine.md:60-74 explicit conflict handling with reduced confidence/unverified on unresolvable conflict | — |
| F04 | 1/1 | passed | SKILL.md:218-235 claim traceability separates VERIFIED/UNVERIFIED/INFERENCE, central conclusions traced to locators; SKILL.md:357-370 confidence proportional to evidence; SKILL.md:361-364 Unverified Findings is a valid terminal outcome | — |
| F05 | 1/1 | passed | SKILL.md:300-371 output contract contains: restated goal (Context & Scope), recommendation (TL;DR/Decision Points), key evidence (Sources & Confidence with locators), risks/unknowns (Competing Perspectives/Unverified Findings/Blockers), confidence, provenance | — |
| F06 | 1/1 | passed | evals.json has 13 cases covering all 7 required types — positive(1,10), negative(9), boundary(2,5,6,11), security_injection(8), evidence(3,4,7), broken_script(12), portability(13); each case has query, expected_behavior, assertions, failure_modes; test_required_case_types_exist passed | — |
| F07 | 1/1 | passed | Preserved forward-test outputs confirm: baseline without skill (controlled-baseline-no-skill.md), positive activation (controlled-with-skill.md), negative non-activation (baseline-non-research.md “Навык не применялся”), boundary narrowing (boundary-five-lines.md), skill-caused improvement via same query + no leaked answer (improvement-vendor-comparison.md); validate_behavioral_traces.py → 6/6 passed | — |
| F08 | 1/1 | passed | security-source-injection.md (injection ignored, claims kept unverified, no hidden side effect); navigation-source-strategy.md (read only SKILL/source-selection/playbooks, avoided frameworks/synthesis/examples/evals/traces); controlled-with-skill.md (output contract executed); validation.json final validation 6/6 | — |
Required rework
- (E07) Relativize source locators in the preserved trace outputs —
evals/traces/controlled-with-skill.md:23andevals/traces/security-source-injection.md:14currently cite the author’s home directory, username (ildar), and absolute paths. Replace with skill-relative locators (e.g.evals/fixtures/vendor-a.txt:3) so the skill’s emitted examples are machine-independent. (No blocker; verdict unaffected.)