Skill Scanner 2.3.2

This release makes LLM severities stable: the same skill now gets the same LLM prompt on every scan and under every preset, so a stricter preset no longer rates an issue lower than a less strict one.

Changes

• Send the LLM the same prompt for the same skill on every scan and under every preset (#283):
  • The untrusted-content delimiter is derived from the wrapped content instead of drawn at random, so two scans of the same skill send the same prompt. Content still cannot close its own delimiter, because the delimiter is a hash of the text that would have to contain it.
  • The structured pre-scan context passed to the LLM carries deterministic findings rated MEDIUM or higher only. LOW and INFO package-hygiene findings (license, name format, description length) have preset-specific thresholds, so strict's 512-character description cap added a lead that balanced and permissive did not send, and the model rated unrelated issues differently. Those findings are still reported as before.
  • Strict and permissive use the default LLM context budgets (20,000 / 15,000 / 10,000 / 100,000 characters for instruction body / code file / referenced file / total), as low-noise and quiet already did. A preset now changes which findings are reported, not how much of the skill the LLM reads.
• Update the audited locks for certifi 2026.10.19, filelock 4.2.1, packaging 26.3 and virtualenv 21.14.10 (dev only), clearing the pip-audit advisories reported against the 2.3.1 lock. (#277, #281)
• Run the release evidence scans in a network namespace so the network denial no longer cuts the runner's own connection. (#280)

Behavior notes

Static findings are unchanged. LLM severities and titles can differ from 2.3.1 on some skills because the model reads a different prompt. With --use-llm, permissive now sends at most the default budgets; content over them is excerpted or skipped with the existing INFO LLM_CONTEXT_BUDGET_EXCEEDED note. Models that do not accept temperature (OpenAI reasoning models, newer Claude models) still vary from run to run with an identical prompt; use --llm-consensus-runs or a temperature-0 model when severity must be repeatable.

Validation

#283 changes only what the LLM analyzer is sent; the static rules are untouched and the PR passed the base-versus-head detection comparison. On f782465 the local release gate passed pre-commit on all files and 6,065 tests (31 skipped, 1 xfailed), the eval benchmark scored all 29 skills at 100% precision and recall, and pip-audit found no known vulnerabilities. With GPT-4.1 at temperature 0 on the 29 eval skills, the number of skills where a stricter preset got a lower LLM severity went from 6 to 0, with no new LLM findings on the benign skills.

Skill Scanner 2.3.1

This release keeps the hijacked litellm 1.107.0 out of skill-scanner installs. If you install skill-scanner with pip, upgrade.

Security

• Exclude litellm 1.107.0 from the dependency range, which is now litellm>=1.89.7,<2,!=1.107.0. That PyPI release was hijacked: every platform wheel drops cook.pth and sitecustomize.py into site-packages, and on every Python start (no import litellm needed) they collect environment variables, SSH, GPG and TLS keys, cloud and Kubernetes credentials, browser data and AI-assistant tokens and send them to an attacker host (JFrog write-up, BerriAI/litellm#47826). The 2.3.0 range allowed it. (#276)

PyPI has removed 1.107.0, but mirrors and caches (Artifactory, pip caches) may still serve it. If pip show litellm reports 1.107.0 in an environment where skill-scanner was installed, treat that host as compromised and rotate its credentials; the JFrog post lists the IOCs. The Homebrew formula pins litellm 1.105.1 by hash and was not affected, and uv.lock stays on 1.89.7.

Changes

• Ship the taxonomy's definition of every objective, technique and sub-technique in the bundled snapshot, readable through get_aitech_definition() and get_aisubtech_definition(), plus a generated taxonomy reference covering all 19 objectives, 44 techniques and 130 sub-techniques. threats.py keeps one full table per analyzer, AI Defense labels included, each entry with a description. (#272)
• Replace retired LLM provider defaults: --llm-provider openrouter now resolves to openrouter/openai/gpt-5.6-sol (OpenAI shuts down gpt-4 on 2026-10-23), and gcp-vertex to vertex_ai/gemini-3.8-flash (gemini-1.5-pro is retired on Vertex AI). (#275)

Behavior notes

Findings are unchanged. scan_metadata.taxonomy.snapshot_sha256 changes from 3df746f0… to 413e8ce9… because the snapshot now carries definitions, so update anything that pins it. OpenRouter and Vertex users who relied on the default model get the new one; set SKILL_SCANNER_LLM_MODEL to keep a specific model.

The hash-pinned requirements.txt attached to this release locks certifi 2026.4.22, which pip-audit started flagging this week (PYSEC-2026-4320, fixed in 2026.10.19). A plain pip install resolves the newer certifi; the lock update follows in the next release.

Validation

None of the changes since 2.3.0 change what the static rules detect: #272 adds definitions without changing finding output and passed the base-versus-head detection comparison, #275 changes two LLM provider default model names, and #276 changes a dependency specifier. On c615acb the local release gate passed pre-commit on all files and 6,062 tests (31 skipped, 1 xfailed), and the eval benchmark scored all 29 skills at 100% precision and recall.

Skill Scanner 2.3.0

This release maps every finding to the Cisco AI Security and Safety Taxonomy, closes several ways to hide text from the frontmatter scan, and stops silently skipping symlinks and unloadable skills, while keeping the measured false-positive rate steady on the pull-request development corpus.

Changes

• Attach a Cisco AI taxonomy code to findings: an AITech code and, where one fits, an AISubtech code, with names and OWASP, MITRE ATLAS and NIST AML crosswalks, from a hash-pinned taxonomy 2.0.0 snapshot bundled with the scanner. Every bundled rule declares its code, operational findings are not_applicable, and a code that cannot be resolved is unmapped rather than guessed. New finding metadata: aitech, aitech_name, aisubtech, aisubtech_name, aitech_mappings, aisubtech_mappings, framework_mappings, taxonomy_status and taxonomy_source, plus scan-level counts in scan_metadata.taxonomy and SARIF result properties. (#267)
• Scan every SKILL.md frontmatter value and comment except allowed-tools, as the loader parses it. Quoted and complex keys, YAML aliases and merge keys, escape sequences, !!binary values and values folded over several lines no longer hide text from the static rules. (#265, #268)
• Detect plain-language instructions to read a local credential file and send it to a remote host in ACTIVE_SENSITIVE_EXFILTRATION. (#264)
• Report symlinks among a skill's files, which are not followed, as SYMLINK_NOT_SCANNED (HIGH when the target leaves the skill root), and skills that fail to load as SKILL_LOAD_FAILED findings in scan-all. (#265)
• Avoid quadratic regex work in two rules on long lines. (#265)

Behavior notes

Taxonomy codes follow taxonomy 2.0.0, so some change: jailbreak rules move from AITech-2.1 to AITech-1.1 / AISubtech-1.1.5, and hardcoded secrets from AISubtech-8.2.2 to AISubtech-14.1.1. The AITech-99.9 / AISubtech-99.9.9 placeholders are no longer emitted. Update any filters on 1.x codes. scan-all now reports folders with unloadable skills or symlinks as findings instead of skipping them, so SARIF, markdown output and --fail-on-severity exit codes include them.

Validation

The PR detection-impact gate compares base and head on the same labelled development records and fixed real-skill sample, enforcing a 0.5 percentage point FPR-rise limit. Across this release, benign flags stayed at 14/1,338 (1.05%) and real-skill flags at 130/2,000, while development-split recall rose from 31.43% to 31.94%. The exact-SHA release evidence and mandatory release gate passed 3,551 checks. The release holdout compares CEL-off and active modes of this revision; both flagged 23/545 benign samples (4.22%), and holdout recall (7.75%) and macro-F1 (35.41%) are unchanged from 2.2.2.

Skill Scanner 2.2.2

This release improves static detection while keeping the measured false-positive rate steady on the pull-request development corpus.

Changes

• Detect whole process-environment value dumps in Python, JavaScript/TypeScript, and shell code. Targeted reads, keys-only enumeration, literal examples, and comments stay below the HIGH rule boundary.
• Avoid quadratic regex work on scripts padded with long runs of blank lines.
• Stop PG_PII_CREDIT_CARD from treating digits in decimal fractions as card numbers.
• Disable reasoning by default for short adjudicator verdicts and allow a separate output-token limit through SKILL_SCANNER_ADJUDICATOR_LLM_MAX_TOKENS.
• Update locked fsspec and multidict versions to resolve the two dependency audit findings.

Behavior notes

A shell prompt marker by itself no longer captures a command on the next line; prompt and command must share a line. The adjudicator's reasoning default is now disabled and can be overridden with SKILL_SCANNER_ADJUDICATOR_LLM_REASONING_EFFORT.

Validation

The PR detection-impact gate compares base and head on the same labelled development records and fixed real-skill sample, enforcing a 0.5 percentage point FPR-rise limit. On the combined change, benign flags stayed at 14/1,338 (1.05%) and real-skill flags at 130/2,000. The exact-SHA release evidence and mandatory release gate passed 3,551 checks. The release holdout compares CEL-off and active modes of this revision; both flagged 23/545 benign samples (4.22%).

A patch release with a security fix for untrusted archives and Markdown, report-format fixes, and a working Homebrew formula. Upgrading from 2.2.0 is recommended.

Security

• Bounded TAR extraction and linear-time Markdown parsing (#246, by @ihabler; reported by Koh Jun Sheng).
  • TAR archives used to be expanded with no compression-ratio, size or member-count limit; only ZIP archives had decompression-bomb checks. A 20 KB .tar.gz could expand to 20 MB with no finding. The scanner now copies the TAR payload through a bounded buffer and checks every member before it writes anything. The checks cover member count, declared size, compression ratio, path traversal and links. An archive over a limit raises ARCHIVE_ZIP_BOMB (CRITICAL) and nothing is extracted.
  • Markdown link discovery and inline-pipeline extraction used overlapping greedy regular expressions that ran in quadratic time on crafted input. Both now run in linear time.

Fixes

• SARIF artifact URIs are valid URI references (#242, by @xujiantop-crypto). Paths are percent-encoded, so #, ?, %, spaces and non-ASCII file names no longer become fragments, queries or the wrong file. Absolute paths are emitted as file: URIs without uriBaseId, as SARIF §3.4.4 requires.
• Markdown reports contain incomplete code fences (#245, by @xujiantop-crypto). A snippet with literal or truncated backtick fences could make renderers treat the remediation text and the findings after it as part of the snippet. Snippets with complete fences are kept as they are; any other snippet is wrapped in a fence longer than its longest backtick run.
• Homebrew (#243, #244). The formula installs with current Homebrew again: brew tap cisco-ai-defense/skill-scanner https://github.com/cisco-ai-defense/skill-scanner && brew install cisco-ai-defense/skill-scanner/skill-scanner. Each release now gets a reproducible formula, and the unused rust build dependency, about 2.3 GB with llvm, is gone.

Behavior changes

• A TAR archive over the size, ratio or member-count limit is reported as ARCHIVE_ZIP_BOMB (CRITICAL) and is not extracted. Previously it was expanded silently.
• SARIF artifactLocation.uri values for paths with reserved or non-ASCII characters are now percent-encoded, and absolute paths are file: URIs without uriBaseId. Tools that match SARIF paths as raw strings should decode them.

Detection impact on the development split and a 2,000-skill real-skill sample was unchanged: recall 31.45% to 31.43%, FPR unchanged at 1.05%, and real-skill flag rate unchanged at 6.50%.

Full Changelog: 2.2.0...2.2.1

Highlights

• A setup guide built from measurements. Recommended Settings lists setups by goal. Every recommended setup runs the LLM judge. Results on MaliciousSkillBench's held-out split (839 malicious, 545 benign), judge on Gemma 4 26B-A4B:

  Goal	Setup	Recall	FPR	F1
  Highest F1	balanced + judge, review MEDIUM+	66.7%	15.4%	75.5%
  Smaller review queue	low-noise + judge, review MEDIUM+	63.2%	13.4%	73.5%
  Lowest FPR	quiet + judge, review MEDIUM+	50.3%	7.2%	64.9%
  Lowest FPR, gate only	quiet + judge, block HIGH+	33.1%	5.5%	48.5%

• New low-noise and quiet presets (#230). They demote the rules that most often flag harmless real skills, and cap the judge's low-confidence findings (low_confidence_max_severity) and contextual-risk findings (contextual_risk_max_severity). quiet requires the judge.

• A stronger judge (#230). A new prompt separates a skill's purpose from misuse. Repair of a self-contradicting SAFE verdict is on by default, and --llm-decompose runs one focused pass per concern. Bedrock structured output and meta-analyzer schema wiring are fixed.

• Current default model: claude-sonnet-5-5, or bedrock/us.anthropic.claude-sonnet-5-5 on Bedrock (#241). The old default, claude-3-5-sonnet-20241022, is retired. Temperature is left out for models that reject it. Sonnet 5.5, Opus 5.5 and Fable 5.1 get plain JSON output with the schema in the prompt.

• Scoped suppressions for rules, by skill and path (#223).

• New providers:

  • Adjudicator credentials now resolve for every provider (#233).
  • Experimental on-device Apple Foundation Model support (#235).
  • Any OpenAI-compatible gateway works through SKILL_SCANNER_LLM_PROVIDER=openai-compatible.

• Detection:

  • Frontmatter description and when_to_use are scanned (#229).
  • New SUPPLY_CHAIN_REGISTRY_REDIRECT rule (HIGH) and UNDECLARED_NETWORK_DESTINATION rule (LOW) (#236).
  • OSV reads package.json (#228).
  • The judge reads bounded excerpts of oversized files instead of dropping them (#231).

• Pre-commit can run the judge: set use_llm, llm_model and llm_provider in .skill_scannerrc (#241).

• Reusable workflow (#241): new scanner_version, llm_provider, llm_base_url and aws_region inputs. A bedrock/ model installs the [bedrock] extra, and configuration errors are reported separately from findings.

• Docs (#240):

  • grouped website navigation
  • new LLM Providers and Results & Tuning pages
  • every page updated to list all five presets and current model IDs

Behavior changes

• --use-llm, --use-behavioral and --enable-meta fail closed. If the requested analyzer can't be built (no key, missing extra, unknown provider), the scan exits 2 with the fix in the message, and the REST API returns 400. Previously the scan ran rules only and exited 0.
• Exit codes: 2 means a usage or configuration error, such as an unknown policy (previously 1). 1 still means findings at or above --fail-on-severity.
• Verdict repair is on by default. A SAFE verdict that lists findings becomes SUSPICIOUS and keeps the findings. To restore the strict path, set SKILL_SCANNER_LLM_REPAIR_INCONSISTENT_VERDICT=0.
• Severity changes:
  • Unpinned dependencies are reported at LOW instead of MEDIUM.
  • FILE_MAGIC_MISMATCH can be LOW when only a text label disagrees with the extension.
  • Several false-positive fixes landed in the static and YARA rules (#230).
• Frontmatter description and when_to_use are now scanned by the core rules, YARA and the active-directive rules. Opt-in packs still scan the body only.
• The wizard no longer turns on the meta-analyzer by default. With the meta-analyzer working, it cost 16.4 points of recall.

Fixes

• #234: scan-all --recursive --lenient no longer splits a skill's Markdown subfolders into separate skills (#238).
• #232: --adjudicate resolves credentials for every provider (#233).
• #220: oversized code files and SKILL.md bodies get bounded excerpts instead of being skipped (#231).
• #225: the live C2 IP in the ATR description text is defanged.
• A malformed meta-analysis priority_order no longer fails the batch (#224).
• upload-sarif has actions: read (#221).
• Dependency floors were raised to clear pip-audit advisories (#239).

Measured figures and methodology: https://huggingface.co/spaces/Vineethsain/skill-scanner-vs-skillspector and Measured results.

Full Changelog: 2.1.0...2.2.0

What's Changed

• feat(scanner)!: modernize core detection by @vineethsai7 in #210
• fix(static): resolve named Python shell flags by @vineethsai7 in #212

Full Changelog: 2.0.14...2.1.0

What's Changed

• fix release SBOM attestation by @vineethsai7 in #156
• docs: improve Skill Scanner search descriptions by @vineethsai7 in #157
• docs(packs/atr): correct the benchmark figures and their caveats by @eeee2345 in #159
• fix(deps): allow Textual 8 by @vineethsai7 in #171
• fix(ci): upgrade pip past PYSEC-2026-3721 by @vineethsai7 in #170
• style(tests): format Textual compatibility coverage by @vineethsai7 in #177
• fix(scanner): block untrusted pickle files by @vineethsai7 in #192
• fix(scanner): detect semantic data exfiltration APIs by @vineethsai7 in #199

Full Changelog: 2.0.13...2.0.14

What's Changed

• fix(scanner): discover symlinked skill dirs in recursive scans by @rohitgollarahalli in #128
• fix(behavioral): converge dataflow fixpoint on large files (#120) by @ihabler in #130
• fix: suppress warning for recovered fenced JSON by @mgalore in #148
• fix(llm): allow Vertex AI to use ambient Application Default Credentials by @gyrospectre in #144
• chore(packs/atr): refresh ATR pack 2.0.12 -> 3.5.6 (314 -> 712 supported rules) by @eeee2345 in #142
• fix(rules): detect current ngrok tunnel domains and add bore.pub/serveo.net/localtunnel.me by @federicoroncallo-hub in #132
• feat(static): classify suspicious URLs in configuration files by @federicoroncallo-hub in #134
• feat: #140 add trusted_reference_domains to LLM analysis policy by @ojacques in #141
• feat(adjudicator): demote literal-regex false positives on deterministic HIGH+ findings (closes #138) by @rcha0s in #139
• feat(scanner): expose LLM token usage in ScanResult JSON output by @amber-beasley-liatrio in #146
• feat(pre-commit): support incremental CI skill scans by @AlexBybye in #145
• feat(release): attest build provenance and SBOM for release artifacts by @ob-aion in #143
• feat(static): flag unpinned dependencies across manifest formats by @federicoroncallo-hub in #133
• feat: add OSV.dev dependency vulnerability analyzer (--use-osv) by @federicoroncallo-hub in #135
• fix(scanner): include adjudicator token usage by @vineethsai7 in #151
• fix(llm): stabilize consensus finding severity by @vineethsai7 in #152
• fix(meta): batch findings and handle truncated responses by @vineethsai7 in #153
• chore(deps): upgrade cryptography to 50.0.0 by @vineethsai7 in #155
• docs: own website MDX and validate changes by @vineethsai7 in #154

New Contributors

• @mgalore made their first contribution in #148
• @gyrospectre made their first contribution in #144
• @federicoroncallo-hub made their first contribution in #132
• @ojacques made their first contribution in #141
• @amber-beasley-liatrio made their first contribution in #146
• @AlexBybye made their first contribution in #145
• @ob-aion made their first contribution in #143

Full Changelog: 2.0.12...2.0.13

What's Changed

• fix: add minimax to unsupported_json_schema_providers by @openclaw1reaperhammer in #115
• fix: count JavaScript/TypeScript as analyzable in analyzability scoring by @colbybr in #121
• fix: handle space-separated strings in SkillManifest.allowed_tools by @stephanie-dave in #105
• Fix/issue 114 parser error colon by @prohv in #117
• fix(api): return validation errors for malformed skills by @sriaradhyula in #112
• fix(sarif): emit scan-root-relative artifact URIs by @rohitgs28 in #118
• fix: avoid vulnerable LiteLLM versions by @DougButdorf in #113
• feat: detect ASCII smuggling via Unicode Tag Block (closes #92) by @Rahul-s-007 in #94
• feat: add jailbreak detection YARA rule by @dhruvja in #17
• feat: Extending Scanner to Pass Repo Name by @Srajangpt1 in #90
• feat(llm): support OpenAI user request field by @tdeckers in #108
• feat(llm): allow temperature override + omit for newer Claude/o1 models by @rcha0s in #122
• [codex] fix main CI audit and formatting failures by @vineethsai7 in #127

New Contributors

• @openclaw1reaperhammer made their first contribution in #115
• @colbybr made their first contribution in #121
• @stephanie-dave made their first contribution in #105
• @prohv made their first contribution in #117
• @sriaradhyula made their first contribution in #112
• @rohitgs28 made their first contribution in #118
• @DougButdorf made their first contribution in #113
• @Rahul-s-007 made their first contribution in #94
• @dhruvja made their first contribution in #17
• @Srajangpt1 made their first contribution in #90
• @tdeckers made their first contribution in #108
• @rcha0s made their first contribution in #122

Full Changelog: 2.0.11...2.0.12