LintLang: full technical reference #

LintLang performs local, deterministic static analysis of recognized agent instructions, tool/configuration structures, and extractable Python prompt patterns. It makes no model, telemetry, or network calls during a scan. Package installation, GitHub uploads, and host-provider activity are separate operations.

Use the README for onboarding, integration guide for installation routes, and research lineage for provenance. This reference describes the software contract, not a runtime-safety guarantee.

Install #

Requires Python 3.10+. The sole runtime dependency is pyyaml>=6.0.3.

uvx lintlang scan AGENTS.md

For an isolated persistent command:

pipx install lintlang
lintlang scan AGENTS.md

If pipx's app directory is missing from PATH, run pipx ensurepath, open a new shell, and retry. To install into the current Python environment:

pip install lintlang

Installers can download packages. Once dependencies are available, the scanner operates offline. Host-specific tested pins belong in their integration guides; they are not automatically interchangeable with the newest package release.

CLI commands #

lintlang --version
lintlang --help
lintlang scan --help
lintlang scan AGENTS.md
lintlang scan config.yaml prompts/
lintlang scan config.yaml --format json
lintlang scan config.yaml --format sarif > lintlang.sarif
lintlang scan AGENTS.md --format gitlab > gl-code-quality-report.json
lintlang scan config.yaml --fail-on fail
lintlang scan config.yaml --fail-on review
lintlang scan config.yaml --min-severity high
lintlang scan config.yaml --patterns H1 H3
lintlang scan prompts/ --exclude 'archive/**'
lintlang scan --discover
lintlang scan AGENTS.md --discover subdir/
lintlang scan config.yaml --allow-empty
printf 'Do not stop until done.' | lintlang scan - --stdin-filename AGENTS.md
lintlang patterns
lintlang init --github --path AGENTS.md

--discover [ROOT] (default .) additionally scans every recognized agent-instruction file found anywhere under ROOT: AGENTS.md, CLAUDE.md, GEMINI.md, SKILL.md, agent.yaml/.yml/.json, .github/copilot-instructions.md, and *.instructions.md under .github/instructions/, matched case-sensitively. A plain Markdown file kept in that directory is not an instruction surface. Editor and host layouts are not discovery targets — .cursor/rules, .claude/agents and .windsurfrules are known omissions; pass such a file as an explicit argument. Explicit file arguments stay canonical; --discover unions its discovered set with them, deduplicated. Symlinks are not followed, as in a directory scan; a recognized instruction file skipped for that reason is named on stderr, because an unscanned instruction file is a coverage gap rather than a clean result. --exclude globs and a repository's .lintlangignore filter discovered files exactly as they filter a directory scan, and an excluded symlink is not reported. It does not change generic directory scanning (lintlang scan <dir>), which keeps its own broader, extension-based sweep. Because ROOT is optional, lintlang scan --discover FILE reads FILE as the discovery root; write lintlang scan FILE --discover instead.

- scans exactly one document from standard input and requires --stdin-filename <virtual-path>, which selects the parser (including .py AST extraction) and supplies the identity used in locations, JSON/SARIF output, and baseline matching; the virtual path is never opened. More than one -, a - without --stdin-filename, --stdin-filename without a - input, and - combined with --discover are all rejected with the usage-error exit (2). One invocation takes one unambiguous source of files.

--allow-empty exits 0 when the scan inspected zero files; without it, that case is an input/coverage error (see below).

Scan arguments are files or directories, resolved from the invocation directory. --patterns/-p selects H1-H7 families, not individual sub-codes such as H1.6. lintlang patterns lists those structural families. P1/P2 are Python-specific checks; they still run in Python extraction mode when an H-series filter is set.

--min-severity accepts info (default), low, medium, high, or critical. It filters structural findings before verdict computation and baseline matching. --format/-f accepts terminal (default), markdown, json, sarif, or gitlab. --no-suggestions hides suggestions in terminal, Markdown, SARIF, and GitLab reports; it does not remove the JSON suggestion field or redact finding evidence. --fail-under is a separate legacy HERM quality-score gate; prefer --fail-on for structural findings. None of these flags changes what arbitrary prose means.

The GitHub initializer creates .github/workflows/lintlang.yml for one existing repository-contained target. It searches for the nearest Git root, interprets --path from that root, leaves identical content unchanged, and refuses different existing content without --force. Automatic candidate order, the released Action pin, and upload permissions are documented in GitHub CI. The initializer does not scan, commit, publish, or enable repository features.

Verdicts and exit behavior #

The CLI has no verdict-failure threshold by default; a scannable input can report FAIL and still exit 0. --fail-on fail exits 1 for HIGH/CRITICAL findings; --fail-on review exits 1 for MEDIUM or higher. The first-party GitHub Action instead defaults its fail-on input to fail and always passes that threshold.

Input errors remain exit 1 regardless of severity filtering, a baseline, or the presence of another valid input. Baseline validation/write errors and SARIF location/output errors are also nonzero. Invalid CLI arguments use argparse's nonzero usage-error exit (2). A positive --fail-under threshold can independently produce exit 1 when the HERM score is below it; baselines do not change that score.

A scan that inspects zero files is an input/coverage error: it exits 1, with a matching ERROR result on every output channel. JSON output is a one-entry array with "verdict": "ERROR"; SARIF carries the corresponding error tool result, and executionSuccessful: false to match the exit status. --allow-empty exits 0 instead (a stderr note, [] for JSON, and an empty SARIF run reporting executionSuccessful: true) for a caller that intentionally scans an input that may sometimes be empty; that is different from an uninspectable requested input and does not prove that the intended instructions were scanned. --write-baseline keeps its own stricter, pre-existing behavior: zero scanned files is always an error and writes nothing, --allow-empty or not.

Every result states what it inspected: the terminal and Markdown reports print an Inspected: line ("12 tools (11 described, 12 with a schema)", "skill front matter, instruction text (418 lines)"), and JSON carries the same counts under inspected. A verdict covers that content and nothing else.

A file in which nothing was inspected is SKIPPED, with the reason (for example "this is a JSON Schema document, not agent-facing content", or "no embedded prompt literals or threshold assignments were found in this Python file"). SKIPPED is never PASS and does not by itself change the exit status, so a wrapper may hand LintLang a package.json beside an AGENTS.md. Two cases are input errors (exit 1), because a scan the author believes covers a file must not look clean:

• a scan in which EVERY file was SKIPPED (the same rule as a zero-file scan);
• a named file that holds tool-like objects (a name with a description) none of which could be inspected as tools. The error names them by path.

--allow-uninspected reports both as SKIPPED instead. Files met while walking a directory are never errors for this reason. In a multi-file terminal scan, SKIPPED files are listed once at the end and clean files appear only in the summary table. Terminal output shows five findings per finding code and counts the rest; --show-all, JSON and SARIF carry every finding.

PASS applies only to the inspected content and selected checks. It does not mean that prose is true or that a model will behave correctly. Determinism describes findings and verdicts for the same inputs and rules; terminal elapsed-time summaries are not a byte-identical-output promise.

Supported formats and extraction #

For YAML/JSON objects, the first string among system_prompt, system, systemPrompt, instructions, and prompt supplies the system prompt.

Tools are recognized by shape, not by file name or an allowlist of keys:

• STRONG: a mapping with a string name and a parameter schema under inputSchema, input_schema, parameters, parametersJsonSchema or parameters_json_schema, or an OpenAI {"type": "function", "function": {...}} / {"type": "custom", ...} wrapper. Read wherever it sits: a root array, a vendor key such as contributes.languageModelTools, result.tools, any depth. parameters only counts when it is empty or carries type, properties or $schema, which keeps CI pipeline templates out.
• Under a tools, functions or functionDeclarations key, a mapping with a string name is a tool even without a schema (at depth it also needs a description key). A tools MAP is read as name-keyed tools when its values carry a schema or a description. An unknown-key map is read as tools only when it has at least two entries and every one carries a schema.
• mcpServers.<server>.tools and servers[].tools are read and the tools are attributed to their server. Launch-only server entries (command, args, url) define no tool language and are not units.
• The model-facing description is the first non-empty string among modelDescription, model_description, description_for_model, description.
• The same tool listed twice in one file (a short declaration and a full tools/list response) is read once, keeping the copy with a schema.
• H1.4/H1.5/H1.6 compare tools within one container, so two MCP servers may each expose a search.

A document that is itself one STRONG tool object (a per-tool snapshot file) is read as one tool. JSON with // or /* */ comments and trailing commas is accepted, and YAML application tags (!Ref, !!python/name:) are read as plain data; nothing is constructed or executed.

Prompts under nested keys are read too: a string of at least 40 characters under a key such as system_template, instance_template, system_prompt, instructions, *_prompt, persona or backstory, at any depth. They are joined and checked as templates: evidence-bearing checks (H2, H4 phrases, H5 qualifiers, H6 conflicts) run; the chat-prompt shape heuristics do not, because several templates are not one prompt.

In 0.8.0, tool findings use parser-backed source regions for the relevant tool or schema construct where available. YAML aliases can retain their anchor's source mark; transformed values can anchor to the enclosing construct. Exact token-level locations are not guaranteed.

Never read as tools: dependencies-style maps, JSON Schema properties/$defs, components/packages lists, and whole documents recognized as JSON Schema ($schema with properties/$defs/definitions), OpenAPI, a package lockfile, an SBOM (CycloneDX/SPDX) or an in-toto attestation. A bare {name, description} object outside a tools container is not claimed (it is as likely a package or a server); when a file has no tools at all, such objects are reported as not inspected. Known limits: a custom container key whose members have no parameter schema is not read; tool definitions built in code are not read; YAML anchors are resolved by the YAML parser, but templated YAML is not.

Messages come from a top-level messages list; its first system message can supply the prompt when one was not otherwise found.

Recognized schema fields are response_format, output_schema, schema, and schemas; constraint mappings come from constraints, config, settings, and parameters. These are extraction rules, not comprehensive provider-schema validation.

An explicitly named file with an unknown extension is tried as JSON, then YAML, then text. Directory discovery uses the listed, case-sensitive extensions only. Directories are traversed in sorted order; dependency/cache/build trees, issue/pull-request template directories and conventional non-prompt names such as README, CHANGELOG, LICENSE (and other licence, NOTICE and third-party notice files), CONTRIBUTING, and SECURITY are skipped. Python test code (tests/, test_*.py, *_test.py, conftest.py) is returned as SKIPPED during a walk: it holds fixtures, not what an agent is given, and its explicit result keeps that exclusion visible. A .txt file met during a walk is scanned only if it contains prompt language; name it explicitly to scan it regardless. Front matter is read as skill metadata only when it has a description, or a name in a SKILL.md or under a skills/agents/commands directory, so GitHub issue templates are not skills. Symlink files and directories are not followed during directory discovery. Explicit file scans do not use these directory exclusions. Traversal errors remain input errors rather than disappearing as clean results.

.lintlangignore is read at the root of each directory passed to the scanner. Its limited glob matching and --exclude operate on paths relative to that scan directory; blank/comment lines are ignored. They are not a promise of every Git-ignore feature, nested ignore inheritance, or negation semantics. Use a known explicit input and inspect the report when scan coverage matters.

Python extraction and P1/P2 #

Python source is parsed, not imported or executed. The extractor recognizes string literals of at least 50 characters with a supported prompt signal. F-string literal portions are retained with {...} placeholders for expressions; runtime interpolation and data flow are not evaluated. Prompt candidates are deduplicated by their first 200 characters, so distinct strings with the same prefix can collapse to one candidate. Python syntax errors produce ERROR.

Only H2, H4, H5, and H6 run on extracted prompts. Literal Python tool definitions provide a separate declared surface on which H1 and H3 run; H7 does not run in Python extraction mode. Selecting a family for which a Python input provides no supported surface is not proof that the check passed. P1/P2 still run independently of that selection.

• P1: Uncalibrated Threshold. Inspects supported nonzero numeric assignments to threshold/confidence-like names, excluding recognized counters. No recognized nearby calibration comment produces MEDIUM; a recognized comment containing uncertainty markers can produce LOW. This tests a documented justification signal, not whether the threshold was actually calibrated or is correct.
• A string is extracted as a prompt when the code uses it as one: it is bound to a prompt-like name, keyword argument or message-dict key (prompt, system, instructions, template, content, ...), or it addresses a model ("You are", "your task"), or it matches three or more prompt signals. Docstrings, bare string statements, and text passed to logging, argparse/click help, or an exception are never prompts.
• Tool definitions written as literals are read: a call or dict literal with a literal name, a schema keyword (inputSchema, input_schema, parameters, parameters_json_schema, args_schema) and a literal or absent description — the way Python MCP servers declare Tool(name=..., description=..., inputSchema=...). H1 and H3 run on them, with the call's line. Tools declared by decorator and docstring, or with a computed description, are not read. A dynamic or non-object schema expression is not evaluated: the literal name and description remain inspected, while the schema is excluded from tools_with_schema and named under not_inspected.
• P2: Embedded Scaffold. An extracted prompt longer than 500 characters produces LOW; one longer than 200 and at most 500 produces INFO. It suggests reviewing whether the prompt should be externalized, not that embedded prompts necessarily cause runtime failures.

A Python file with no recognized prompt may still have P1 findings. A clean scan cannot establish that dynamically assembled prompts or every pipeline were inspected. Python decoding currently ignores invalid UTF-8 bytes; ordinary text/configuration loading uses strict UTF-8. Neither path executes source code.

Structural detectors (H1-H7) #

H1: Tool Description Ambiguity #

Checks empty and underspecified short descriptions, selected vague opening verbs, duplicate tool names, high word overlap, and relational nondistinction. Findings keep pattern_id: H1; code provides the more specific stable identifier.

One-sided H1.6 containment and non-identical H1.5 overlap are suppressed when both tools declare different input names, types, or choices in nonempty property schemas: those inputs provide a selection distinction. Descriptions that are nearly identical (95% or greater word overlap) still receive H1.5. H3 does not demand duplicate descriptions for scalar parameters explained by an enum, const, format, a named boolean switch, or the tool description. Ambiguous parameters such as an unconstrained role or line/column coordinates still need semantics. These are bounded heuristics, not proof of semantic completeness.

Server manifests with server, tools, and root instructions retain their instruction text for evidence-bearing checks, but do not inherit host-agent requirements for a retry budget, output format, version, or priority ordering. Percent-delimited localization references are not counted as inspected tool prose; not_inspected names unresolved tool and schema descriptions. No sibling files or remote localization resources are fetched. Skill source catalogs with repo and skillPath are explicitly skipped; their bodies are not present.

For a Markdown file with name/description front matter (a skill or sub-agent definition) the description is the selection-time text, so H1.1 (no description, HIGH) and H1.2 (under 20 characters, MEDIUM) apply to it, located at frontmatter.description with its line. H1.8 looks for trigger vocabulary ("use when", "if the user", "before", a description written as the situation); it is a finite vocabulary, which is why a long description is only LOW. H1.9's directory comparison runs only for a file named SKILL.md.

H1.3's vague verbs are handle, process, manage, do, perform, deal, work, and it reports only a description under 60 characters: a vague opener followed by the specifics has said what the tool does. get, set, run, execute, use and make were removed: on real MCP manifests every hit on them was a precise description ("Get the current time in a timezone"). One-sided H1.6 (domination) additionally requires that the two tools share a domain term, not only a generic verb class, and open with the same action.

H1.5 uses Jaccard word similarity with stopword removal. Differently named tools whose descriptions each carry a term the other lacks are a parallel family ("List code scanning alerts" / "List secret scanning alerts", add / remove) and are not reported below 95% overlap; nor is a pair in which a description states the selection rule ("Prefer this tool over X"). H1.6 compares analyzed terms from tool names and descriptions under a finite synonym lexicon. It can reach some pairs missed by word overlap: "Look up an order" and "Search for orders in the system" have Jaccard 0.00 under H1.5 but no differentia under H1.6's term model. Mutual nondistinction means neither member distinguishes itself; directional domination identifies the less-specific member when one contributes no terms beyond the other. Explicit named boundaries and distinguishing tool names can exempt a pair; declared aliases are not automatically exempted.

The comparison scope is one parsed input. Directory scans do not aggregate tools across files or infer a shared selection namespace. H1.6 is MEDIUM and cannot alone trip --fail-on fail; use --fail-on review to gate MEDIUM findings. Its lexicon is finite: pairs outside it, such as kill/terminate or approve/authorize, are not detected. External labeled-corpus precision and recall have not been measured. The absence of a finding is not semantic or runtime validation.

Alias recognition is also phrase-bound. Compatibility alias for X, Deprecated. Use X, and Superseded by X are recognized. Paraphrases such as "does the same thing as X, kept for backward compatibility" or "older entry point, prefer X in new code" are not equivalent coverage promises. See Tool Differentia and research lineage for provenance.

H2: Missing Constraint Scaffolding #

Checks for selected missing termination conditions, unbounded retry language such as "keep trying until", negative termination such as "don't stop until", "continue until" without limits, "retrying / try again / repeat until", and missing retry budgets. A match is not reported when the same sentence states a bound (max_iterations, "at most", "up to 3", "5 attempts", a timeout) or when "loop" is a noun being described ("block the agent loop until answered"). These are static indicators, not a proof that a loop will or will not terminate.

H3: Schema-Intent Mismatch #

Checks phantom required fields absent from properties, undescribed parameters, generic parameter names such as data, input, value, and payload, undescribed anyOf/oneOf variants when at least one undescribed variant is structural (an object, array or $ref; a union of scalar types explains itself), and nested objects without descriptions. Boolean property schemas are skipped. It is not full JSON Schema validation or an oracle for a tool's implementation.

H4: Context Boundary Erosion #

Checks selected unscoped requests to remember everything or use all conversation, history, or context; unbounded always-remember language; and, for chat prompts only, long prompts without recognized boundary markers. It does not inspect a host's actual context window.

H4.5 (MEDIUM; only AGENTS.md, CLAUDE.md, GEMINI.md, SKILL.md, Copilot instruction files and files with skill front matter, read from disk) reports a referenced project file that does not exist. It needs all of: a literal relative path with a directory part and a known file extension, in backticks or a Markdown link, outside fenced code; a first segment that exists beside the document or at the repository root; a path that resolves from neither; and a line that does not talk about creating, renaming, removing or exemplifying it. It reads the filesystem, so it does not run on standard input, and it is deliberately quiet: directories, globs, placeholders and paths into other projects are never reported.

H5: Implicit Instruction Failure #

Checks selected vague qualifiers ("be concise", "be helpful", "use common sense"), ambiguous conditionals ("as needed", "when appropriate"), figurative verbs ("lean into", "err on the side of", "keep it simple"). The instruction-count finding is LOW for a literal extracted from Python. In a Markdown document, H2 does not report "loop / repeat / continue until " (a stated termination condition) or text inside a quoted example. For chat prompts only (.txt, .prompt, a config's system prompt, an extracted Python literal) it also checks negative-instruction density and high instruction count without priority ordering; those two judge the shape of a single prompt and do not run on Markdown instruction documents, where they fired on most real files and named no sentence. Negative-instruction exemptions have three layers:

1. Structural: HTML comments, fenced/inline code, and generated-file markers.
2. Phrase-level: selected privacy disclaimers, UI labels, descriptive wording, idiomatic phrases, and "to avoid" constructions.
3. Safety context: nearby security/authentication/policy terms within a 100-character window.

These exemptions and matches are implemented patterns, not unrestricted intent understanding; a useful false-positive report includes a minimal reproduction.

H6: Template Format Contract Violation #

Strips fenced code, inline code, filenames, and CLI flags before counting format keywords. Checks selected conflicting output formats without disambiguation, template variables and, for chat prompts only, a missing format specification and (not for extracted Python literals) a missing version marker. It does not prove arbitrary output contracts satisfiable.

H7: Role Confusion #

Checks multiple system messages, a system message not at position zero, consecutive same-role messages, orphan tool results without preceding tool use, and messages missing a role. Coverage is limited to recognized message arrays, not every provider's protocol.

Programmatic API #

from lintlang import compute_verdict, scan_directory, scan_file

result = scan_file("config.yaml")
print(compute_verdict(result))  # ERROR, SKIPPED, PASS, REVIEW, or FAIL
if result.input_error is not None:
    print(result.input_error)
else:
    for finding in result.structural_findings:
        print(f"[{finding.severity.value}] {finding.code}: {finding.description}")
        print(f"  -> {finding.suggestion}")

results = scan_directory("prompts/", patterns=["H2", "H4"])
for path, scanned in results.items():
    print(path, compute_verdict(scanned))

scan_file(path, patterns=None) returns a ScanResult with file, score, herm, structural_findings, input_error, inspected, notes, and skipped. Load/parse failures use the input_error channel; inputs with no recognized agent-facing content return SKIPPED, never PASS. scan_directory(directory, patterns=None, extensions=(".yaml", ".yml", ".json", ".txt", ".md", ".prompt", ".py"), exclude=None) returns a path-keyed result dictionary. Directory failures remain represented in that dictionary. Severity gates are CLI policy, not exceptions raised by compute_verdict.

Findings expose pattern_id, code, pattern_name, severity, location, description, suggestion, and evidence; where available, source_region provides source spans. Use finding.code, not a nonexistent finding.pattern attribute. HERM dimensional scores are separate from structural findings and verdicts. They are not a validated probability of correctness or safety.

JSON and SARIF #

--format json writes an array, including for a single input. Each entry has file, verdict, input_error, inspected (counts of what the verdict covers, such as tools, tools_described, tools_with_schema, instructions, system_prompt, messages, python_prompts), not_inspected (coverage notices), skipped (the reason, or null), structural_findings, and herm. Each finding contains pattern_id, the specific code, pattern_name, lowercase severity, location, line (the start of an evidence-backed source region for text or structured inputs, else null), description, suggestion, and evidence. HERM contains score, dimensions, signal_counts, coverage, confidence, findings, and context_flags; it is null for input errors. Applying a baseline adds baseline: {"suppressed": N} to each scanned result. Operational baseline errors are ERROR entries, not a successful empty scan.

--format sarif writes SARIF 2.1.0. Rule identifiers preserve diagnostic codes; HIGH/CRITICAL map to error, MEDIUM to warning, and LOW/INFO to note. Reporting uses the nearest Git root, or the invocation directory outside Git. Paths are repository-relative and URI-encoded; outside-root locations are rejected. Source regions are included only where the parser/extractor can justify them. LintLang does not invent line numbers for every logical YAML/JSON location.

Input errors are invocation notifications rather than ordinary lint results. Reports do not include absolute source paths, source snippets, or custom fingerprints. Descriptions and suggestions can still contain source-derived text; SARIF is not a guarantee that diagnostics contain no sensitive information. Treat JSON and ordinary scan output as potentially sensitive too.

--format gitlab writes a single GitLab Code Quality JSON array. Each structural finding with an evidence-backed source line has a repository-relative path, integer beginning line, specific check code, stable fingerprint, description, and GitLab severity. Prompt text offsets map to physical lines when safe; whole-prompt and structured YAML/JSON findings anchor to the actual prompt, scalar, message, schema, or tool construct. An unexpected finding with no supported source line cannot satisfy GitLab's required line field: the report omits it, reports the count on standard error, and exits nonzero even in advisory mode. Input, baseline, and outside-root path failures also remain nonzero while stdout stays a valid array. The GitLab guide provides a copyable CI job and complete severity mapping.

Use a report path different from every input and baseline. Shell redirection can truncate an input before the CLI starts; the CLI cannot undo that. The Action's sarif-file route guards input/baseline aliases and writes via a temporary file. Report generation does not perform an upload. See GitHub Code Scanning for the sole maintained full workflow, artifact handoff, least-privilege permissions, and fork restrictions.

Baselines #

--write-baseline PATH and --baseline PATH are mutually exclusive. The former records a reviewed starting inventory to a new file; the latter applies an existing inventory after rule/severity selection. Follow baseline adoption for review, CI setup, and safe refresh.

The lintlang/baseline-v1 document contains exactly schema, generator, and entries. Each entry has exactly a canonical relative POSIX path, lowercase 64-character SHA-256 fingerprint, and positive integer count. Boolean counts, unknown fields/versions, duplicate JSON keys or entries, noncanonical paths, and invalid hashes are errors. generator records a nonempty version string; it is not a demand that every reader use that exact package version.

The fingerprint hashes canonical JSON of code, severity, location, description, and evidence (sorted keys, compact separators, ASCII escaping). Python source prefixes in diagnostic locations are normalized to repository- relative paths; encoded line spans remain identity. Suggestions, display names, and separate source-region metadata are not identity. Matching is exact and count-limited, not a wildcard or rule-wide suppression. Extra occurrences and changed identities remain visible. Aliases cannot multiply allowances.

All source paths must resolve inside the baseline root. Creation rejects any input error or an empty scan; one inspected file with zero findings is allowed. Publishing is atomic and never replaces an existing destination, including a dangling symlink. The parent directory must exist. Applying a baseline preserves input errors, HERM scores, and the legacy quality gate. JSON records suppressed counts; SARIF's run properties include lintlangBaseline with suppressed and verdictScope: "remaining findings".

Paths and hashes are stored, not raw prompt/evidence text. Hashing is not encryption. Unused entries are allowed; an identical reintroduced finding can match an old entry until the baseline is deliberately pruned. Baselines record acknowledged identities, not the history of when defects were fixed.

Preflight #

Preflight is separate from repository scanning. It inspects one present UTF-8 instruction and explicit caller-supplied context; it does not retrieve history, call a provider, silently edit a file, or send an instruction.

printf '%s' 'Is it true that X?' | lintlang preflight - --format json
lintlang preflight --help

from lintlang import PreflightRequest, preflight_text

result = preflight_text(PreflightRequest(prompt="Is it true that X?"))
print(result.status.value)  # NOTICE
print(result.to_json())     # raw prompt/context/patch text redacted by default

PF001 validation-seeking framing, PF002 presupposed causality, and PF003 unresolved context-reference heuristics are notice-only. PF004 can hold for an exact missing required binding; a known IN_PROMPT binding can instead produce an insertion suggestion. PF005 can hold for exact typed-format or mechanical conflicts. These are input-risk labels, not evidence that a model exhibited one of the research taxonomy's output behaviors. ALLOW proves neither truth nor safety, quality, or provider compatibility.

Context uses explicit requirements, bindings, and typed constraints. IN_PROMPT means a supplied value must appear in the outgoing instruction; SIDE_CHANNEL means the host promises to deliver it separately. LintLang performs neither provider-specific delivery. Rule bundle v1 recognizes JSON and Markdown typed output formats. See the preflight guide for the context schema and runnable examples.

Default serialization omits raw prompt, context, snippets, replacement text, and diffs. --include-snippets explicitly discloses local evidence/patch previews. --apply CORRECTION_ID separately checks the source hash, applies one correction in memory, reruns preflight once with the same context and policy, and writes corrected text to stdout. It does not establish semantic preservation or modify the original file.

Required unsupported-language or unsafe/unbalanced-scope analysis yields UNAVAILABLE, not ALLOW. With no enabled rules, components are NOT_REQUIRED; enabled-but-optional unavailable analysis reports NONE coverage plus a warning. Empty, oversized, malformed, or ambiguous inputs yield ERROR. Scan verdicts and scan exit semantics remain independent of these states.

Reading a repository sample #

From a source checkout:

lintlang scan samples/bad_tool_descriptions.yaml

Excerpt from lintlang 0.8.0:

FAIL — 1 CRITICAL, 1 HIGH, 5 MEDIUM, 3 LOW
H1.1: Tool 'process_ticket' has no description

This is a deliberately failing fixture, not a broken installation. Compare it with lintlang scan samples/clean_config.yaml --fail-on fail from a source checkout, or use the checkout-free clean example. A clean static scan is not evidence of runtime correctness or safety.

First run without a checkout #

No clone or credentials are required. Write two small files and run three scans. Only the install reaches the network to obtain the package and dependencies. Every lintlang scan below is offline.

python -m pip install lintlang==0.8.0

cat > /tmp/agent.yaml <<'YAML'
system_prompt: |
  You are a support agent. Use the tools to help the user.
tools:
  - name: process_ticket
    description: ""
    parameters:
      type: object
      properties:
        ticket_id:
          type: string
YAML

lintlang scan /tmp/agent.yaml --fail-on fail

lintlang 0.8.0 reports FAIL — 1 CRITICAL, 1 HIGH, 1 MEDIUM and exits 1. H1.1 identifies the empty tool description. This is a successful detection; do not hide the exit status with || true. To use the newest package instead, run python -m pip install --upgrade lintlang and re-read the counts below as approximate.

cat > /tmp/agent-fixed.yaml <<'YAML'
system_prompt: |
  You are a support agent. Use the tools to help the user.
  Stop and report the failure once the retry limit is reached.
tools:
  - name: process_ticket
    description: "Apply a resolution action to one existing support ticket. Use this only after the ticket has been read; do NOT use it to look tickets up."
    parameters:
      type: object
      properties:
        ticket_id:
          type: string
          description: "Identifier of the existing ticket to act on"
      required: [ticket_id]
constraints:
  max_iterations: 3
  timeout_seconds: 30
YAML

lintlang scan /tmp/agent-fixed.yaml --fail-on fail

With the pinned release, the fixed file scans PASS — 0 findings and exits 0; the original /tmp/agent.yaml still scans FAIL. An absent input remains a distinct error:

lintlang scan /tmp/does-not-exist.yaml --fail-on fail

This produces ERROR, an input_error, and a nonzero exit, not a clean scan. PASS here means that selected checks found no covered defects in recognized content extracted from /tmp/agent-fixed.yaml. It does not establish correctness of tool behavior, semantic truth, runtime outcomes, or safety.

Keep a successful check in GitHub CI #

Run this from the repository root, choosing its real instruction file:

lintlang init --github --path AGENTS.md

An identical generated workflow is unchanged. A different existing workflow is left alone; review it before replacing it with --force. See GitHub CI for the generated file and verification steps.

Windows PowerShell #

Use the same inputs and outcomes with PowerShell-native temporary paths:

python -m pip install lintlang==0.8.0
$badPath = Join-Path $env:TEMP "agent.yaml"
$fixedPath = Join-Path $env:TEMP "agent-fixed.yaml"

@'
system_prompt: |
  You are a support agent. Use the tools to help the user.
tools:
  - name: process_ticket
    description: ""
    parameters:
      type: object
      properties:
        ticket_id:
          type: string
'@ | Set-Content -LiteralPath $badPath -Encoding utf8

python -m lintlang scan $badPath --fail-on fail

@'
system_prompt: |
  You are a support agent. Use the tools to help the user.
  Stop and report the failure once the retry limit is reached.
tools:
  - name: process_ticket
    description: "Apply a resolution action to one existing support ticket. Use this only after the ticket has been read; do NOT use it to look tickets up."
    parameters:
      type: object
      properties:
        ticket_id:
          type: string
          description: "Identifier of the existing ticket to act on"
      required: [ticket_id]
constraints:
  max_iterations: 3
  timeout_seconds: 30
'@ | Set-Content -LiteralPath $fixedPath -Encoding utf8

python -m lintlang scan $fixedPath --fail-on fail
python -m lintlang scan (Join-Path $env:TEMP "does-not-exist.yaml") --fail-on fail

Related documentation #

GitHub CI, baselines, integrations and public ecosystem evidence, research lineage, product intent, contributing, security policy, and Apache License 2.0. LintLang is maintained by Hermes Labs.