Skip to main content

Multi-layer analysis

Level Advanced
Reading time ⏱ 15 min
words 1342
Topics securitytaint

Multi-Layer Analysis — Detailed Specification

Goal

Move from a single-layer analysis (one mode per rule) to a multi-layer analysis where each file is examined by 3 independent strategies. The results are cross-referenced to produce high-confidence findings with a chain of evidence verifiable by a human.


Current architecture (single-layer)

Today, each rule uses ONE mode:

Rule sql_injection_fstring
    → regex mode: searches for ".execute(f""
    → Match? → Finding (confidence 80, fixed)
    → No match? → Nothing

Issues:

  • If the pattern is too narrow, the vulnerability is missed (false negative)
  • If the pattern is too broad, noise is detected (false positive)
  • No proof of data flow (the dev sees "SQL injection" but not WHY)
  • Confidence is fixed (80 or 95), not based on actual evidence

Proposed architecture (multi-layer)

For each file, 3 layers run in parallel:

                    ┌─────────────────┐
                    │  Source file     │
                    └────────┬────────┘
                             │
              ┌──────────────┼──────────────┐
              │              │              │
              ▼              ▼              ▼
     ┌────────────┐  ┌────────────┐  ┌────────────┐
     │  Layer 1    │  │  Layer 2    │  │  Layer 3    │
     │  Pattern    │  │  Taint      │  │  Context    │
     │  matching   │  │  analysis   │  │  analysis   │
     └──────┬─────┘  └──────┬─────┘  └──────┬─────┘
            │               │               │
            ▼               ▼               ▼
     ┌─────────────────────────────────────────┐
     │           Evidence merger                │
     │                                         │
     │  Cross-references results from 3 layers.│
     │  Computes the actual confidence.        │
     │  Builds the chain of evidence.          │
     └────────────────┬────────────────────────┘
                      │
                      ▼
              ┌───────────────┐
              │   Finding     │
              │  + evidence   │
              │  + confidence │
              └───────────────┘

Layer 1: Pattern matching (existing, improved)

What it does

Searches for syntactic patterns in the code (text or regex). This is the current layer — fast, broad coverage.

What it produces

{
  "layer": "pattern",
  "rule_key": "sql_injection_fstring",
  "file": "src/db.py",
  "line": 42,
  "code": "cursor.execute(f\"SELECT * FROM users WHERE id = {user_id}\")",
  "evidence": "Pattern '.execute(f\"' matched at line 42"
}

Confidence alone: 65%

A pattern match alone is a signal, not proof. The pattern can match safe code (e.g. a test, a comment, a documentation string).


Layer 2: Taint analysis (existing, enriched)

What it does

Traces the data flow from an untrusted source (HTTP input, CLI, file) to a dangerous sink (execute, open, system).

What it produces

{
  "layer": "taint",
  "rule_key": "taint_sqli",
  "file": "src/db.py",
  "flow": {
    "source": {
      "line": 12,
      "code": "user_id = request.args.get('id')",
      "kind": "http"
    },
    "propagation": [
      {"line": 15, "code": "user_id = int(user_id) if safe else user_id"},
      {"line": 38, "code": "query = f\"SELECT * FROM users WHERE id = {user_id}\""}
    ],
    "sink": {
      "line": 42,
      "code": "cursor.execute(query)",
      "type": "sql_execute"
    },
    "sanitizers_found": false
  },
  "evidence": "HTTP data (request.args) propagated without sanitization to cursor.execute()"
}

Confidence alone: 85%

A taint flow is strong evidence — it shows the complete path. But it can be a false positive if the sanitizer lives in a function called indirectly (outside the file).


Layer 3: Context analysis (new)

What it does

Analyzes the file context to reinforce or refute findings from layers 1 and 2. Answers questions such as:

  • Does the file use an ORM (SQLAlchemy, Django ORM)? If so, a SQL injection is less likely.

  • Does the file import a sanitization framework?

  • Is the file a test, a mock, a fixture? If so, ignore.
  • Does the file handle HTTP requests (routes, handlers)?
  • Does the file contain input validation (schemas, validators)?

What it produces

{
  "layer": "context",
  "file": "src/db.py",
  "signals": {
    "is_test_file": false,
    "has_orm": false,
    "has_raw_sql": true,
    "has_http_handler": true,
    "has_input_validation": false,
    "has_sanitizer": false,
    "framework": "flask"
  },
  "evidence": "Flask file with raw SQL, no ORM, no input validation"
}

Implementation

The context layer is a set of file-level checks:

def analyze_context(filepath, content):
    signals = {}

    # Test/mock/fixture detection
    signals["is_test_file"] = any(x in filepath.lower()
        for x in ["test_", "_test.", "mock", "fixture", "spec."])

    # ORM present?
    signals["has_orm"] = bool(re.search(
        r"(?:from sqlalchemy|from django\.db|from tortoise|"
        r"from peewee|from prisma|\.objects\.|\.filter\(|\.query\.)",
        content))

    # Raw SQL?
    signals["has_raw_sql"] = bool(re.search(
        r"(?:SELECT|INSERT|UPDATE|DELETE)\s+", content))

    # HTTP handler?
    signals["has_http_handler"] = bool(re.search(
        r"(?:@app\.route|@router\.|def\s+\w+\(request|"
        r"async\s+def\s+\w+\(request)", content))

    # Input validation?
    signals["has_input_validation"] = bool(re.search(
        r"(?:pydantic|marshmallow|cerberus|voluptuous|"
        r"wtforms|django\.forms|Schema\(|validate\()", content))

    # Sanitizer present?
    signals["has_sanitizer"] = bool(re.search(
        r"(?:bleach|escape|sanitize|parameterize|"
        r"quote\(|html\.escape|markupsafe)", content))

    return signals

Evidence merger

Confidence calculation

The final confidence is not an average — it is a calculation based on independent pieces of evidence:

Confidence = base + taint_bonus + context_bonus - penalties

Concrete cases:

  Pattern alone, no context:
    65% (weak signal, may be a false positive)

  Pattern + confirming context (raw SQL, no ORM, HTTP handler):
    65% + 15% = 80%

  Pattern + taint flow (source → sink confirmed):
    65% + 25% = 90%

  Pattern + taint + context (triple confirmation):
    65% + 25% + 10% = 95% (near certainty)

  Pattern + REFUTING context (ORM present, test file):
    65% - 30% = 35% (likely false positive → removed from report)

Merging rules

1. If the context layer detects a test file → REMOVE the finding
    (test files intentionally contain vulnerable code)

2. If the taint layer confirms the flow → INCREASE the confidence
    and ADD the chain of evidence to the finding

3. If the context layer refutes (ORM present, sanitizer found) →
    DECREASE the confidence. If < 40% → REMOVE the finding

4. If two different rules detect the same vulnerability at the same place →
    MERGE into a single finding with the highest confidence

5. If the taint flow crosses several files → ADD the "cross-file"
    label and INCREASE the confidence by 5%

Chain of evidence in the report

What the developer sees today

[HIGH] SQL Injection (f-string) — src/db.py:42
  Risk: An attacker can inject malicious SQL...
  Solution: Use parameterized queries.

No context. The developer has to figure out alone why this is a problem.

What the developer would see with multi-layer analysis

[HIGH] SQL Injection (f-string) — src/db.py:42
  Confidence: 95% (pattern + taint + context)

  Evidence:

  1. Source (line 12):
     user_id = request.args.get('id')
     → Untrusted HTTP data (kind: http)

  2. Propagation (line 38):
     query = f"SELECT * FROM users WHERE id = {user_id}"
     → Variable injected into a SQL query via f-string

  3. Sink (line 42):
     cursor.execute(query)
     → Query execution without parameterization

  4. Context:
     - Flask file with HTTP handler (@app.route)
     - Raw SQL (no ORM)
     - No input validation detected
     - No sanitizer detected

  Solution: Use parameterized queries.
  Playbook: 3 steps (diagnosis → fix → verification)
  References: CWE-89, OWASP A03:2021, ISO A.8.28

The developer can verify each step of the evidence. If they think it is a false positive, they know exactly what to check.


Impact on false positives

The context layer drastically reduces false positives:

Situation Today Multi-layer
Pattern in a test file Finding (FP) Removed (is_test_file)
Pattern in a file with ORM Finding (FP) Confidence 35% → removed
Pattern + confirmed taint Finding (80%) Finding (95%)
Pattern in a comment/string Finding (FP) Confidence 35% → removed

Estimate on the current benchmark

Metric Current Multi-layer (estimated)
Precision 81% 93%+ (context filters FPs)
Recall 99% 99% (same coverage)
F1 score 89% 96%+

Files to modify

File Action
templates/finding-item.html Modify — <details> block for the evidence

What this does NOT solve

Multi-layer analysis improves the quality of existing detections. It cannot detect:

  • Pure logic bugs (e.g. IDOR Lunary CVE-2024-1625, where the issue is a missing WHERE in a query — not a syntactic pattern)

  • Vulnerabilities in dependencies (covered by pip-audit, not by SAST)

  • Vulnerabilities in compiled/obfuscated code

For these cases, the solutions are:

  • IDOR: dedicated rule "CRUD without ownership check" (rule 4 of BountyBench)
  • Dependencies: pip-audit/npm audit scan (already implemented)
  • Compiled code: out of SAST scope