Skip to content
Skill+DEV
🆕 New skill: Pstack Skill! Lauren Tan's rigorous engineering orchestrator — 23 playbooks, 21 procedures, and 21 principles in a single skill for any agent.View skill →
🧠

TypeSafe Jev — Calibrated Assessments for Agent Skills

Calibrated probability assessments for subjective criteria in Agent Skills
View Jev SkillsWhat is Jev?

What is TypeSafe Jev?

Section titled “What is TypeSafe Jev?”

TypeSafe Jev is a judgment engine that provides calibrated probability assessments for subjective criteria. Instead of relying solely on deterministic heuristics, Jev enables skills to make nuanced judgments about quality, severity, and classification.

Two Types of Assessment

Section titled “Two Types of Assessment”

Score (0.0 — 1.0)

Calibrated probability for criteria where gradations matter: pattern severity, documentation quality, API elegance, rewrite confidence.

Noul (Classification)

Categorical classification for binary or few-class decisions: pass/fail, present/missing, structural/cosmetic, suspected/not_suspected.

Why Use Jev in Skills?

Section titled “Why Use Jev in Skills?”
Without Jev With Jev
Binary heuristics (regex, pattern matching) Calibrated probability assessments
“Pattern present” or “pattern absent” “Pattern present with severity 0.73”
Fixed classification Classification with confidence level
Frequent false positives on edge cases Nuanced judgments on edge cases

Skills with Jev

Section titled “Skills with Jev”

Four skills in the catalog support optional TypeSafe Jev integration:

Humanizar

8 Score + 12 Noul — AI pattern severity, voice adequacy, rewrite confidence, change classification (structural/cosmetic/hybrid). Brazilian Portuguese.

View skill →

Human-AI

8 Score + 12 Noul — Same structure as Humanizar, calibrated for English writing patterns and 7 different voice presets.

View skill →

Slop Eval

8 Score + 16 Noul — Design tell severity, per-axis quality (color, typography, layout, motion), section classification (CLEAN/SUSPICIOUS/INFLATED/CRITICAL).

View skill →

Agent Plugin Eval

7 Score + 19 Noul — UX coherence, documentation clarity, API elegance, gate classification, secret detection, quality checklist.

View skill →

How It Works

Section titled “How It Works”

Discovery Protocol

Section titled “Discovery Protocol”

Each skill checks Jev availability before using it:

from typesafe import jev_available


if jev_available():
    from typesafe import Score, Noul
    # Use Jev for subjective criteria
else:
    # Fall back to heuristic scoring

Question Structure

Section titled “Question Structure”

Each skill includes a jev_questions.json file with definitions:

{
  "version": "1.0",
  "skill": "human-ai",
  "questions": {
    "score": [
      {
        "id": "pattern_severity",
        "question": "How severe is this AI writing pattern in the context?",
        "context_required": ["text_segment", "pattern_type", "surrounding_text"]
      }
    ],
    "noul": [
      {
        "id": "change_type",
        "question": "What type of change is required to fix this pattern?",
        "labels": ["structural", "cosmetic", "hybrid"]
      }
    ]
  }
}

Graceful Fallback

Section titled “Graceful Fallback”

When Jev is unavailable, skills use internal heuristic methods:

Skill Fallback without Jev
Humanizar / Human-AI Scoring based on pattern count and weight
Slop Eval Classification by tell density
Agent Plugin Eval Deterministic validation + regex for secrets

The fallback is functional but less nuanced on edge cases.

Integration Files

Section titled “Integration Files”

Each skill with Jev support includes:

File Description
scripts/jev_questions.json Score and Noul question definitions
references/jev-integration.md Integration protocol and code examples
Section in SKILL.md Documentation on when and how to use Jev

FAQ

Section titled “FAQ”

Do I need Jev installed to use the skills?

Section titled “Do I need Jev installed to use the skills?”

No. Jev is optional. All skills work without it using heuristic fallback methods.

How do I install TypeSafe Jev?

Section titled “How do I install TypeSafe Jev?”

See the official TypeSafe documentation for installation and configuration instructions.

Can I add Jev support to my own skill?

Section titled “Can I add Jev support to my own skill?”

Yes! Use the discovery pattern (jev_available()) and create a jev_questions.json following the documented structure. See the existing skills’ jev-integration.md files as reference.

Why do only 4 skills have Jev support?

Section titled “Why do only 4 skills have Jev support?”

Jev is useful for subjective criteria where gradations matter. Skills with purely deterministic evaluations (JSON validation, file existence, schema conformance) don’t benefit from Jev.

References

Section titled “References”