Evally
Start free
Engineering

Semantic code lint

Lint code against conventions written in plain English, the rules a regex can't express.

Write your team's conventions the way you'd explain them to a new hire, and check code against them in CI or in your agent's write-check-fix loop. It also catches names that lie, stale comments and swallowed errors.

Use this judgeCustomize it

Try it live

Real calls to Jev. Pick an example or paste your own.

Examples
119
206
No verdict yet

Pick an example or paste your own input, then run it.

Under the hood

Every question and rule, in the open. Fork it to change any of them.

What it asks

Reads conventions and code, then asks 5 questions in one call.

  1. Q1Follows the conventionsYes / no

    Does the code follow every rule in conventions?

  2. Q2Clear namesYes / no

    Do the function and variable names in the code say what they do?

  3. Q3Does what its name saysYes / no

    Does each function in the code do what its name says, with no hidden side effects?

  4. Q4Stale commentsYes / no

    Does the code contain a comment that no longer matches the code next to it?

  5. Q5Swallowed errorsYes / no

    Does the code swallow errors or ignore failure cases a caller would care about?

How it decides

Rules run in code, top to bottom. The first one that matches gives the verdict.

  1. 1
    If Follows the conventions < 0.35Violations
  2. 2
    If Swallowed errors ≥ 0.75Violations
  3. 3
    If Follows the conventions < 0.6Warnings
  4. 4
    If Clear names < 0.4Warnings
  5. 5
    If Does what its name says < 0.4Warnings
  6. 6
    If Stale comments ≥ 0.6Warnings
  7. 7
    If Swallowed errors ≥ 0.5Warnings
  8. ·
    DefaultClean

Call it from your code

Every template runs as-is with an API key, no setup: POST /api/v1/judges/semantic-lint/run.

curl -X POST https://tryevally.com/api/v1/judges/semantic-lint/run \  -H "Authorization: Bearer $EVALLY_API_KEY" \  -H "Content-Type: application/json" \  -d '{  "input": {    "conventions": "Never swallow errors: log them with context or rethrow. Functions that start with '\''get'\'' must not write to the database.",    "code": "async function getUser(id) {\n  try {\n    const user = await db.users.find(id);\n    await db.audit.insert({ event: '\''viewed'\'', id }); // read-only lookup\n    return user;\n  } catch (e) {\n    return null;\n  }\n}"  }}'

Seen in the wild

Community builds on Jev that use this pattern. Not affiliated with Evally.

More in Engineering