How Agents Determine Which Skills Are Useful and Which to Retire

Why file existence cannot infer skill usefulness

In a ~/.hermes/skills/ directory containing many skill folders, sorting by file timestamps and archiving items unchanged for 30 days fails for three reasons:

Creation time ≠ last active time: a skill created months ago but used daily would be archived incorrectly.

Bundled vs. agent‑created skills are indistinguishable on disk: without additional data Curator cannot know whether a skill is shipped with Hermes (which must never be removed) or created by an Agent (which Curator manages).

"Never used" vs. "just created" have identical metadata: a newly created skill and a 90‑day‑old unused skill look the same to the file system.

Therefore Hermes requires an independent telemetry layer.

Provenance system – three skill types, two defense lines

Skills are classified as:

Bundled – shipped with Hermes; marked by .bundled_manifest; Curator never manages them.

Hub‑installed – installed via Skills Hub; marked by .hub/lock.json; Curator never manages them.

Agent‑created – created via skill_manage(action="create"); Curator manages only those that contain created_by="agent" in .usage.json.

The two defense lines are the union of bundled manifest names and hub‑installed names. Any skill not present in either set is considered agent‑created. The helper implements this logic:

# tools/skill_usage.py – two‑line defense merge

def is_agent_created(skill_name: str) -> bool:
    """Return True if the skill is not bundled and not hub‑installed."""
    off_limits = _read_bundled_manifest_names() | _read_hub_installed_names()
    return skill_name not in off_limits

def _read_bundled_manifest_names() -> set:
    manifest = _skills_dir() / ".bundled_manifest"
    if not manifest.exists():
        return set()
    return {line.split(":", 1)[0].strip()
            for line in manifest.read_text().splitlines() if line.strip()}

Even for agent‑created skills, Curator only manages those explicitly marked. The opt‑in check and marker are:

# tools/skill_usage.py – opt‑in check and marker

def _is_curator_managed_record(record: any) -> bool:
    if not isinstance(record, dict):
        return False
    return record.get("created_by") == "agent" or record.get("agent_created") is True

def mark_agent_created(skill_name: str) -> None:
    """Automatically called by skill_manage(action='create') to mark the skill for Curator."""
    def _apply(rec: dict) -> None:
        rec["created_by"] = "agent"
    _mutate(skill_name, _apply)

Usage telemetry – sidecar design with three counters

All usage data are stored in a sidecar file .usage.json rather than in the skill's frontmatter. This avoids contaminating user content, prevents git‑pull conflicts for bundled skills, and enables atomic reads/writes.

The usage record has the following fields:

# tools/skill_usage.py – full usage record structure

def _empty_record() -> dict:
    return {
        "created_by": None,          # "agent" → Curator jurisdiction
        "use_count": 0,              # Times loaded into prompt path (hard signal)
        "view_count": 0,             # Calls to skill_view() (soft signal)
        "last_used_at": None,        # ISO‑8601 timestamp
        "last_viewed_at": None,
        "patch_count": 0,            # Number of skill_manage patch/edit operations
        "last_patched_at": None,
        "created_at": _now_iso(),    # Creation time, used only as fallback anchor
        "state": STATE_ACTIVE,       # active / stale / archived
        "pinned": False,             # True → bypass all automatic state changes
        "archived_at": None,
    }

Three counters are incremented at distinct points:

# Increment when the skill is actually executed

def bump_use(skill_name: str) -> None:
    def _apply(rec):
        rec["use_count"] = int(rec.get("use_count") or 0) + 1
        rec["last_used_at"] = _now_iso()
    _mutate(skill_name, _apply)

# Increment when the skill is merely viewed

def bump_view(skill_name: str) -> None:
    def _apply(rec):
        rec["view_count"] = int(rec.get("view_count") or 0) + 1
        rec["last_viewed_at"] = _now_iso()
    _mutate(skill_name, _apply)

# Increment when the skill content is patched

def bump_patch(skill_name: str) -> None:
    def _apply(rec):
        rec["patch_count"] = int(rec.get("patch_count") or 0) + 1
        rec["last_patched_at"] = _now_iso()
    _mutate(skill_name, _apply)

The helper latest_activity_at() returns the most recent of the three activity timestamps, deliberately excluding created_at so that a newly created but never used skill is not considered active:

def latest_activity_at(record: dict) -> str | None:
    """Return the latest real activity time – the max of three timestamps, excluding created_at."""
    latest_dt, latest_raw = None, None
    for key in ("last_used_at", "last_viewed_at", "last_patched_at"):
        raw = record.get(key)
        dt = _parse_iso_timestamp(raw)
        if dt and (latest_dt is None or dt > latest_dt):
            latest_dt, latest_raw = dt, str(raw)
    return latest_raw  # None means never used/viewed/patched

Atomic write – sidecar concurrency safety

The sidecar file is shared across Agent, Curator, and CLI processes. Hermes guarantees safety with two layers:

Layer 1: a file lock ( fcntl on Unix, msvcrt on Windows) serialises cross‑process read‑modify‑write.

Layer 2: tempfile + os.replace ensures writes are either fully applied or not applied at all.

# Two‑layer atomic write
@contextmanager
def _usage_file_lock():
    lock_path = _usage_file().with_suffix(".json.lock")
    fd = open(lock_path, "a+", encoding="utf-8")
    try:
        fcntl.flock(fd, fcntl.LOCK_EX)   # exclusive lock
        yield
    finally:
        fcntl.flock(fd, fcntl.LOCK_UN)
        fd.close()

def save_usage(data: dict) -> None:
    path = _usage_file()
    fd, tmp_path = tempfile.mkstemp(dir=str(path.parent), prefix=".usage_", suffix=".tmp")
    try:
        with os.fdopen(fd, "w", encoding="utf-8") as f:
            json.dump(data, f, indent=2, sort_keys=True, ensure_ascii=False)
            f.flush()
            os.fsync(f.fileno())
        os.replace(tmp_path, path)  # atomic replace
    except BaseException:
        try:
            os.unlink(tmp_path)
        except OSError:
            pass
        raise

How Curator reads signals to make decisions

Curator calls agent_created_report() to aggregate each agent‑created skill's state, back‑fill missing fields for backward compatibility, and compute derived metrics such as last_activity_at and activity_count (the sum of use, view, and patch counts).

# tools/skill_usage.py – report aggregation

def agent_created_report() -> list[dict]:
    data = load_usage()
    rows = []
    for name in list_agent_created_skill_names():
        rec = data.get(name) or _empty_record()
        for k, v in _empty_record().items():
            rec.setdefault(k, v)  # back‑fill missing fields
        row = {"name": name, **rec}
        row["last_activity_at"] = latest_activity_at(row)
        row["activity_count"] = activity_count(row)  # use+view+patch total
        rows.append(row)
    return rows

The aggregated rows are rendered as an LLM‑readable candidate list, for example:

Agent‑created skills (12):

- pdf-extraction   state=active   pinned=no   activity=23   use=18   view=5   patches=2   last_activity=2026-05-20T14:32:11+00:00
- slack-notifier   state=stale    pinned=no   activity=2    use=1    view=1   patches=0   last_activity=2026-03-01T09:11:22+00:00
- old-debug-helper state=stale    pinned=no   activity=0    use=0    view=0   patches=0   last_activity=never

Skills with activity=0 and last_activity=never become primary targets for Curator's LLM review (merge into an umbrella skill or archive).

Common pitfalls

Pitfall 1: Assuming a high view_count means usefulness. view_count only reflects calls to skill_view(), which Curator itself triggers. The hard usefulness signal is use_count, which counts actual executions.

Pitfall 2: Creating a skill file manually without invoking mark_agent_created(). Such a skill lacks the created_by="agent" flag and is invisible to Curator. Call mark_agent_created(skill_name) after manual creation.

Pitfall 3: Confusing latest_activity_at with created_at. A skill created 35 days ago but never used has latest_activity_at=None, causing Curator to treat it as stale based on the age of created_at.

Pitfall 4: Treating pinned=True as a temporary protection. It is a permanent opt‑out that bypasses all automatic state migrations. Use hermes curator pin <skill-name> to protect and hermes curator unpin <skill-name> to remove.

Summary

The .usage.json sidecar is the single source of truth for skill telemetry—no file‑system inference, no contamination of user content, independent storage, and atomic writes.

Three skill types and two defense lines (bundled manifest + hub lock) ensure that only explicitly marked agent‑created skills fall under Curator's management, preventing accidental deletion of official skills. use_count is the hard signal of actual execution; view_count is a soft signal. Curator prioritises the former when evaluating usefulness. latest_activity_at deliberately excludes created_at, so newly created but never active skills are not mis‑judged as "active today".

Atomic writes combined with file‑level locks guarantee safe concurrent updates, preventing data corruption even if a process crashes.