AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

scholar-evaluation

Provides qualitative-first, evidence-traceable developmental review of scholarly works and audit low-stakes research-assessment rubrics with optional local quality controls. Never use for ranking people or consequential decisions.

Original source
K-Dense-AI
License
MIT
Pinned revision
5f2bc0341354fd5590aef76a533ada1e1e5b4e62

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read k-dense-ai--scholar-evaluation at revision 5f2bc0341354fd5590aef76a533ada1e1e5b4e62, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 6

references/source_ledger.md

Dated Source Ledger

Refreshed on 2026-10-01 against the current official pages, source PDFs, and ScholarEval project record below. This updates the 2026-07-23 review. Policy-source verification and local synthetic-tool tests are distinct: no researcher profiles, bibliometric datasets, authenticated services, external model pipeline, or private work were evaluated.

The bundled workflow is local JSON/CSV only. It contains no scholarly-database, author-identity, citation-count, authentication, pagination, or rate-limit API contract. The shared collection citation's arXiv request is a metadata lookup, not an evaluation service. Do not graft provider endpoints into this rubric.

Responsible research assessment

San Francisco Declaration on Research Assessment (DORA)

  • Primary source: Read the Declaration
  • Origin: developed in 2012; official page rechecked 2026-10-01.
  • Verified points: assess work on its own merits; do not use journal-based measures as surrogates for an article or a person's contribution; state criteria explicitly; consider data, software, and other outputs; use qualitative evidence; make metric methods transparent and account for field and output-type variation.
  • Use here: categorical ban on scored prestige proxies and a requirement for explicit criteria, diverse evidence, and traceability.

DORA quantitative-indicator guidance

  • Primary source: Guidance on the responsible use of quantitative indicators
  • Document: official PDF
  • Release: 2024; official resource page accessible in 2026.
  • Verified points: no indicator captures research quality in one number; uses should be clear, transparent, specific, contextual, and fair. The guidance addresses journal measures, citation counts, the h-index, field-normalized indicators, and altmetrics; it warns about reductive, aggregate, composite, lagging, field, career-stage, and bias effects.
  • Use here: quantitative indicators are excluded from rubric scores. If mentioned descriptively outside the tools, their purpose, data, coverage, time window, field normalization, uncertainty, bias, and non-quality meaning must be explicit.

Leiden Manifesto

  • Primary source: Hicks, Wouters, Waltman, de Rijcke, and Rafols, “Bibliometrics: The Leiden Manifesto for research metrics”, Nature 520, 429–431.
  • Published: 2015-04-22. Publisher metadata remains current. The publisher full text was access-limited during this refresh; principles were checked against the author-linked Spanish translation hosted by the official manifesto site.
  • Verified points: quantitative evaluation should support qualitative expert assessment; measure against missions; protect locally relevant research; account for field variation; keep data and analysis open and verifiable; allow those evaluated to verify data; account for career age and database differences; avoid false precision; recognize gaming and system effects; review indicators regularly.
  • Use here: contextualization, inspectability, uncertainty, fairness review, and periodic revision.

Agreement on Reforming Research Assessment / CoARA

  • Primary record: Agreement on Reforming Research Assessment, version 1.
  • Published: 2022-07-20; Zenodo record modified 2024-08-29.
  • Current official overview: CoARA Agreement
  • Verified points: recognize diverse outputs, practices, activities, roles, and careers; base assessment primarily on qualitative judgment with peer review central; use quantitative indicators responsibly; abandon inappropriate uses of journal- and publication-based measures, especially Journal Impact Factor and h-index; publish criteria; train assessors; review and evaluate criteria, tools, and processes.
  • Use here: qualitative-first process, rubric provenance, rater training, monitoring, and no metric shortcut.

Hong Kong Principles

The Metric Tide and its commissioned revisit

  • Primary public-sector source: Research England/UKRI, The Metric Tide.
  • Published: 2015-07-06.
  • Revisit: Curry, Gadd, and Wilsdon, Harnessing the Metric Tide, version 2 (DOI 10.6084/m9.figshare.21701624).
  • Posted: 2022-12-12; commissioned by the joint UK higher-education funding bodies for the Future Research Assessment Programme.
  • Verified points: the revisit recommends putting principles into practice, evaluating with those evaluated, avoiding all-metric approaches, using data for public benefit, and rethinking rankings.
  • Status limitation: Harnessing the Metric Tide describes itself as an independent input to deliberations, not the eventual policy conclusion.
  • Use here: stakeholder participation, no all-metric process, and explicit scrutiny of rankings and system effects.

Current UKRI guidance

  • Primary policy: UKRI funding assessment and decision-making policy and principles
  • Primary implementation guidance: Résumé for Research and Innovation (R4RI)
  • R4RI last updated: 2026-04-30.
  • Verified points: UKRI will not use journal-based measures as surrogates for article quality, individual contribution, or funding decisions. R4RI evidences a wider range of team contributions; assessors do not score its individual modules or view it in isolation.
  • Use here: diverse contribution evidence and contextual review. This skill nevertheless blocks funding decisions entirely; the UKRI material is guidance context, not authorization to support such decisions.

UNESCO Recommendation on Open Science

  • Primary source: UNESCO Recommendation on Open Science.
  • Adopted: 2021-11-23 by the UNESCO General Conference.
  • Official overview: UNESCO Open Science
  • Verified points: quality and integrity, collective benefit, equity, fairness, diversity, inclusion, open engagement, training, and incentives aligned with open science; open science must not leave people, languages, disciplines, or knowledge systems behind.
  • Use here: assess responsible openness in context. Privacy, safety, consent, sovereignty, and legitimate restrictions can outweigh openness.

INORMS SCOPE framework

  • Primary source: SCOPE Framework full guide, v1.0.
  • Current official page: INORMS SCOPE Framework for Research Evaluation.
  • Verified points: Start with values; consider Context; identify Options; Probe for discrimination, gaming, unintended effects, and cost-benefit; and Evaluate the evaluation. Evaluate only where needed, with those evaluated, and with evaluation expertise.
  • Use here: process design and the bias/process checklist.

CRediT contributor taxonomy

  • Primary source: CRediT.
  • Standard: ANSI/NISO Z39.104-2022, approved 2022-01-14 and published 2022-02-08.
  • Verified points: 14 roles provide transparent attribution of diverse contributions. CRediT does not determine authorship or contribution quality.
  • Use here: optional vocabulary for contribution evidence, never a score.

Measurement, fairness, accessibility, and privacy

Standards for Educational and Psychological Testing

  • Primary source: AERA, APA, and NCME, Standards for Educational and Psychological Testing, 2014 edition.
  • Official status page: NCME Testing Standards.
  • Status: NCME still links the open-access 2014 edition. The APA page was not readable during this refresh; no later completed edition or current revision timetable was verified.
  • Verified points: intended interpretations and uses require validity evidence; reliability/precision and relevant errors should be reported; rater selection, training, qualification, monitoring, agreement, accuracy, and drift need documentation; fairness and subgroup validity require evidence; uncertainty should accompany estimates.
  • Use here: these are measurement principles, not proof that this rubric is a psychological test. The template records evidence gaps and must not be described as validated psychometrics.

Accessibility

  • Primary source: W3C, Web Content Accessibility Guidelines 2.2.
  • Status: W3C Recommendation published 2023-10-05; update noted 2024-12-12.
  • Use here: accessible materials and reasonable accommodation processes are required; local legal and institutional requirements may be broader.

Data protection

  • Primary guidance: UK Information Commissioner's Office, purpose limitation and data minimisation.
  • Current guidance dates found: purpose limitation updated 2026-03-23; data minimisation page published 2025-09-09.
  • Verified points: specify legitimate purposes and process only adequate, relevant, necessary data; review and delete data no longer needed.
  • Use here: scripts accept only minimized IDs, scores, statuses, and local references. They reject common private-application fields and never reproduce raw source documents.

ScholarEval paper and project status

  • Exact paper: Hanane Nour Moussa, Patrick Queiroz Da Silva, Daniel Adu-Ampratwum, Alyson East, Zitong Lu, Nikki Puccetti, Mingyi Xue, Huan Sun, Bodhisattwa Prasad Majumder, and Sachin Kumar, ScholarEval: Research Idea Evaluation Grounded in Literature.
  • Verified status: arXiv:2510.16234, submitted 2025-10-17; latest verified version v2, revised 2026-02-28. The displayed DOI 10.48550/arXiv.2510.16234 is an arXiv/DataCite DOI, not evidence of journal publication.
  • Official project: skai-research/ScholarEval. The repository describes itself as official code and data and cites the work as @misc; no release or peer-reviewed publication claim was verified.
  • Review-status caution: the OpenReview forum returned a browser-verification challenge on 2026-10-01. Its indexed submission PDF says under review; that historical PDF does not establish a current decision. ICLR pages were inaccessible to this review. Acceptance/peer-reviewed publication remains unverified, not disproven.
  • What the preprint reports: a retrieval-augmented framework assessing research ideas for soundness and contribution; a 117-idea, four-discipline dataset; coverage comparisons against expert-annotated review points; and a user study.
  • What it does not establish: validated psychometric measurement of scholar quality, transportability to personnel or funding decisions, validity of this skill's generalized rubric, stable cross-discipline score meaning, or freedom from subgroup bias.

Work-version and correction metadata

  • Primary source: Crossmark.
  • Verified points: publishers deposit changes and correction/retraction information; Crossref explicitly says the presence of Crossmark is not a guarantee. Missing metadata cannot establish absence of an update.
  • Use here: preserve exact version/access date, inspect available publisher notices, record unresolved status, and issue a fresh evaluation after changes. An identifier or reachable URL does not prove that a cited source supports a claim. Read the relevant content and separate metadata verification from substantive evidence review.

Review cadence

Re-check this ledger before any rubric adoption and at least annually. Re-check the ScholarEval arXiv and official project records before describing its publication status. Record any local disciplinary standards separately; a global source cannot substitute for local construct validation.