A bounded test of technical and workflow fit.
This package defines what a qualified platform may evaluate, what JRS supplies, what the evaluator must control, how success is measured, and what happens to access and evaluation materials when the test ends. The public page is the scope baseline. Evaluation rights arise only from a written agreement. The Review Engine remains unvalidated and in operational validation.
1. Evaluation scope
Technical fit
- Current versioned API request and response
- Authentication and error behavior
- Request correlation
- Manifest generation and validation
- Customer-controlled evidence storage pattern
Workflow fit
- Selected constructed or approved de-identified records
- Human review and routing
- Output usefulness
- Implementation effort
- Documented limitations and failure modes
Production activity
- Autonomous decisions
- Production customer records
- High-volume or availability testing
- End-customer exposure
- Public endorsement or case-study rights
Outcome claims
- Legal or regulatory compliance
- Organizational effectiveness
- Accuracy on real-world records
- Security certification
- Market adoption or production readiness
2. Materials supplied
| Material | Purpose | Public status |
|---|---|---|
| JRS Standard and Codebook | Defines the public method and condition vocabulary. | Available |
| Review Engine documentation | Documents current API behavior, limitations, request, response, and errors. | Available |
| OpenAPI 3.1 specification | Machine-readable current endpoint contract. | Available |
| Manifest v1.0 package | Schema, example, generator controls, hashing model, and offline validation. | Available |
| Data-flow and security disclosures | Identifies processors, transmission, persistence, telemetry, access, and limitations. | Available |
| Evaluation access credential | Bounded access to the controlled endpoint. | Issued only under written scope |
| Evaluation ledger and closeout | Records test cases, results, defects, decisions, and disposition. | Created for the engagement |
3. Current architecture and API relationship
The evaluator sends one approved representation to the current Review Engine and receives a structured response. An authenticated request can explicitly include a development Manifest in the response. The controlled generator refuses unsupported versions, vocabularies, condition sets or routing values rather than supplying a missing mapping. The evaluator must validate and retain the artifact; the current API does not create a customer evidence store.
Fail-closed mapping
The generator refuses unknown condition or routing vocabularies, incomplete condition sets, unsupported statuses, missing versions, and silent relabeling of Review Engine keys as Codebook conditions.
Integrity boundary
Hashing can detect change against a retained reference. It does not authenticate the creator. Signing and governed key custody are not represented as deployed.
4. Data flow and handling
| Stage | Data | Control |
|---|---|---|
| Evaluator environment | Authoritative record or constructed case. | Evaluator controls source, approval, de-identification, custody, and retention. |
| Transmission | Approved record representation. | TLS transport; evaluator confirms privilege, confidentiality, personal-data, and processor acceptability. |
| Model inference | Submitted representation. | Current model provider and processing path disclosed before use. |
| JRS persistence | No raw record text. Defined derived telemetry may be retained on the versioned route. | Exact retained fields, purpose, access, retention period, and deletion process must be stated in the written scope. |
| Platform evidence store | Response, request identifier, Manifest, and platform audit entry. | Customer-controlled storage, access, retention, legal hold, and deletion. |
5. Test procedure
Baseline
Freeze API, engine, model, JRS, Codebook, Manifest, schema, and adapter versions. Record the approved data set and exclusions.
Contract tests
Test successful calls, missing authentication, invalid input, short records, rate behavior, upstream failure, identifiers, and schema conformance.
Manifest tests
Generate artifacts, validate offline, confirm content classification, verify hashes, test tampering, and confirm refusal of unsupported mappings.
Workflow test
Store the artifact beside a synthetic platform record, route for human review, and confirm that a later reviewer can identify versions, findings, and limitations.
Operational review
Record integration work, support requests, defects, unresolved assumptions, security questions, and dependencies on founder explanation.
Closeout
Compare evidence with success measures, classify failures, decide whether to stop or continue, revoke credentials, and complete the agreed disposition.
6. Expected outputs and success measures
| Measure | Pass evidence | Failure evidence |
|---|---|---|
| Contract fidelity | Documented requests and responses conform; errors fail clearly. | Undocumented fields, silent coercion, ambiguous errors, or contract drift. |
| Traceability | Input hash, request identifier, versions, results, routing, and human-review status remain connected. | A later reviewer cannot tie the stored artifact to the evaluated input or configuration. |
| Integrity | Offline validator accepts the artifact and detects controlled tampering. | Alteration passes unnoticed or unverifiable data is treated as valid. |
| Data boundary | Every transmitted, retained, and customer-stored field has an identified purpose and disposition. | Unknown retention, undisclosed processor path, or sensitive test data outside scope. |
| Workflow fit | Platform can store, display, route, and retrieve the result within the bounded test. | Material workflow reconstruction or unsupported manual intervention is required. |
| Operational independence | Evaluator completes the documented path without undocumented founder knowledge. | Essential steps depend on oral explanation or unrecorded decisions. |
7. Procurement and legal terms to resolve
| Area | Required written treatment | Current public posture |
|---|---|---|
| Evaluation rights | Environment, users, test cases, versions, prohibited uses, copying, reverse engineering, publication, and end-customer exposure. | Agreement required |
| Confidentiality | Protected information, exclusions, permitted recipients, security, compelled disclosure, survival, and return or destruction. | Agreement required |
| Support | Named contacts, support window, response expectations, defect handling, change notice, and excluded services. | Scope required |
| Retention and deletion | Raw text, derived telemetry, logs, artifacts, backups, evaluator copies, time periods, deletion confirmation, and legal exceptions. | Scope required |
| Subprocessors | Identities, processing functions, locations where relevant, change notice, and evaluator approval or objection process. | Diligence required |
| Incident handling | Definition, notification channel, timing, cooperation, containment, evidence preservation, and responsibility. | Agreement required |
| Service expectations | Availability, maintenance, limits, suspension, no-production boundary, and disclaimer of uncommitted service levels. | No public SLA |
| Termination and disposition | Credential revocation, retained results, deletion or return, survival, publication limits, and transition. | Agreement required |
| Rights available | Assets, evaluation rights, commercial-use restrictions, trademark treatment, derivatives, improvements, exclusivity, and transferability. | Counsel-reviewed schedule required |
8. Closeout record
The evaluation ends with a written record of scope completed, test evidence, defects, limitations, support used, data disposition, credential status, unresolved procurement matters, and the decision to stop, revise, extend, negotiate a licence, or begin transaction diligence. Silence is not treated as success.