Skip to content

Commit 305711e

Browse files
aplfaplf
committed
Finish benchmark scripts and docs
1 parent 3cd59ec commit 305711e

3 files changed

Lines changed: 210 additions & 31 deletions

File tree

‎README.md‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -193,11 +193,10 @@ _For more examples, please refer to the [Documentation](https://example.com)_
193193

194194
# 🧪 `Evaluation and Static Analysis`
195195

196-
The on-chain layer ships with a reproducible evaluation harness (gas, cost,
197-
scalability, and latency benchmarks) and a Slither static-analysis setup, all
198-
under `code/services/ethereum`. The run commands and the full methodology —
199-
research questions, metrics, and environments — live in the on-chain layer's
200-
**[README → Evaluation](code/services/ethereum/README.md#evaluation)**.
196+
The on-chain layer includes benchmarking scripts (gas, cost, scalability, and
197+
latency) and a Slither static-analysis setup, all under `code/services/ethereum`.
198+
The scripts, run commands, and caveats are documented in the on-chain layer's
199+
**[README → Benchmarks](code/services/ethereum/README.md#benchmarks-eval)**.
201200

202201
<p align="right">(<a href="#readme-top">back to top</a>)</p>
203202

‎code/services/ethereum/README.md‎

Lines changed: 175 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,180 @@
1-
# Sample Hardhat Project
1+
# On-Chain Layer — SCAR / DiGo Certify
22

3-
This project demonstrates a basic Hardhat use case. It comes with a sample contract, a test for that contract, and a Hardhat Ignition module that deploys that contract.
3+
This is the [Hardhat](https://hardhat.org/) project for the **on-chain layer** of
4+
SCAR / DiGo Certify: an academic-certificate registry built by repurposing
5+
**OnchainID** (ERC-734/735 self-sovereign identity) and **T-REX / ERC-3643**
6+
(permissioned-token compliance) into a registry for identity-bound, accredited,
7+
updatable, and privately verifiable credentials. Verification runs directly
8+
against the ledger, so it does not depend on the client application.
49

5-
Try running some of the following tasks:
10+
Contracts are compiled with Solidity **0.8.17 / 0.8.20**, optimizer **enabled**,
11+
**`runs = 200`** (see `hardhat.config.js`).
612

7-
```shell
8-
npx hardhat help
13+
## Project structure
14+
15+
| Path | Contents |
16+
|------|----------|
17+
| `contracts/` | Solidity sources — the reused OnchainID / T-REX suite plus this project's additions (`contracts/config/FileStorage.sol`, `contracts/config/HashAnchor.sol`). |
18+
| `scripts/` | The certificate life-cycle operations (deploy identities/issuers, add keys, add/read claims, encryption helpers) used by the app and the benchmarks. |
19+
| `test/` | Hardhat/Mocha tests, including the negative/security tests in `test/security/` and the shared `test/fixtures.js`. |
20+
| `eval/` | Benchmarking scripts (`measure.js`, `baseline.js`, `sweeps.js`, `latency.js`), their CSV output under `eval/results/`, and the committed Slither reports. |
21+
22+
## Getting started
23+
24+
```sh
25+
npm install
26+
npx hardhat compile
927
npx hardhat test
10-
REPORT_GAS=true npx hardhat test
11-
npx hardhat node
12-
npx hardhat ignition deploy ./ignition/modules/Lock.js
1328
```
29+
30+
## Certificate life-cycle operations
31+
32+
The registry is driven by a small set of operations, each implemented as a script
33+
under `scripts/`:
34+
35+
| Operation | Script / function | When |
36+
|-----------|-------------------|------|
37+
| Deploy OnchainID suite | `suites/OID.js` → `deployOnchainIDSuite` | once |
38+
| Deploy T-REX suite | `suites/TREX.js` → `deployTrexSuite` | once |
39+
| Deploy a claim issuer (institution) | `claimIssuer/deploy-claim-issuer.js` → `deployClaimIssuer` | per institution |
40+
| Accredit an issuer (add to the trusted-issuer registry) | `addTrustedIssuer` (inside `deployClaimIssuer`) | per institution |
41+
| Create a student identity | `identities/deploy-identity.js` → `deployIdentity` | per student |
42+
| Authorise an issuer key on an identity | `claimIssuer/addKeyToIdentity.js` → `addKeyToIdentity` | per (student, issuer) |
43+
| Issue a certificate (add claims) | `claims/add-claim.js` → `addClaim` | per claim |
44+
| Update / revoke a certificate | `addClaim` again (emits `ClaimChanged`) | per correction |
45+
| Validate a certificate | `claims/getClaimsByTopic.js` + `utils/encryption/hash.js` | per verification |
46+
47+
A claim is keyed by `claimId = keccak256(issuer, topic)`. Re-issuing the same
48+
`(issuer, topic)` overwrites the existing claim in place (emitting `ClaimChanged`)
49+
rather than adding a second one — this is the update/revocation path. Holding
50+
several claims of the *same* topic on one identity therefore requires distinct
51+
issuers.
52+
53+
## Benchmarks (`eval/`)
54+
55+
The `eval/` directory contains scripts that measure the gas, cost, scalability,
56+
and latency of the operations above. They are plain `hardhat run` scripts and
57+
append one CSV row per measurement under `eval/results/`.
58+
59+
Gas is deterministic (identical bytecode + input + state ⇒ identical gas on any
60+
EVM chain), so gas is collected on the local Hardhat network. Fiat cost depends
61+
on the gas price and exchange rate, so on the local network the cost columns are
62+
≈0 and are meant to be computed in post-processing from `gasUsed`; against a real
63+
network they are populated from the live fee data. Wall-clock latency is only
64+
meaningful against a network with real block times (a testnet or private net).
65+
66+
### Scripts
67+
68+
- **`measure.js`** — per-operation gas, cost, and latency for each life-cycle
69+
operation, plus contract deployment gas and bytecode size (checked against the
70+
**EIP-170** 24 576-byte limit). Covers `deployIdentity`, `addClaim` (cold and
71+
warm), `addKeyToIdentity`, `addTrustedIssuer`, `deployClaimIssuer`, the contract
72+
deployments, and the validation read path.
73+
- **`baseline.js`** — the same metrics for a minimal hash-anchoring registry
74+
(`contracts/config/HashAnchor.sol`: store a certificate's SHA-256 digest, verify
75+
by lookup). Useful as a lower-bound comparison for the identity + compliance
76+
layer.
77+
- **`sweeps.js`** — scalability sweeps: issuer-resolution cost as the
78+
trusted-issuer registry grows, and `addClaim` gas + validation read latency as
79+
the number of claims on one identity grows. Writes raw and summarised
80+
(median/p95/min/max) CSVs.
81+
- **`latency.js`** — end-to-end issuance and validation latency against a live
82+
network, intended for Sepolia (or a private net). Uses a single funded key.
83+
84+
### Running
85+
86+
All commands run from this directory (`code/services/ethereum`):
87+
88+
```sh
89+
# Per-operation gas + cost, and contract sizes (local) -> eval/results/measure.csv
90+
REPS=30 ETH_USD=3000 npx hardhat run eval/measure.js
91+
92+
# Hash-anchoring baseline, same methodology (local) -> eval/results/baseline.csv
93+
REPS=30 ETH_USD=3000 npx hardhat run eval/baseline.js
94+
95+
# Scalability sweeps (local)
96+
# -> eval/results/sweep-*.csv (raw) + eval/results/sweep-*-summary.csv
97+
REPS=10 STEP=5 MAX_CLAIMS=30 npx hardhat run eval/sweeps.js
98+
99+
# End-to-end latency on a live network (needs a funded key + RPC endpoint)
100+
SEPOLIA_RPC=<your-rpc-url> PRIVATE_KEY=<funded-sepolia-key> \
101+
REPS=30 npx hardhat run eval/latency.js --network sepolia
102+
```
103+
104+
Common environment variables: `REPS` (samples per operation), `ETH_USD` (rate
105+
used for the cost columns), `STEP` / `MAX_ISSUERS` / `MAX_CLAIMS` (sweep sizes),
106+
`CONFIRMATIONS` (confirmations to wait for in `latency.js`).
107+
108+
All output CSVs share the schema
109+
`iso,network,operation,run,gasUsed,gasPriceGwei,costEth,costUsd,latencyMs,block,params`,
110+
one row per run.
111+
112+
### Notes
113+
114+
- **Scripts append.** Each run appends to its CSV; archive or delete the old file
115+
first if you want a clean dataset.
116+
- **Cold vs. warm storage.** The first write to a storage slot is far more
117+
expensive than a rewrite. `addClaim` on a new `(issuer, topic)` emits
118+
`ClaimAdded` (cold); repeating it emits `ClaimChanged` (warm, the update path).
119+
`measure.js` records both as `addClaim:cold` / `addClaim:warm`. It replicates
120+
the on-chain `addClaim` call in a local `buildClaimTx` helper because
121+
`scripts/claims/add-claim.js` returns the event args rather than the
122+
transaction — keep the two in sync if `add-claim.js` changes.
123+
- **Cold-loop signer cap.** `measure.js`'s `addClaim:cold` loop needs one funded
124+
signer per run and reserves signers 0–2, so it caps at 17 with Hardhat's default
125+
20 accounts (it logs a warning when it caps below `REPS`). Add
126+
`accounts: { count: 33 }` to the `hardhat` network in `hardhat.config.js` for a
127+
full 30-run cold loop.
128+
- **Trusted-issuer registry limit.** The T-REX `TrustedIssuersRegistry` reverts
129+
beyond 50 issuers (`require(_trustedIssuers.length < 50)`), so `sweeps.js` clamps
130+
the registry sweep to 49.
131+
- **Claims-per-identity sweep.** Because `claimId = keccak256(issuer, topic)`, N
132+
claims under one topic need N distinct issuers; the sweep deploys real
133+
`ClaimIssuer` contracts, since `Identity.addClaim` validates external issuers
134+
on-chain via `isClaimValid`.
135+
- **`deployFullTREXSuiteFixture` is local-only.** It fires some setup transactions
136+
without awaiting confirmation, which is fine on the auto-mining local net but
137+
races on a live network; `latency.js` therefore builds only what it needs and
138+
awaits every transaction.
139+
140+
## Static analysis (Slither)
141+
142+
Security-relevant static analysis is run with
143+
[Slither](https://github.com/crytic/slither) over the contracts. Slither invokes
144+
`npx hardhat compile` under the hood and writes its report to **stderr**, so
145+
redirect it to a file:
146+
147+
```sh
148+
# one-time: create a Python virtual environment with Slither
149+
python3 -m venv local
150+
source local/bin/activate
151+
pip install slither-analyzer # tested with Slither 0.11.5 on Python 3.13
152+
153+
# full detector report and a human-readable summary
154+
slither . 2> eval/slither.txt
155+
slither . --print human-summary 2> eval/slither-summary.txt
156+
```
157+
158+
The core OnchainID / T-REX contracts come from audited upstream
159+
implementations, so findings are triaged against this project's additions
160+
(`contracts/config/FileStorage.sol`, `contracts/config/HashAnchor.sol`, and the
161+
deployment scripts). The committed reference outputs are in `eval/slither.txt` and
162+
`eval/slither-summary.txt`.
163+
164+
The negative tests under `test/security/` complement the static analysis by
165+
asserting the access-control guards hold — e.g. a non-trusted issuer calling
166+
`addClaim` reverts (the `isTrustedIssuer || isSelfSigner` guard in
167+
`add-claim.js`), a claim written without an authorised key on the identity
168+
reverts, and a claim with a bad issuer signature does not validate.
169+
170+
## Environment
171+
172+
Gas figures depend on the toolchain, so the pinned versions are worth recording
173+
when comparing runs:
174+
175+
- Solidity **0.8.17 / 0.8.20**, optimizer **enabled**, **`runs = 200`**
176+
(`hardhat.config.js`).
177+
- `hardhat ^2.22.5`, `ethers ^6.13.1`, `@onchain-id/identity-sdk ^1.7.1`,
178+
`@openzeppelin/contracts ^4.8.3`, `crypto-js ^4.2.0`.
179+
- Networks: local `hardhat` (`chainId 31337`, default) for gas and sweeps;
180+
`sepolia` for latency (set `SEPOLIA_RPC` and `PRIVATE_KEY`).

‎code/services/ethereum/eval/measure.js‎

Lines changed: 31 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,15 @@
88
* It measures gas + cost + latency for each certificate life-cycle operation
99
* and appends one CSV row per run to eval/results/measure.csv.
1010
*
11-
* STATUS: helpers, CSV writer and setup are working; `deployIdentity` and
12-
* `addClaim` (cold + warm) are fully wired. The remaining operations are marked
13-
* `TODO` — copy the wired patterns using the real script functions (see
14-
* eval/methodology.md §4 for the catalog).
11+
* STATUS: every life-cycle operation in the catalog is wired — deployIdentity,
12+
* addClaim (cold + warm), addKeyToIdentity, addTrustedIssuer, deployClaimIssuer,
13+
* the contract deployments, and the E4 validation read path. See the "Operation
14+
* catalog" section of ../README.md for the catalog.
1515
*
1616
* Gas is the primary, deterministic metric (identical on any EVM chain). On the
1717
* local network the cost columns are ~0 and are computed in post-processing from
18-
* gasUsed using the scenario grid in methodology.md §5; on a testnet the cost
19-
* columns are meaningful.
18+
* gasUsed using the scenario grid in ../README.md (Experiments, E2); on a
19+
* testnet the cost columns are meaningful.
2020
*/
2121

2222
const fs = require('fs');
@@ -37,7 +37,7 @@ const { addKeyToIdentity } = require('../scripts/claimIssuer/addKeyToIdentity');
3737
const { CLAIM_TOPICS, CLAIM_TOPICS_OBJ } = require('../scripts/claims/claimTopics');
3838
const hash = require('../scripts/utils/encryption/hash');
3939
// const { addClaim } = require('../scripts/claims/add-claim'); // see buildClaimTx note
40-
// const { getClaimsByTopic } = require('../scripts/claims/getClaimsByTopic');
40+
const { getClaimsByTopic } = require('../scripts/claims/getClaimsByTopic');
4141

4242
const REPS = Number(process.env.REPS ?? 30);
4343
const DEPLOY_REPS = Number(process.env.DEPLOY_REPS ?? 3); // deploy gas is deterministic
@@ -344,18 +344,31 @@ async function main() {
344344
}
345345

346346
// ====================================================================
347-
// E4 — validation read path (0 gas; latency) [TODO]
347+
// E4 — validation read path (0 gas to the caller; latency + view gas)
348348
// ====================================================================
349-
// Reuse identities[0] (it holds a CERTIFICATE claim from the cold loop):
350-
// const { getClaimsByTopic } = require('../scripts/claims/getClaimsByTopic');
351-
// await measureView(
352-
// 'validate',
353-
// () => warm.identity.getClaimIdsByTopic.estimateGas(ethers.id(claimTopic)),
354-
// async () => {
355-
// const claims = await getClaimsByTopic(warm.identity, claimTopic);
356-
// claims.some(c => c.uri === hash(presentedCertificate)); // the compare
357-
// }
358-
// );
349+
// Verification reads the subject's CERTIFICATE claims and re-computes the
350+
// digest of the presented certificate, comparing it against the stored one
351+
// (methodology: getClaimsByTopic + local SHA-256 compare). It is a read-only
352+
// op, so measureView records the estimated compute gas of the on-chain read
353+
// (getClaimIdsByTopic) plus the RPC round-trip latency of the full compare.
354+
// Reuse warm.identity, which holds a CERTIFICATE claim from the cold/warm
355+
// loops; derive the "presented" digest from the actually-stored claim so the
356+
// compare mirrors a genuine (matching) verification regardless of REPS.
357+
if (warm) {
358+
const stored = await getClaimsByTopic(warm.identity, claimTopic);
359+
const presentedDigest = stored.length
360+
? hash(ethers.toUtf8String(stored[0].data)) // what a verifier recomputes
361+
: null;
362+
await measureView(
363+
'validate',
364+
() => warm.identity.getClaimIdsByTopic.estimateGas(ethers.id(claimTopic)),
365+
async () => {
366+
const claims = await getClaimsByTopic(warm.identity, claimTopic);
367+
// integrity check: recompute each stored certificate's digest and compare
368+
claims.some(c => hash(ethers.toUtf8String(c.data)) === presentedDigest);
369+
}
370+
);
371+
}
359372

360373
console.log('\nDone. Summarise eval/results/measure.csv into the paper tables.');
361374
}

0 commit comments

Comments
 (0)