Improve native contract benchmarking harness #4284

Jim8y · 2025-11-07T12:05:26Z

Summary

add native contract benchmarking harness and manual runner with seeded chain context
expose CLI/env filters + helper script for quick runs
emit text/JSON/HTML reports with mean/median/p95 and highlight large profile variance

Testing

scripts/run-native-benchmarks.sh --job quick

Jim8y · 2025-11-07T12:08:14Z

<title>Native Contract Benchmarks</title> <style>body{font-family:Segoe UI,Arial,sans-serif;margin:2rem;}table{border-collapse:collapse;width:100%;margin-bottom:1.5rem;}th,td{border:1px solid #ccc;padding:0.4rem;text-align:left;}th{background:#f5f5f5;}caption{font-weight:bold;margin-bottom:0.5rem;}</style>

Native Contract Benchmarks

Generated at 2025-11-07 11:51:43Z. Iterations: 20, Warmup: 3, Trim: 10%.

ContractManagement

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
deploy	Tiny	131.044	128.400	161.500	9.586	118.700	222.900	OK
deploy	Tiny	100.912	88.900	894.000	37.941	82.600	3,814.400	OK
deploy	Small	124.644	123.150	154.200	5.502	114.900	175.400	OK
deploy	Small	89.994	87.550	128.000	7.736	80.000	131.900	OK
deploy	Medium	172.925	170.200	296.800	30.054	126.900	320.100	OK
deploy	Medium	87.175	86.150	123.700	4.216	79.900	126.500	OK
deploy	Large	128.238	125.800	148.800	6.046	122.100	208.300	OK
deploy	Large	83.925	83.050	104.900	3.240	79.000	111.300	OK
destroy	Tiny	51.906	51.000	67.300	2.817	35.200	108.500	OK
destroy	Small	50.887	50.450	66.800	1.552	49.100	69.800	OK
destroy	Medium	50.319	50.150	52.100	0.576	49.400	56.300	OK
destroy	Large	51.044	50.650	87.000	1.588	49.300	87.100	OK
getContract	Tiny	8.163	8.300	8.500	0.247	7.600	8.500	OK
getContract	Small	8.150	8.200	9.200	0.276	7.600	9.300	OK
getContract	Medium	8.138	8.200	8.900	0.300	7.600	17.100	OK
getContract	Large	8.150	8.200	8.400	0.209	7.700	8.500	OK
getContractById	Tiny	8.313	8.400	8.800	0.234	7.700	9.100	OK
getContractById	Small	8.138	8.250	8.500	0.262	7.600	8.500	OK
getContractById	Medium	8.113	8.200	8.600	0.267	7.600	16.800	OK
getContractById	Large	8.169	8.300	8.500	0.247	7.700	8.500	OK
getContractHashes	Tiny	11.825	11.500	30.000	0.843	10.500	37.600	OK
getContractHashes	Small	11.381	11.450	11.900	0.283	10.800	11.900	OK
getContractHashes	Medium	11.194	11.250	11.600	0.261	10.600	11.600	OK
getContractHashes	Large	11.200	11.200	11.800	0.260	10.700	19.400	OK
getMinimumDeploymentFee	Tiny	11.500	11.600	11.900	0.252	10.900	12.100	OK
getMinimumDeploymentFee	Small	11.444	11.500	11.700	0.206	10.900	11.700	OK
getMinimumDeploymentFee	Medium	11.425	11.450	11.800	0.230	11.100	11.800	OK
getMinimumDeploymentFee	Large	12.275	11.850	21.600	1.404	11.000	34.600	OK
hasMethod	Tiny	11.575	11.600	11.900	0.208	11.100	12.000	OK
hasMethod	Small	11.688	11.750	12.100	0.206	11.100	22.300	OK
hasMethod	Medium	12.069	12.100	12.500	0.116	11.800	12.800	OK
hasMethod	Large	12.819	12.900	13.200	0.272	12.300	13.400	OK
isContract	Tiny	11.206	11.300	11.900	0.305	10.500	19.800	OK
isContract	Small	11.662	11.650	16.000	0.421	10.500	36.600	OK
isContract	Medium	11.338	11.400	11.700	0.267	10.700	11.800	OK
isContract	Large	11.262	11.350	12.100	0.239	10.700	13.100	OK
setMinimumDeploymentFee	Tiny	43.450	42.050	61.100	3.901	40.900	97.000	OK
setMinimumDeploymentFee	Small	42.100	42.000	43.700	0.379	41.200	50.200	OK
setMinimumDeploymentFee	Medium	42.975	42.050	62.500	3.100	41.100	65.400	OK
setMinimumDeploymentFee	Large	42.413	42.400	43.900	0.401	41.400	52.500	OK
update	Tiny	94.381	90.600	143.500	12.394	83.600	279.000	OK
update	Tiny	88.019	83.600	131.100	11.774	81.300	134.500	OK
update	Small	111.181	89.950	251.500	39.134	86.300	3,230.700	OK
update	Small	111.706	114.550	171.200	26.261	80.800	298.100	OK
update	Medium	88.650	86.200	136.800	6.770	82.900	221.700	OK
update	Medium	118.556	116.000	136.500	4.812	113.300	158.000	OK
update	Large	87.950	84.050	138.200	11.842	81.600	365.100	OK
update	Large	132.737	134.100	161.400	9.491	112.700	163.100	OK

CryptoLib

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
bls12381Add	Tiny	46.794	46.650	48.500	0.515	45.300	56.500	OK
bls12381Add	Small	48.781	48.350	57.000	2.022	45.700	63.300	OK
bls12381Add	Medium	48.587	48.600	49.600	0.563	47.100	58.000	OK
bls12381Add	Large	47.050	46.900	54.500	1.085	45.900	65.500	OK
bls12381Deserialize	Tiny	5,959.463	5,909.550	6,619.500	150.273	5,710.200	9,419.700	OK
bls12381Deserialize	Small	16,815.625	16,770.750	18,127.400	561.252	16,004.500	18,185.900	OK
bls12381Deserialize	Medium	40.463	38.700	55.400	3.137	38.100	439.000	OK
bls12381Deserialize	Large	41.069	40.050	62.800	3.156	38.200	63.100	OK
bls12381Equal	Tiny	0.569	0.500	1.000	0.186	0.400	3.500	OK
bls12381Equal	Small	0.425	0.400	0.900	0.075	0.400	0.900	OK
bls12381Equal	Medium	0.406	0.400	0.500	0.024	0.400	1.100	OK
bls12381Equal	Large	0.406	0.400	0.500	0.024	0.400	1.000	OK
bls12381Mul	Tiny	87,137.738	87,032.750	88,024.700	362.414	85,821.700	88,631.900	OK
bls12381Mul	Small	88,413.269	88,397.100	89,591.900	635.385	86,747.700	90,067.700	OK
bls12381Mul	Medium	88,704.256	88,688.700	89,670.500	536.358	87,152.900	89,938.300	OK
bls12381Mul	Large	87,800.219	87,544.850	89,919.400	770.565	86,943.500	89,994.500	OK
bls12381Pairing	Tiny	67,081.369	66,641.850	68,827.800	811.915	66,058.400	68,924.600	OK
bls12381Pairing	Small	66,693.512	66,450.150	68,064.800	471.974	66,034.100	68,888.500	OK
bls12381Pairing	Medium	66,415.506	66,386.050	67,215.500	280.987	65,954.700	67,396.800	OK
bls12381Pairing	Large	66,253.494	66,120.850	66,846.900	268.166	65,872.200	67,193.700	OK
bls12381Serialize	Tiny	3.094	3.100	4.400	0.075	3.000	5.400	OK
bls12381Serialize	Small	3.100	3.100	4.500	0.079	3.000	4.900	OK
bls12381Serialize	Medium	3.050	3.050	3.200	0.050	3.000	3.200	OK
bls12381Serialize	Large	3.069	3.050	3.300	0.077	3.000	17.700	OK
keccak256	Tiny	2.038	2.000	3.200	0.206	1.900	4.300	OK
keccak256	Small	4.750	4.700	5.900	0.146	4.600	6.100	OK
keccak256	Medium	33.094	26.700	75.300	10.551	24.400	222.600	OK
keccak256	Large	46.125	45.700	52.500	1.100	43.200	52.500	OK
murmur32	Tiny	0.356	0.200	1.600	0.377	0.200	2.600	OK
murmur32	Small	1.450	1.450	2.000	0.050	1.300	4.500	OK
murmur32	Medium	7.719	7.700	8.200	0.194	7.400	8.400	OK
murmur32	Large	12.881	12.950	13.700	0.305	12.400	15.000	OK
recoverSecp256K1	Tiny	1.431	1.400	1.900	0.058	1.300	2.500	OK
recoverSecp256K1	Small	2.763	2.650	3.600	0.285	2.400	28.200	OK
recoverSecp256K1	Medium	7.438	7.600	7.800	0.322	6.900	7.900	OK
recoverSecp256K1	Large	8.919	8.900	9.100	0.053	8.800	9.300	OK
ripemd160	Tiny	0.869	0.800	1.400	0.085	0.800	2.300	OK
ripemd160	Small	3.375	3.300	3.900	0.211	3.200	9.000	OK
ripemd160	Medium	17.650	17.650	18.200	0.304	17.000	25.000	OK
ripemd160	Large	30.550	30.500	31.500	0.255	29.900	31.900	OK
sha256	Tiny	0.344	0.300	0.400	0.050	0.300	0.400	OK
sha256	Small	0.838	0.800	1.600	0.060	0.800	3.400	OK
sha256	Medium	3.444	3.300	4.000	0.252	3.200	4.000	OK
sha256	Large	5.062	5.000	9.200	0.420	4.400	19.000	OK
verifyWithECDsa	Tiny	169.263	167.300	181.100	3.697	165.200	196.200	OK
verifyWithECDsa	Tiny	168.550	167.100	177.200	3.344	164.900	212.800	OK
verifyWithECDsa	Small	168.619	167.700	175.000	2.902	164.000	175.300	OK
verifyWithECDsa	Small	167.250	166.600	174.900	1.670	164.700	177.500	OK
verifyWithECDsa	Medium	41.287	41.050	47.400	1.371	40.000	50.300	OK
verifyWithECDsa	Medium	40.706	40.450	43.400	0.579	40.100	44.000	OK
verifyWithECDsa	Large	40.456	40.350	41.100	0.247	40.100	43.800	OK
verifyWithECDsa	Large	40.406	40.300	41.300	0.284	40.000	45.900	OK
verifyWithEd25519	Tiny	0.344	0.300	0.400	0.050	0.300	1.700	OK
verifyWithEd25519	Small	2.250	2.200	2.800	0.146	2.100	2.900	OK
verifyWithEd25519	Medium	10.519	10.600	10.800	0.194	9.900	10.800	OK
verifyWithEd25519	Large	12.944	13.000	13.500	0.306	12.500	19.900	OK

GasToken

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
balanceOf	Tiny	10.087	9.950	12.700	0.485	9.100	13.700	OK
balanceOf	Small	10.931	10.550	13.500	0.937	9.800	33.400	OK
balanceOf	Medium	9.650	9.700	10.400	0.274	9.100	19.800	OK
balanceOf	Large	9.669	9.700	12.500	0.299	9.000	29.700	OK
decimals	Tiny	0.000	0.000	0.000	0.000	0.000	0.000	OK
decimals	Small	0.000	0.000	0.000	0.000	0.000	1.000	OK
decimals	Medium	0.000	0.000	0.000	0.000	0.000	0.000	OK
decimals	Large	0.000	0.000	0.000	0.000	0.000	0.300	OK
symbol	Tiny	0.000	0.000	0.000	0.000	0.000	0.000	OK
symbol	Small	0.000	0.000	0.000	0.000	0.000	0.000	OK
symbol	Medium	0.000	0.000	0.000	0.000	0.000	0.000	OK
symbol	Large	0.000	0.000	0.000	0.000	0.000	0.000	OK
totalSupply	Tiny	10.081	10.100	10.900	0.313	9.400	12.300	OK
totalSupply	Small	9.994	10.000	11.800	0.338	9.400	20.400	OK
totalSupply	Medium	9.800	9.900	10.100	0.242	9.200	10.100	OK
totalSupply	Large	9.750	9.900	10.600	0.226	9.200	11.200	OK
transfer	Tiny	16.919	16.700	26.500	1.365	15.200	28.500	OK
transfer	Small	16.950	16.800	19.900	0.696	16.000	50.800	OK
transfer	Medium	18.712	18.800	20.200	0.597	17.500	25.000	OK
transfer	Large	18.650	18.700	20.800	0.257	17.600	51.100	OK

LedgerContract

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
currentHash	Tiny	10.306	10.350	11.100	0.325	9.500	11.300	OK
currentHash	Small	10.350	10.400	11.200	0.180	9.700	21.600	OK
currentHash	Medium	10.113	10.200	10.300	0.239	9.600	10.300	OK
currentHash	Large	10.169	10.300	10.400	0.226	9.500	10.500	OK
currentIndex	Tiny	11.300	10.650	34.300	1.628	9.700	675.300	OK
currentIndex	Small	10.188	10.200	10.600	0.229	9.600	10.600	OK
currentIndex	Medium	10.238	10.300	10.600	0.280	9.500	16.600	OK
currentIndex	Large	10.244	10.300	11.500	0.283	9.600	11.700	OK
getBlock	Tiny	17.400	17.200	20.500	0.660	16.100	53.800	OK
getBlock	Small	16.875	16.850	19.600	0.282	16.200	26.200	OK
getBlock	Medium	16.525	16.600	17.200	0.282	15.800	17.700	OK
getBlock	Large	16.531	16.550	17.100	0.406	15.800	17.100	OK
getTransaction	Tiny	14.831	14.700	16.400	0.437	13.900	16.800	OK
getTransaction	Small	14.688	14.550	26.500	0.427	14.200	82.900	OK
getTransaction	Medium	14.400	14.400	14.800	0.265	13.900	15.000	OK
getTransaction	Large	14.275	14.150	14.900	0.270	13.900	14.900	OK
getTransactionFromBlock	Tiny	18.900	18.850	20.400	0.308	18.000	21.600	OK
getTransactionFromBlock	Small	18.625	18.600	20.200	0.135	18.100	22.300	OK
getTransactionFromBlock	Medium	18.600	18.600	23.000	0.262	17.700	51.300	OK
getTransactionFromBlock	Large	18.556	18.550	19.800	0.420	17.600	20.000	OK
getTransactionHeight	Tiny	14.638	14.550	15.000	0.289	14.100	16.600	OK
getTransactionHeight	Small	14.406	14.300	14.900	0.273	14.000	15.200	OK
getTransactionHeight	Medium	14.531	14.350	16.000	0.375	14.000	21.900	OK
getTransactionHeight	Large	14.538	14.450	16.200	0.314	13.900	45.700	OK
getTransactionSigners	Tiny	14.913	14.850	16.100	0.406	14.100	16.200	OK
getTransactionSigners	Small	14.494	14.500	15.000	0.295	13.900	15.000	OK
getTransactionSigners	Medium	14.331	14.250	14.700	0.220	14.000	15.000	OK
getTransactionSigners	Large	14.656	14.550	16.000	0.507	14.000	19.300	OK
getTransactionVMState	Tiny	14.881	14.900	24.400	0.429	14.100	40.100	OK
getTransactionVMState	Small	14.625	14.550	15.200	0.317	14.000	15.400	OK
getTransactionVMState	Medium	14.538	14.450	15.600	0.300	13.900	24.100	OK
getTransactionVMState	Large	14.400	14.250	15.000	0.316	13.800	15.300	OK

NeoToken

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
balanceOf	Tiny	10.719	10.700	11.700	0.323	9.900	20.800	OK
balanceOf	Small	9.800	9.800	10.900	0.389	9.100	11.000	OK
balanceOf	Medium	9.794	9.800	10.900	0.377	9.000	31.000	OK
balanceOf	Large	9.544	9.650	9.900	0.260	9.000	12.400	OK
decimals	Tiny	0.000	0.000	0.000	0.000	0.000	0.000	OK
decimals	Small	0.000	0.000	0.000	0.000	0.000	0.300	OK
decimals	Medium	0.000	0.000	0.000	0.000	0.000	0.000	OK
decimals	Large	0.000	0.000	0.000	0.000	0.000	0.000	OK
getAccountState	Tiny	10.775	10.700	12.000	0.286	10.100	20.700	OK
getAccountState	Small	9.450	9.550	9.700	0.224	8.900	9.900	OK
getAccountState	Medium	9.688	9.700	11.600	0.436	9.000	36.500	OK
getAccountState	Large	9.506	9.600	9.800	0.249	9.000	9.800	OK
getAllCandidates	Tiny	9.813	9.800	10.500	0.245	9.200	11.000	OK
getAllCandidates	Small	9.631	9.700	10.200	0.278	9.000	13.500	OK
getAllCandidates	Medium	9.744	9.800	9.900	0.177	9.100	10.000	OK
getAllCandidates	Large	9.688	9.700	10.900	0.328	9.100	11.000	OK
getCandidateVote	Tiny	9.713	9.750	10.300	0.287	9.000	20.400	OK
getCandidateVote	Small	9.650	9.700	10.100	0.274	9.100	11.300	OK
getCandidateVote	Medium	9.800	9.700	15.800	0.517	9.100	29.500	OK
getCandidateVote	Large	9.638	9.700	10.100	0.291	9.000	10.100	OK
getCandidates	Tiny	40.225	39.100	61.000	2.894	21.800	94.800	OK
getCandidates	Small	38.631	38.400	43.300	0.716	37.600	78.900	OK
getCandidates	Medium	38.400	38.300	39.500	0.392	37.700	42.200	OK
getCandidates	Large	38.631	38.600	39.700	0.278	21.600	42.200	OK
getCommittee	Tiny	22.737	22.500	24.700	0.587	21.800	25.500	OK
getCommittee	Small	23.013	22.050	31.400	2.516	21.500	38.100	OK
getCommittee	Medium	21.975	21.900	24.100	0.456	21.300	24.700	OK
getCommittee	Large	21.613	21.500	22.300	0.190	21.300	22.500	OK
getCommitteeAddress	Tiny	27.450	27.200	30.000	0.542	25.900	40.700	OK
getCommitteeAddress	Small	26.831	26.850	36.400	0.262	26.400	50.900	OK
getCommitteeAddress	Medium	26.631	26.650	27.300	0.231	26.100	33.900	OK
getCommitteeAddress	Large	26.762	26.700	29.100	0.403	25.900	32.000	OK
getGasPerBlock	Tiny	39.406	39.350	41.800	0.373	24.100	48.400	OK
getGasPerBlock	Small	39.138	39.150	43.200	0.377	21.700	47.800	OK
getGasPerBlock	Medium	41.006	39.050	55.500	4.482	38.400	86.100	OK
getGasPerBlock	Large	38.825	38.750	39.500	0.211	22.400	43.800	OK
getNextBlockValidators	Tiny	28.506	28.400	34.200	1.083	25.800	65.700	OK
getNextBlockValidators	Small	26.694	26.600	28.000	0.370	25.800	30.600	OK
getNextBlockValidators	Medium	26.200	26.150	27.100	0.237	25.800	32.300	OK
getNextBlockValidators	Large	26.656	26.500	43.500	0.769	25.800	44.600	OK
getRegisterPrice	Tiny	9.744	9.800	10.300	0.229	9.200	10.300	OK
getRegisterPrice	Small	9.806	9.900	10.800	0.327	9.100	11.100	OK
getRegisterPrice	Medium	9.681	9.800	10.000	0.263	9.100	10.200	OK
getRegisterPrice	Large	9.800	9.850	10.300	0.215	9.200	23.100	OK
onNEP17Payment	Tiny	22.331	22.200	24.600	0.734	21.500	27.300	OK
onNEP17Payment	Small	23.119	21.950	42.500	3.813	21.400	405.300	OK
onNEP17Payment	Medium	21.837	21.850	22.500	0.283	21.300	31.000	OK
onNEP17Payment	Large	21.669	21.600	22.500	0.335	21.000	23.400	OK
registerCandidate	Tiny	20.944	20.900	32.900	0.343	20.200	46.900	OK
registerCandidate	Tiny	20.219	20.200	22.400	0.170	19.600	52.500	OK
registerCandidate	Small	20.587	20.500	21.500	0.280	20.000	29.500	OK
registerCandidate	Small	20.281	20.200	21.300	0.255	19.900	26.600	OK
registerCandidate	Medium	20.338	20.350	20.900	0.132	19.600	21.000	OK
registerCandidate	Medium	20.137	20.200	20.400	0.136	19.500	20.400	OK
registerCandidate	Large	20.306	20.200	21.600	0.368	19.700	25.800	OK
registerCandidate	Large	20.225	20.200	21.100	0.249	19.300	24.400	OK
setGasPerBlock	Tiny	34.619	34.500	37.300	0.700	33.500	44.200	OK
setGasPerBlock	Small	34.606	34.200	43.200	1.323	33.300	69.000	OK
setGasPerBlock	Medium	34.044	33.800	42.700	1.521	32.500	58.500	OK
setGasPerBlock	Large	33.463	33.600	35.000	0.431	32.400	35.000	OK
setRegisterPrice	Tiny	33.644	33.500	36.300	0.722	32.400	71.200	OK
setRegisterPrice	Small	33.050	33.100	33.400	0.209	32.400	35.700	OK
setRegisterPrice	Medium	33.812	33.400	48.200	1.593	32.300	55.800	OK
setRegisterPrice	Large	33.081	33.000	33.800	0.383	32.200	39.800	OK
symbol	Tiny	0.000	0.000	0.000	0.000	0.000	0.000	OK
symbol	Small	0.000	0.000	0.000	0.000	0.000	0.000	OK
symbol	Medium	0.000	0.000	0.000	0.000	0.000	0.000	OK
symbol	Large	0.000	0.000	0.000	0.000	0.000	0.000	OK
totalSupply	Tiny	9.500	9.600	10.000	0.255	8.900	10.300	OK
totalSupply	Small	9.425	9.450	10.300	0.301	8.900	19.600	OK
totalSupply	Medium	9.469	9.550	9.700	0.239	9.000	10.000	OK
totalSupply	Large	9.444	9.550	9.800	0.215	9.000	10.200	OK
transfer	Tiny	17.025	17.000	19.300	0.439	16.100	27.900	OK
transfer	Small	15.931	16.000	16.600	0.320	15.400	16.800	OK
transfer	Medium	18.475	18.500	20.600	0.388	17.600	24.100	OK
transfer	Large	18.700	18.750	19.100	0.322	17.800	19.700	OK
unclaimedGas	Tiny	40.969	40.800	42.200	0.380	40.300	50.600	OK
unclaimedGas	Small	41.544	40.750	52.800	2.337	39.900	63.800	OK
unclaimedGas	Medium	40.569	40.600	41.300	0.412	39.800	44.200	OK
unclaimedGas	Large	41.119	40.650	54.100	1.718	40.200	54.700	OK
unregisterCandidate	Tiny	17.119	17.100	17.900	0.371	16.500	18.000	OK
unregisterCandidate	Tiny	16.850	16.950	17.600	0.400	16.100	17.600	OK
unregisterCandidate	Small	17.231	17.250	18.800	0.310	16.500	28.000	OK
unregisterCandidate	Small	16.637	16.600	17.200	0.289	16.000	17.200	OK
unregisterCandidate	Medium	16.963	17.000	18.500	0.335	16.300	50.100	OK
unregisterCandidate	Medium	16.694	16.800	17.900	0.321	16.100	18.600	OK
unregisterCandidate	Large	16.781	16.550	23.300	1.302	15.800	45.900	OK
unregisterCandidate	Large	16.387	16.550	24.700	0.355	15.700	34.800	OK
vote	Tiny	16.913	16.900	17.800	0.324	16.000	20.400	OK
vote	Tiny	16.850	16.900	18.700	0.190	16.200	22.700	OK
vote	Small	15.488	15.500	17.000	0.320	14.900	26.000	OK
vote	Small	15.231	15.300	15.800	0.269	14.800	16.100	OK
vote	Medium	15.381	15.450	16.500	0.392	14.600	17.200	OK
vote	Medium	15.263	15.350	15.600	0.226	14.700	20.100	OK
vote	Large	15.375	15.300	16.400	0.385	14.600	18.100	OK
vote	Large	15.200	15.050	16.000	0.308	14.600	16.500	OK

Notary

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
balanceOf	Tiny	10.450	10.400	10.800	0.170	9.900	10.800	OK
balanceOf	Small	10.375	10.400	12.400	0.214	9.600	19.600	OK
balanceOf	Medium	10.250	10.300	10.500	0.197	9.700	10.500	OK
balanceOf	Large	10.237	10.300	10.700	0.234	9.600	10.800	OK
expirationOf	Tiny	10.387	10.450	10.800	0.218	9.800	21.100	OK
expirationOf	Small	10.344	10.400	10.700	0.184	9.700	10.700	OK
expirationOf	Medium	10.312	10.350	10.600	0.215	9.800	10.600	OK
expirationOf	Large	10.625	10.500	12.400	0.432	9.800	33.200	OK
getMaxNotValidBeforeDelta	Tiny	9.763	9.800	10.900	0.226	9.200	19.400	OK
getMaxNotValidBeforeDelta	Small	9.706	9.800	10.000	0.244	9.200	10.200	OK
getMaxNotValidBeforeDelta	Medium	9.681	9.750	10.100	0.258	9.100	12.000	OK
getMaxNotValidBeforeDelta	Large	9.650	9.750	10.000	0.272	9.100	10.700	OK
lockDepositUntil	Tiny	20.575	19.950	28.400	1.458	18.600	333.500	OK
lockDepositUntil	Small	19.300	19.300	20.400	0.281	18.600	26.100	OK
lockDepositUntil	Medium	25.225	20.100	90.200	14.882	18.400	100.600	OK
lockDepositUntil	Large	19.606	19.400	22.000	0.745	18.800	51.200	OK
onNEP17Payment	Tiny	18.744	18.600	20.300	0.444	17.900	30.000	OK
onNEP17Payment	Small	19.894	19.900	27.000	0.472	17.900	49.000	OK
onNEP17Payment	Medium	18.312	18.400	18.700	0.209	17.300	27.700	OK
onNEP17Payment	Large	18.219	18.300	19.500	0.272	17.500	19.500	OK
setMaxNotValidBeforeDelta	Tiny	48.237	53.300	98.400	10.119	34.400	120.800	OK
setMaxNotValidBeforeDelta	Small	34.344	34.300	42.100	0.394	33.400	82.500	OK
setMaxNotValidBeforeDelta	Medium	35.025	34.500	46.400	1.335	33.800	56.000	OK
setMaxNotValidBeforeDelta	Large	34.263	34.300	34.800	0.322	33.400	36.000	OK
verify	Tiny	173.763	174.000	198.300	6.192	164.300	206.800	OK
verify	Small	169.613	167.250	178.300	4.142	164.400	203.000	OK
verify	Medium	169.188	167.750	190.000	4.449	148.900	200.000	OK
verify	Large	168.694	167.600	182.600	3.300	163.800	206.500	OK
withdraw	Tiny	34.694	34.000	40.200	1.575	33.400	61.700	OK
withdraw	Small	34.000	33.200	55.500	1.727	32.300	76.200	OK
withdraw	Medium	32.694	32.650	33.500	0.268	31.900	34.200	OK
withdraw	Large	32.769	32.800	49.500	0.304	31.900	59.700	OK

OracleContract

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
finish	Tiny	30.050	28.750	82.400	2.337	27.900	142.300	OK
finish	Small	35.650	30.400	107.500	12.259	27.400	188.600	OK
finish	Medium	31.694	29.500	51.300	5.174	28.200	157.600	OK
finish	Large	28.313	28.100	46.300	0.814	27.300	87.100	OK
getPrice	Tiny	10.325	10.350	11.100	0.319	9.600	11.400	OK
getPrice	Small	10.106	10.200	10.900	0.295	9.500	11.300	OK
getPrice	Medium	10.081	10.150	10.600	0.213	9.500	10.600	OK
getPrice	Large	10.263	10.100	18.100	1.124	9.300	23.200	OK
request	Tiny	27.169	26.700	41.400	1.322	25.400	72.400	OK
request	Small	30.938	28.600	71.600	5.900	25.900	260.300	OK
request	Medium	25.856	26.000	26.400	0.302	24.900	26.700	OK
request	Large	25.887	25.850	40.500	0.306	25.200	54.100	OK
setPrice	Tiny	34.156	33.800	46.000	1.136	32.100	71.700	OK
setPrice	Small	33.581	33.450	44.200	0.718	32.300	63.900	OK
setPrice	Medium	33.475	33.250	41.300	0.796	32.100	69.700	OK
setPrice	Large	32.981	32.950	34.000	0.410	32.200	41.400	OK
verify	Tiny	14.975	14.850	15.900	0.375	14.300	254.300	OK
verify	Small	14.894	14.600	26.600	0.916	14.000	37.500	OK
verify	Medium	14.163	14.050	14.600	0.237	13.800	14.700	OK
verify	Large	14.588	14.400	25.600	0.685	13.700	419.100	OK

PolicyContract

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
blockAccount	Tiny	34.844	33.750	61.700	2.426	32.800	76.000	OK
blockAccount	Small	33.700	33.500	39.200	1.012	32.400	97.500	OK
blockAccount	Medium	33.206	33.250	35.800	0.567	32.100	67.600	OK
blockAccount	Large	33.244	33.200	34.200	0.332	32.700	35.800	OK
getAttributeFee	Tiny	9.769	9.800	10.200	0.254	9.100	10.300	OK
getAttributeFee	Tiny	9.713	9.800	10.200	0.255	9.100	10.400	OK
getAttributeFee	Small	9.787	9.800	10.400	0.287	9.200	13.300	OK
getAttributeFee	Small	9.706	9.800	10.500	0.251	9.100	10.700	OK
getAttributeFee	Medium	10.019	10.050	11.800	0.477	9.200	185.500	OK
getAttributeFee	Medium	9.850	9.800	18.200	0.621	9.100	26.000	OK
getAttributeFee	Large	9.906	9.800	17.700	0.887	9.100	20.100	OK
getAttributeFee	Large	9.588	9.700	9.800	0.215	9.100	10.000	OK
getBlockedAccounts	Tiny	9.581	9.600	10.300	0.334	9.000	10.700	OK
getBlockedAccounts	Small	9.394	9.500	9.800	0.241	8.900	9.800	OK
getBlockedAccounts	Medium	9.744	9.650	17.900	0.674	9.000	30.300	OK
getBlockedAccounts	Large	9.462	9.500	9.700	0.223	8.900	9.800	OK
getExecFeeFactor	Tiny	9.638	9.750	10.300	0.255	9.100	11.300	OK
getExecFeeFactor	Small	9.569	9.700	9.900	0.217	9.200	10.000	OK
getExecFeeFactor	Medium	9.888	9.850	21.900	0.609	9.100	27.200	OK
getExecFeeFactor	Large	9.612	9.700	9.900	0.274	9.100	11.000	OK
getFeePerByte	Tiny	9.738	9.800	10.400	0.226	9.100	11.000	OK
getFeePerByte	Small	9.625	9.700	9.900	0.244	9.000	10.000	OK
getFeePerByte	Medium	9.756	9.800	10.100	0.265	9.200	11.300	OK
getFeePerByte	Large	9.644	9.700	9.900	0.200	9.100	10.000	OK
getMaxTraceableBlocks	Tiny	9.662	9.700	10.000	0.239	9.200	10.300	OK
getMaxTraceableBlocks	Small	9.662	9.700	17.400	0.276	9.100	26.100	OK
getMaxTraceableBlocks	Medium	9.556	9.700	9.900	0.287	9.000	10.000	OK
getMaxTraceableBlocks	Large	9.500	9.600	9.800	0.226	9.000	9.900	OK
getMaxValidUntilBlockIncrement	Tiny	9.612	9.650	9.900	0.239	9.000	10.000	OK
getMaxValidUntilBlockIncrement	Small	10.181	9.700	16.900	1.479	9.100	23.100	OK
getMaxValidUntilBlockIncrement	Medium	9.606	9.700	9.900	0.219	9.100	9.900	OK
getMaxValidUntilBlockIncrement	Large	9.531	9.600	9.800	0.193	9.100	9.900	OK
getMillisecondsPerBlock	Tiny	9.581	9.700	10.000	0.248	9.100	10.100	OK
getMillisecondsPerBlock	Small	9.975	9.750	17.000	0.554	9.100	26.700	OK
getMillisecondsPerBlock	Medium	9.713	9.800	10.100	0.232	9.200	10.100	OK
getMillisecondsPerBlock	Large	9.675	9.750	9.900	0.189	9.100	9.900	OK
getStoragePrice	Tiny	9.750	9.800	10.300	0.242	9.200	10.600	OK
getStoragePrice	Small	10.088	9.800	16.800	1.188	9.100	28.700	OK
getStoragePrice	Medium	9.562	9.650	10.000	0.239	9.000	10.000	OK
getStoragePrice	Large	9.575	9.600	9.900	0.251	9.000	10.400	OK
isBlocked	Tiny	9.562	9.650	10.000	0.239	9.100	10.100	OK
isBlocked	Small	9.944	9.750	16.400	0.622	9.000	25.500	OK
isBlocked	Medium	9.531	9.600	9.800	0.205	8.900	9.800	OK
isBlocked	Large	9.450	9.500	9.800	0.229	9.000	9.800	OK
setAttributeFee	Tiny	36.531	36.250	43.800	2.572	33.300	45.400	OK
setAttributeFee	Tiny	39.769	35.700	102.400	12.098	33.600	159.700	OK
setAttributeFee	Small	34.056	33.900	36.400	0.698	33.000	73.400	OK
setAttributeFee	Small	34.881	34.250	42.500	1.643	32.900	64.400	OK
setAttributeFee	Medium	34.950	34.900	37.000	0.647	33.600	51.500	OK
setAttributeFee	Medium	33.112	33.200	33.800	0.278	32.400	34.700	OK
setAttributeFee	Large	35.344	35.250	37.400	0.931	33.500	46.400	OK
setAttributeFee	Large	33.744	33.200	49.900	1.586	32.100	53.500	OK
setExecFeeFactor	Tiny	33.475	33.400	35.800	0.496	32.200	36.100	OK
setExecFeeFactor	Small	34.125	33.500	50.200	2.223	32.200	55.100	OK
setExecFeeFactor	Medium	32.950	32.950	34.500	0.495	32.000	37.400	OK
setExecFeeFactor	Large	32.894	33.000	33.500	0.344	31.800	41.700	OK
setFeePerByte	Tiny	34.394	34.400	47.000	1.575	32.000	49.500	OK
setFeePerByte	Small	32.906	33.000	34.600	0.368	32.100	38.300	OK
setFeePerByte	Medium	33.037	32.950	37.400	0.566	32.100	69.100	OK
setFeePerByte	Large	33.950	33.300	84.100	1.480	32.300	5,616.400	OK
setMaxTraceableBlocks	Tiny	33.769	33.500	35.500	0.747	32.500	44.900	OK
setMaxTraceableBlocks	Small	33.363	33.450	34.200	0.359	32.300	40.100	OK
setMaxTraceableBlocks	Medium	33.781	33.150	36.700	1.235	32.500	38.800	OK
setMaxTraceableBlocks	Large	32.906	32.850	35.400	0.288	32.300	38.700	OK
setMaxValidUntilBlockIncrement	Tiny	33.719	33.350	43.400	0.931	32.400	45.900	OK
setMaxValidUntilBlockIncrement	Small	33.369	33.300	35.500	0.553	32.300	39.000	OK
setMaxValidUntilBlockIncrement	Medium	33.262	33.050	35.500	0.594	32.300	69.600	OK
setMaxValidUntilBlockIncrement	Large	33.019	32.950	33.900	0.391	32.100	38.800	OK
setMillisecondsPerBlock	Tiny	34.300	34.050	45.800	0.743	33.000	60.900	OK
setMillisecondsPerBlock	Small	33.456	33.350	35.100	0.310	33.000	39.900	OK
setMillisecondsPerBlock	Medium	33.363	33.300	34.500	0.366	32.500	37.800	OK
setMillisecondsPerBlock	Large	60.062	54.550	118.400	13.452	33.900	182.200	OK
setStoragePrice	Tiny	34.369	34.000	40.500	1.259	32.900	68.400	OK
setStoragePrice	Small	33.013	33.100	33.600	0.278	32.200	33.700	OK
setStoragePrice	Medium	32.944	33.000	33.700	0.250	32.200	43.600	OK
setStoragePrice	Large	33.700	33.250	51.400	1.596	32.100	51.700	OK
unblockAccount	Tiny	33.844	33.800	34.800	0.494	32.300	44.200	OK
unblockAccount	Small	34.013	33.300	40.800	1.828	32.400	66.000	OK
unblockAccount	Medium	33.031	33.050	33.900	0.297	31.800	34.100	OK
unblockAccount	Large	33.462	33.400	48.200	0.461	32.300	64.500	OK

RoleManagement

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
designateAsRole	Tiny	45.625	36.900	162.700	16.293	34.300	196.200	OK
designateAsRole	Small	39.512	36.000	101.200	10.465	34.300	104.200	OK
designateAsRole	Medium	36.975	36.800	45.700	1.668	35.600	63.000	OK
designateAsRole	Large	36.644	36.100	55.100	2.149	35.100	55.400	OK
getDesignatedByRole	Tiny	37.519	37.150	41.600	0.968	21.300	47.600	OK
getDesignatedByRole	Small	37.312	37.100	44.100	0.576	20.700	45.400	OK
getDesignatedByRole	Medium	37.306	37.000	48.500	1.378	36.300	57.300	OK
getDesignatedByRole	Large	37.944	37.900	42.400	0.335	21.200	42.400	OK

StdLib

Method	Profile	Mean (µs)	Median (µs)	P95 (µs)	StdDev (µs)	Min (µs)	Max (µs)	Status
atoi	Tiny	0.869	0.600	2.500	0.491	0.500	4.000	OK
atoi	Tiny	0.988	1.000	1.100	0.048	0.900	2.000	OK
atoi	Small	0.869	0.800	2.100	0.193	0.800	5.900	OK
atoi	Small	1.300	1.300	1.400	0.000	1.300	2.100	OK
atoi	Medium	0.900	0.900	0.900	0.000	0.900	1.600	OK
atoi	Medium	1.400	1.400	1.600	0.100	1.300	2.100	OK
atoi	Large	0.900	0.900	0.900	0.000	0.900	1.400	OK
atoi	Large	1.525	1.500	1.600	0.043	1.400	2.200	OK
base58CheckDecode	Tiny	6.606	6.550	7.300	0.201	6.000	8.400	OK
base58CheckDecode	Small	94.375	93.050	109.700	2.542	91.800	695.000	OK
base58CheckDecode	Medium	4,343.144	4,322.800	4,602.000	71.319	4,191.800	6,139.100	OK
base58CheckDecode	Large	19,045.850	19,865.350	25,485.100	1,391.683	16,396.500	25,916.500	OK
base58CheckEncode	Tiny	6.600	6.500	8.300	0.482	6.000	11.700	OK
base58CheckEncode	Small	134.381	134.500	139.700	2.691	129.900	147.100	OK
base58CheckEncode	Medium	4,023.450	3,969.450	4,343.700	104.357	3,930.700	4,348.200	OK
base58CheckEncode	Large	13,846.938	13,940.550	14,645.200	612.383	12,917.700	15,077.400	OK
base58Decode	Tiny	4.112	4.050	4.800	0.331	3.300	5.800	OK
base58Decode	Small	78.438	77.950	84.400	1.243	76.900	97.300	OK
base58Decode	Medium	4,136.287	4,110.500	4,329.800	75.679	4,005.100	4,367.000	OK
base58Decode	Large	15,935.612	15,904.800	17,101.800	162.462	15,590.900	18,521.100	OK
base58Encode	Tiny	1.306	1.200	1.800	0.175	1.200	3.800	OK
base58Encode	Small	46.838	46.400	50.500	1.281	45.400	64.900	OK
base58Encode	Medium	3,200.119	3,178.150	3,343.100	48.198	3,119.400	3,415.500	OK
base58Encode	Large	13,117.869	13,039.050	13,847.200	261.997	12,789.500	15,719.300	OK
base64Decode	Tiny	0.106	0.100	0.300	0.024	0.100	1.500	OK
base64Decode	Small	0.531	0.500	1.100	0.121	0.500	1.100	OK
base64Decode	Medium	4.056	4.000	4.700	0.177	3.900	4.700	OK
base64Decode	Large	9.425	10.050	10.500	0.961	8.100	10.800	OK
base64Encode	Tiny	0.044	0.000	0.200	0.050	0.000	0.900	OK
base64Encode	Small	0.463	0.400	1.000	0.122	0.400	1.000	OK
base64Encode	Medium	3.737	3.800	4.000	0.154	3.000	15.400	OK
base64Encode	Large	6.038	6.050	6.500	0.285	5.500	6.500	OK
base64UrlDecode	Tiny	0.831	0.800	1.600	0.046	0.800	1.700	OK
base64UrlDecode	Small	8.594	8.600	8.800	0.097	8.500	9.900	OK
base64UrlDecode	Medium	85.219	81.200	97.400	8.935	66.900	97.700	OK
base64UrlDecode	Large	126.806	127.550	140.500	4.741	112.800	175.500	OK
base64UrlEncode	Tiny	0.100	0.100	0.100	0.000	0.100	1.400	OK
base64UrlEncode	Small	0.537	0.400	1.100	0.226	0.400	5.100	OK
base64UrlEncode	Medium	3.188	3.200	3.800	0.296	2.800	4.900	OK
base64UrlEncode	Large	7.375	7.700	8.600	0.791	5.800	8.700	OK
deserialize	Tiny	21.062	19.750	33.700	2.734	18.200	36.900	OK
deserialize	Small	24.938	24.800	52.900	0.630	23.800	54.000	OK
deserialize	Medium	27.431	27.300	29.100	0.510	26.500	38.300	OK
deserialize	Large	28.181	28.200	30.200	0.384	27.100	31.500	OK
hexDecode	Tiny	0.100	0.100	0.800	0.000	0.100	0.900	OK
hexDecode	Small	0.331	0.300	1.000	0.121	0.300	2.200	OK
hexDecode	Medium	2.544	2.700	2.800	0.234	2.000	2.800	OK
hexDecode	Large	5.481	5.100	7.800	0.846	4.800	7.900	OK
hexEncode	Tiny	0.900	0.900	0.900	0.000	0.900	1.000	OK
hexEncode	Small	6.938	6.900	7.500	0.145	6.900	7.600	OK
hexEncode	Medium	59.981	61.850	70.900	2.962	55.500	80.800	OK
hexEncode	Large	117.581	116.950	136.900	4.065	111.000	604.300	OK
itoa	Tiny	0.013	0.000	0.300	0.033	0.000	0.900	OK
itoa	Tiny	0.100	0.100	0.200	0.000	0.100	0.700	OK
itoa	Small	0.113	0.100	0.200	0.033	0.100	3.100	OK
itoa	Small	0.294	0.300	0.300	0.024	0.200	0.600	OK
itoa	Medium	0.300	0.300	0.300	0.000	0.300	1.900	OK
itoa	Medium	0.500	0.500	0.500	0.000	0.400	1.100	OK
itoa	Large	0.700	0.700	0.700	0.000	0.700	1.300	OK
itoa	Large	0.900	0.900	0.900	0.000	0.800	1.500	OK
jsonDeserialize	Tiny	21.744	21.550	24.900	0.809	19.800	30.000	OK
jsonDeserialize	Small	20.225	20.250	23.700	0.581	19.200	44.100	OK
jsonDeserialize	Medium	21.262	21.150	23.900	0.693	20.200	49.800	OK
jsonDeserialize	Large	20.506	20.350	21.400	0.351	19.800	25.600	OK
jsonSerialize	Tiny	20.525	20.300	30.000	0.814	18.800	45.100	OK
jsonSerialize	Small	24.919	24.850	29.400	0.303	23.500	30.500	OK
jsonSerialize	Medium	26.481	26.500	43.700	0.539	25.200	103.400	OK
jsonSerialize	Large	28.362	28.350	34.700	0.863	26.000	46.800	OK
memoryCompare	Tiny	0.013	0.000	0.100	0.033	0.000	0.100	OK
memoryCompare	Small	0.231	0.200	1.100	0.098	0.200	1.900	OK
memoryCompare	Medium	1.888	2.050	2.200	0.271	1.500	2.200	OK
memoryCompare	Large	3.625	3.600	3.700	0.043	3.500	3.700	OK
memorySearch	Tiny	0.100	0.100	0.200	0.000	0.100	1.100	OK
memorySearch	Tiny	0.100	0.100	0.100	0.000	0.100	0.200	OK
memorySearch	Tiny	0.100	0.100	0.200	0.000	0.100	0.500	OK
memorySearch	Small	0.100	0.100	0.100	0.000	0.100	0.100	OK
memorySearch	Small	0.100	0.100	0.100	0.000	0.100	0.700	OK
memorySearch	Small	0.100	0.100	0.100	0.000	0.100	0.700	OK
memorySearch	Medium	0.100	0.100	0.100	0.000	0.100	0.800	OK
memorySearch	Medium	0.100	0.100	0.100	0.000	0.100	0.100	OK
memorySearch	Medium	0.100	0.100	0.100	0.000	0.100	0.100	OK
memorySearch	Large	0.100	0.100	0.100	0.000	0.100	0.100	OK
memorySearch	Large	0.100	0.100	0.100	0.000	0.100	0.700	OK
memorySearch	Large	0.100	0.100	0.100	0.000	0.100	0.600	OK
serialize	Tiny	18.356	18.300	19.200	0.458	17.500	19.400	OK
serialize	Small	18.237	18.100	20.000	0.469	17.300	29.500	OK
serialize	Medium	19.037	19.050	26.200	0.459	17.900	39.500	OK
serialize	Large	19.556	19.600	20.400	0.289	18.700	32.100	OK
strLen	Tiny	0.400	0.400	0.500	0.000	0.400	0.500	OK
strLen	Small	2.719	2.700	3.300	0.073	2.700	21.900	OK
strLen	Medium	21.644	20.100	35.000	3.856	18.200	329.200	OK
strLen	Large	33.413	33.250	35.700	0.632	32.800	38.600	OK
stringSplit	Tiny	0.200	0.200	0.200	0.000	0.200	0.900	OK
stringSplit	Tiny	0.213	0.200	0.300	0.033	0.200	1.100	OK
stringSplit	Small	0.431	0.400	1.100	0.202	0.300	1.200	OK
stringSplit	Small	0.438	0.400	1.000	0.122	0.400	1.100	OK
stringSplit	Medium	2.319	2.300	2.600	0.081	2.100	203.300	OK
stringSplit	Medium	2.562	2.550	4.200	0.099	2.300	12.500	OK
stringSplit	Large	4.412	4.400	4.900	0.086	4.300	5.000	OK
stringSplit	Large	4.319	4.450	4.800	0.354	3.400	5.000	OK

Improve native contract benchmarking harness

7f64cdb

Jim8y marked this pull request as draft November 7, 2025 12:07

Merge branch 'master' into feat/native-bench-html

fb8051f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve native contract benchmarking harness #4284

Improve native contract benchmarking harness #4284

Jim8y commented Nov 7, 2025

Uh oh!

Jim8y commented Nov 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Improve native contract benchmarking harness #4284

Are you sure you want to change the base?

Improve native contract benchmarking harness #4284

Conversation

Jim8y commented Nov 7, 2025

Summary

Testing

Uh oh!

Jim8y commented Nov 7, 2025

Native Contract Benchmarks

ContractManagement

CryptoLib

GasToken

LedgerContract

NeoToken

Notary

OracleContract

PolicyContract

RoleManagement

StdLib

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants