diff --git a/CHANGELOG.md b/CHANGELOG.md index c8e9b259de..115f333366 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,7 +30,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Documentation -- **Regression-history audit** (#840): [scheduled-route evidence and durable archive](docs/benchmarks/840-regression/README.md) verify main/v3 executed/uploaded sets. All 244 native and 26 binding comparison gaps remain; API evidence identifies v3 head replacement when main advances. Separate workflow fixes are proposed in #853/#854. Live notification checks remain open; no runtime or nightly workflow change in this audit. +- **Regression history and reporting verification** (#840): [durable API/run evidence](docs/benchmarks/840-regression/completion.md) establishes numeric comparisons for all 427 native and 26 binding IDs with at least three independent v3 runs each. Separate main/v3 fixes #853/#854 preserve v3 history across measured main advances. Actual scheduled routes, 16 isolated live notification scenarios, four delivered messages and complete synthetic-resource cleanup are verified. This adds no database runtime behavior or Rust tests; further runtime lanes are tracked in #858. - **Manual migration sweep** (#838): [frozen fixtures, timings and correctness evidence](docs/benchmarks/838-migration/README.md) for native v7-to-v8 migration through 200k facts and Chromium/Firefox/WebKit through 20k facts. Includes nonmigrating native read-only open, v8 reopen and bounded post-migration cursor churn; larger browser cases and sustained pin scaling remain excluded. No production or nightly-route change. diff --git a/CLAUDE.md b/CLAUDE.md index 60d798e0c8..bc78d9698c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,12 +10,13 @@ See `ROADMAP.md` for planned work and current direction. See `CHANGELOG.md` for The [#838 manual migration sweep](docs/benchmarks/838-migration/README.md) covers native v7 migration through 200k facts and Chromium/Firefox/WebKit through 20k facts, including bounded post-migration cursor churn. It changes neither nightly coverage nor the Rust test inventory; larger browser cases remain excluded. -The [#840 regression-history audit](docs/benchmarks/840-regression/README.md) -verifies actual main/v3 scheduled benchmark sets and archives their evidence. -All 244 native and 26 binding comparison gaps remain: uploads rebase the shared -v3 Bencher head when main advances. Separate workflow fixes are proposed in -#853/#854; baseline accumulation and live notification delivery remain open. -This audit changes no runtime behavior or Rust test counts. +The [#840 regression-history follow-up](docs/benchmarks/840-regression/completion.md) +verifies numeric comparisons for all 427 native and 26 binding IDs, at least three +independent v3 runs per ID, preserved history after main advances, actual scheduled +routes and isolated live notification delivery/cleanup. Fixes #853/#854 are merged; +checksummed evidence outlives Actions retention. This changes no database runtime +behavior or Rust test counts. Additional JVM/WASI and qualified mobile nightly +lanes are tracked separately in #858. ## Core Philosophy - CRITICAL diff --git a/README.md b/README.md index a9b8c0efac..801cea7cd9 100644 --- a/README.md +++ b/README.md @@ -36,10 +36,11 @@ The unreleased v3.0.0 branch adds bounded sliding windows such as `(avg ?salary The [manual v7-to-v8 migration sweep](docs/benchmarks/838-migration/README.md) (#838) measures native fixtures through 200k facts and browser fixtures through 20k facts, with separate read-only/reopen controls and bounded cursor churn. These measurements do not expand nightly coverage or establish cold-start and sustained-reader bounds. -The [regression-history audit](docs/benchmarks/840-regression/README.md) (#840) -verifies scheduled main/v3 benchmark execution and preserves the raw evidence. -Numeric comparisons for 244 native and 26 binding IDs, and live notification -delivery checks, remain release work; successful uploads alone do not close them. +The [regression-history follow-up](docs/benchmarks/840-regression/completion.md) (#840) +verifies numeric comparisons for all 427 native and 26 binding IDs, independent v3 +history, scheduled main/v3 routes and isolated live notification delivery with +cleanup. Raw evidence and offline verifiers are retained in Git. JVM/WASI and +qualified mobile nightly expansion are tracked separately in #858. On v3.0.0, an ascending value-only query such as `[:find ?v :where [?e :score ?v] :order-by [?v] :limit 10]` can read directly in index order and stop early when all committed `:score` history has one value kind (integer, float, or Boolean) and its source range has no pending writes. A query whose full order cannot be proved uses the existing sort (#641). Built-in `count`, `count-distinct`, `sum`, and `avg` can release completed groups when the pinned source proves their grouping order (#781). `min`/`max` also qualify when their arguments are proved Integer/Float/Boolean/nil; `sum-distinct` requires Integer/nil. A single Integer/Boolean/Float key (including mixed numeric groups) is supported. A longer effective tuple, including hidden `:with` keys, must reduce to one varying Integer/Boolean slot plus repeated occurrences and proved Integer/Boolean/nil constants. Any requested final Sort is retained for `:with`. Small grouped point queries pay proof startup cost; see the [measurements](docs/benchmarks/781-ordered-aggregate.md). Memory follows the active group and batches; distinct members within one group still grow. A limit can stop after enough emitted groups, but dropped min/max groups do not count and a retained Sort drains the input. Callbacks, `collect-set`, aggregate+window queries and unproved arguments keep full-input evaluation. When every built-in window uses the same proved nonconstant partition key, the engine can also finish and release one partition at a time. Its memory follows the largest partition plus bounded batches; one growing partition still grows. Unpartitioned windows, incompatible keys, aggregate queries, callbacks and `collect-set` keep full-input evaluation. A surrounding sort or distinct operation can still require more memory. diff --git a/ROADMAP.md b/ROADMAP.md index bbffb0da04..4541346398 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -130,7 +130,8 @@ Scope — release-gate testing and documentation: - Long-running soak test at scale (#392; done on `v3`: `tests/soak_test.rs`, 10M facts and a 5-hour churn weekly, alternating `main` and `v3`) - Benchmarks at 100K–10M facts and a published performance envelope (#394; on `v3`: `examples/perf_envelope.rs`, results in `docs/BENCHMARKS.md`; follow-ups #528–#532, with #532 superseded by #551's removal of repeated committed-fact net-assert; [audit and measurements](docs/benchmarks/532-net-assert-audit.md)) - Acceptance evidence for memory/scaling (#837): named-test and Linux RSS gates, nonlinear/multistratum recursion and custom chunk bounds; [validation status](docs/benchmarks/837-acceptance-gaps.md) -- Regression-history/reporting evidence (#840): [scheduled-route verification and durable archive](docs/benchmarks/840-regression/README.md). All 244 native and 26 binding comparison gaps remain; the v3 history-rebasing fix awaits separate review (#853/#854), then independent accumulation and live notification checks. +- Regression-history/reporting evidence (#840) is complete: [all 453 numeric comparisons, preserved independent v3 history, scheduled routes and live notification/cleanup evidence](docs/benchmarks/840-regression/completion.md). Separate main/v3 fixes #853/#854 are merged; raw evidence is retained beyond artifact expiry. +- Nightly runtime expansion (#858): add Java/JVM and WASI; qualify Swift/iOS simulator and Android emulator timings against physical-device workloads before enabling representative mobile lanes. The platform restriction from #839 applies only to that manual evidence task. - Manual migration evidence (#838): native v7-to-v8 through 200k facts; Chromium/Firefox/WebKit through 20k facts, with separate read-only/reopen controls and bounded post-migration cursor churn. [Results, frozen fixtures and exclusions](docs/benchmarks/838-migration/README.md); no new nightly route or sustained-pin bound. - Expression keyword/ref equality (#774; on `v3`): resolve literal keywords once per operator, with memory bounded by the expression. [Measurements and regression coverage](docs/benchmarks/774-expression-keywords.md); performance-only, with no v2.x backport. - Higher storage/WAL coverage gates and branch-coverage gates (#396; on `v3`: line and branch floors per file in `coverage-thresholds.toml`, checked on every PR) diff --git a/docs/TEST_COVERAGE.md b/docs/TEST_COVERAGE.md index 78e73b6f27..a8b7b50cdc 100644 --- a/docs/TEST_COVERAGE.md +++ b/docs/TEST_COVERAGE.md @@ -18,11 +18,14 @@ The 2 golden-corpus and 7 migration-matrix tests pass, alongside five standalone Python evidence-integrity tests. These manual probes add no Rust tests to the inventory above and do not establish crash, cold-start or sustained-pin bounds. -**#840 operational benchmark audit**: [offline verification of four Actions runs](benchmarks/840-regression/README.md) -checks exact selected/executed/uploaded/API sets, hashes and 0.999/30 models. -Twelve fixture-based notification-policy checks pass, but do not prove live -delivery. The 244 native and 26 binding baseline gaps remain. No Rust tests are -added and the inventory above is unchanged. +**#840 operational benchmark verification**: [the completed follow-up](benchmarks/840-regression/completion.md) +checks all 453 numeric comparisons, at least three independent v3 runs per ID, +exact selected/executed/uploaded/API sets, source hashes and 0.999/30 models. +The initial four-run schedule archive and five new measurement runs are retained. +Sixteen isolated live notification scenarios verify four actual deliveries and +cleanup of all 46 synthetic resources. Fresh extraction reproduces the evidence +reports, and removing delivery/comparison evidence is rejected. No Rust tests +are added; the inventory above is unchanged. Additional runtime lanes are #858. **#794 verification (2026-10-30)**: one added `storage::keys` unit test checks that a set's element encoding equals its elements' encodings, ordered and deduplicated by a `BTreeSet`, over 2,000 random sets of up to about 60 elements with duplicates, signed zeros, NaNs and strings up to 300 bytes, in shuffled, sorted and reversed order. It passes with both the old stable sort and the new unstable sort. The full default suite passed before merging #795 (2,038 active, 19 ignored). CI-equivalent native, browser and WASI clippy pass on Rust 1.99.1, and Rust 1.89 checks all features. The release binary is 1,260,568 bytes (3,024 smaller), below the unchanged 1,263,616-byte cap. diff --git a/docs/bench-coverage-v3.md b/docs/bench-coverage-v3.md index 6a25be5410..2cee694282 100644 --- a/docs/bench-coverage-v3.md +++ b/docs/bench-coverage-v3.md @@ -1,11 +1,13 @@ # v3 nightly benchmark coverage close-out (#776) -Follow-up on 2026-11-05: the [#840 operational audit](benchmarks/840-regression/README.md) -verifies actual final main/v3 scheduled routes and archives their evidence beyond -30 days. The same 244 native and 26 binding baselines remain null: native uploads -replace the shared v3 Bencher head when main advances. Separate fixes #853/#854 -await review; independent baseline accumulation and live notification checks -remain open. The original validation snapshot below is retained unchanged. +Follow-up on 2026-11-05: [#840's completed verification](benchmarks/840-regression/completion.md) +establishes numeric comparisons for all 427 native and 26 binding IDs with at +least three independent v3 runs each. Merged fixes #853/#854 preserve the v3 head +after a measured main advance. Actual scheduled routes, isolated live notification +delivery and cleanup are verified; complete evidence is archived beyond 30 days. +JVM/WASI and qualified iOS-simulator/Android-emulator expansion are tracked in +#858. The original validation snapshot below is retained unchanged; its null +baselines and X05 open status describe that earlier snapshot. Audit date: 2026-11-03. P01–P17 are merged. This closes out the implementation plan and gives every matrix row a disposition; it does **not** close diff --git a/docs/benchmarks/840-regression/.gitignore b/docs/benchmarks/840-regression/.gitignore index ba62b7c24f..b24b8f30ef 100644 --- a/docs/benchmarks/840-regression/.gitignore +++ b/docs/benchmarks/840-regression/.gitignore @@ -1,2 +1,4 @@ /evidence/ __pycache__/ +/live-evidence/ +/completion-evidence/ diff --git a/docs/benchmarks/840-regression/README.md b/docs/benchmarks/840-regression/README.md index 6d0f48fcb3..1411d2142d 100644 --- a/docs/benchmarks/840-regression/README.md +++ b/docs/benchmarks/840-regression/README.md @@ -1,17 +1,22 @@ # Regression history and scheduled reporting audit -Audit date: 2026-11-05. [Issue #840](https://github.com/project-minigraf/minigraf/issues/840) -remains open. Actual main/v3 scheduled routes are verified, but the same **244 -native and 26 binding IDs still lack numeric comparisons**. Native uploads -rebase the shared v3 Bencher head when main advances, preventing those histories -from accumulating reliably. The proposed fix has separate draft reviews for +Initial snapshot: 2026-11-05, before the history fixes were merged. The +[follow-up report](completion.md) records subsequent real measurements, live +notification delivery and cleanup. The findings and acceptance table below are +retained as the initial state, not a current status report for +[issue #840](https://github.com/project-minigraf/minigraf/issues/840). + +At this snapshot, actual main/v3 scheduled routes were verified, but **244 +native and 26 binding IDs still lacked numeric comparisons**. Native uploads +rebased the shared v3 Bencher head when main advanced, preventing those histories +from accumulating reliably. Separate fixes were proposed in [main #853](https://github.com/project-minigraf/minigraf/pull/853) and [v3 #854](https://github.com/project-minigraf/minigraf/pull/854). This is operational evidence under PHILOSOPHY.md's reliability and documentation principles. No benchmark was built, enumerated or timed locally. The observations -come from existing Actions runs. The proposed workflow fix has not been merged; -no live notification has been sent by this audit. +come from existing Actions runs. The fixes and live notifications had not yet +been executed at the time of this initial snapshot. ## Scheduled routes and exact coverage @@ -146,7 +151,7 @@ requests. Preserve the existing archive before refreshing; a newer active head or additional observations can legitimately invalidate this frozen audit's assertions. Repack and regenerate checksums only after reviewing the new results. -## Acceptance status +## Acceptance status at the initial snapshot | Requirement | Status | | --- | --- | diff --git a/docs/benchmarks/840-regression/SHA256SUMS b/docs/benchmarks/840-regression/SHA256SUMS index d617558034..7af9c55657 100644 --- a/docs/benchmarks/840-regression/SHA256SUMS +++ b/docs/benchmarks/840-regression/SHA256SUMS @@ -1,13 +1,25 @@ -c623b8b7d4c1591427747a620c2ec8276212a95a4b9e85a62f83dd17f38ff3b8 .gitignore -f0e9c92691729f598785e5d6abf05ff509180c62e04ae8c620d1e66571626a81 README.md +18a2d252eba0e733230ca84d2a83e319a362be15eaa06b35a119dc4aeb8f570a .gitignore +1001894e5dc6edbf9ce3b3b0c940e718fcbca33254b0c3d07179e23ff87eb059 README.md c0883999ce578ada4818280db12e6a8ef80f402aac6c0eda310ccf6b971bd4a0 active-head-summary.json -f4a21622ccfdb56aa30ecb91c07df14e8a578c23872234ad24b740e104adfb39 audit.py +52a31196b90142b1611543b8cd2ad6b5ab7647ed03c4a6f8478048b8963fcf1c archive-evidence.py +9d4db25ce636ab7c5edb7050706cdea4aab0a74257f5e24b9e151f4982d3b882 audit.py +584ae79c5319d545fcedb6e93f3cd1f59b49e1e33d942cd60172f6d9d79ac6bb collect-completion.py 9c486d21e2301f47bc25309259916d8bc93803a6b21227fcd4e77ea281f0d400 collect.py +67b13cb713cf3e7a53ce2ab7f1d502a0be2037fc76dfd41edb4c21056100562d comparison-inventory.json +7d647fca38cbf2449061b5e05bca67cd32696dcce1556c2179785d025ff84134 completion-summary.json +fa78970117547ae11538bcf362453acdda49a95e0ae700a6fbaf4b805c67b518 completion-verification.txt +004b7b68d9b625da260934d4bf384b869d7c9e7bb5f454c68a60f8889f30a0cd completion.md +d35246a3b3783b8cb7522847169299e534c33a3296175e7e87c46dcbc1639168 completion.tar.gz 9151da470a9aaefae263534cd8b2aefac4efff3363841f6e063ef490644b880a evidence.tar.gz 3b2e46689dd84b00e52479cda19b773101dd2bcb7f17ce0e53f6932f658eb386 inventory.json -5aaef5dfc1166053dc09df5db17dd31ac53e4e1a2293306813d68cf5018ed9d8 live-checks.md +24fc5ef735cda3ff2e7c1bdbea4597745ff7cf0ac2148c2afb255c31cf23f31c live-checks.md +3ec9c24bd9023471adc2af0fdb919cf0e18d572699014352b3466a39eeeb8deb live-notifications.tar.gz +2fa1462c2b6b6b064e52d1ba89e97481d698793df3824ae94e8fa2eefd905643 live-summary.json +dd4bdda3955548d891b699c07989efa1c83b2f5fe47e48a6620024349ec32064 live-verification.txt c7026121a1c294999504f4356cd1480a520fa96b329d7417414a03876cdab35f missing-baselines.json 2717e5411af7c9fbd139406a311fb1732f4e1b3e2585005c497634078015bb63 notification-preview.cjs 01e4b1cbd805b0ae24c0b9cae1d5d8a4a859ed65252ebc1828119d2ee868797c notification-preview.json b0f545189c37135338b29ff1e1c509aaaf9b5dda27c296511a6100e1d02fc3bf summary.json 59e0e8685d879ba82731f46ef2466cc6da7e466254eaa881105e94bd26862f70 verification.txt +49fdc560d01c4b69c7ee8ecb664e5433cc105146f3f3659b1ce1dcc7f2ea5f5c verify-completion.py +85ff1dd61608d395b02a8f5a1fd095778aaf71f60fa0bfd391684f9dd53482d4 verify-live.py diff --git a/docs/benchmarks/840-regression/archive-evidence.py b/docs/benchmarks/840-regression/archive-evidence.py new file mode 100644 index 0000000000..825895855d --- /dev/null +++ b/docs/benchmarks/840-regression/archive-evidence.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python3 +"""Deterministically package one #840 follow-up evidence directory.""" +import gzip +import hashlib +from pathlib import Path +import sys +import tarfile + +root = Path(__file__).resolve().parent +name = sys.argv[1] +output = {'live-evidence': 'live-notifications.tar.gz', 'completion-evidence': 'completion.tar.gz'}[name] +source = root / name +files = sorted(p for p in source.rglob('*') if p.is_file() and p.name != 'SHA256SUMS') +assert files and all(not p.is_symlink() for p in files) +(source / 'SHA256SUMS').write_text(''.join( + hashlib.sha256(p.read_bytes()).hexdigest() + ' ' + str(p.relative_to(source)) + '\n' for p in files)) +with (root / output).open('wb') as raw: + with gzip.GzipFile(filename='', fileobj=raw, mode='wb', mtime=0) as compressed: + with tarfile.open(fileobj=compressed, mode='w') as archive: + for p in sorted(source.rglob('*')): + if p.is_file(): + info = archive.gettarinfo(str(p), arcname=str(p.relative_to(root))) + info.uid = info.gid = 0 + info.uname = info.gname = '' + info.mtime = 0 + with p.open('rb') as stream: + archive.addfile(info, stream) +files = sorted(p for p in root.iterdir() if p.is_file() and p.name != 'SHA256SUMS') +(root / 'SHA256SUMS').write_text(''.join( + hashlib.sha256(p.read_bytes()).hexdigest() + ' ' + p.name + '\n' for p in files)) +print(output, (root / output).stat().st_size, 'bytes') diff --git a/docs/benchmarks/840-regression/audit.py b/docs/benchmarks/840-regression/audit.py index 435965a1c4..218b4e21d3 100644 --- a/docs/benchmarks/840-regression/audit.py +++ b/docs/benchmarks/840-regression/audit.py @@ -7,6 +7,9 @@ import re from urllib.parse import parse_qs, urlparse +if not __debug__: + raise RuntimeError('Evidence verification requires Python assertions; do not use -O') + ROOT = Path(__file__).resolve().parent RUNS = (41642356471, 41711848743, 41801278030, 41908407297) diff --git a/docs/benchmarks/840-regression/collect-completion.py b/docs/benchmarks/840-regression/collect-completion.py new file mode 100644 index 0000000000..3af8ebf713 --- /dev/null +++ b/docs/benchmarks/840-regression/collect-completion.py @@ -0,0 +1,128 @@ +#!/usr/bin/env python3 +"""Collect approved #840 real-run evidence. No public API writes.""" +import concurrent.futures +from datetime import datetime +import json +from pathlib import Path +import re +import shutil +import subprocess +import sys +import tarfile +from urllib.parse import parse_qs, urlparse, urlencode +from urllib.request import urlopen + +ROOT = Path(__file__).resolve().parent +OUT = ROOT / 'completion-evidence' +REPO = 'project-minigraf/minigraf' + + +def gh(args): + return subprocess.check_output(['gh'] + args) + + +def get(url, path): + if not path.exists(): + with urlopen(url, timeout=60) as response: + data = response.read() + json.loads(data) + path.write_bytes(data) + return json.loads(path.read_text()) + + +def collect_run(item): + run = item['run'] + folder = OUT / str(run) + folder.mkdir(exist_ok=True) + if not (folder / 'downloaded').exists(): + metadata = gh(['api', f'repos/{REPO}/actions/runs/{run}']) + assert json.loads(metadata)['status'] == 'completed' + (folder / 'run.json').write_bytes(metadata) + for kind in ('jobs', 'artifacts'): + (folder / f'{kind}.json').write_bytes(gh(['api', '--paginate', '--slurp', f'repos/{REPO}/actions/runs/{run}/{kind}?per_page=100'])) + subprocess.run(['gh', 'run', 'download', str(run), '--repo', REPO, + '--pattern', 'bench-evidence-*', '--pattern', 'binding-evidence-*', + '--pattern', 'bench-enumeration', '--dir', str(folder / 'artifacts')], check=True) + (folder / 'downloaded').write_text('Complete artifact download\n') + metadata = json.loads((folder / 'run.json').read_text()) + source = folder / 'workflow.yml' + if not source.exists(): + source.write_bytes(gh(['api', '--header', 'Accept: application/vnd.github.raw+json', + f'repos/{REPO}/contents/.github/workflows/bench.yml?ref={metadata["head_sha"]}'])) + jobs = [job for page in json.loads((folder / 'jobs.json').read_text()) for job in page['jobs']] + for job in jobs: + if any(step['name'] == 'Report regression' and step['conclusion'] == 'success' for step in job['steps']): + log = folder / f'notification-job-{job["id"]}.txt' + if not log.exists(): + # Preserve raw bytes in a file; never render terminal control sequences. + log.write_bytes(gh(['api', '--allow-escape-sequences', f'repos/{REPO}/actions/jobs/{job["id"]}/logs'])) + (folder / 'api').mkdir(exist_ok=True) + for log in list(folder.glob('artifacts/*/bencher_out.txt')) + list(folder.glob('artifacts/*/binding-bencher.txt')): + urls = re.findall(r'https://bencher.dev/console/projects/minigraf/perf\?\S+', log.read_text()) + reports = {parse_qs(urlparse(url).query)['report'][0] for url in urls} + assert len(reports) == 1, log + report = reports.pop() + get(f'https://api.bencher.dev/v0/projects/minigraf/reports/{report}', folder / 'api' / f'{report}.json') + print('Collected', item, flush=True) + + +def main(): + OUT.mkdir(exist_ok=True) + ledger = Path(sys.argv[1]) + plan = json.loads(ledger.read_text()) + (OUT / 'runs.json').write_text(json.dumps(plan, indent=2) + '\n') + with concurrent.futures.ThreadPoolExecutor(max_workers=3) as pool: + list(pool.map(collect_run, plan)) + active = OUT / 'active-head' + active.mkdir(exist_ok=True) + branch = get('https://api.bencher.dev/v0/projects/minigraf/branches/v3', active / 'branch.json') + assert branch['head']['uuid'] == '01a1a84c-e947-745c-98e7-018e0c2245a3', 'v3 head was unexpectedly replaced' + # A report's CLI start can precede the head it creates. Use the earliest + # actual v3 report start, not the head creation timestamp; verify provenance + # separately so inherited main observations cannot count as independent v3. + original = json.loads((ROOT / 'inventory.json').read_text())['41908407297'] + prior_ids = {v['report'] for v in original.values() if v['head'] == branch['head']['uuid']} + with tarfile.open(ROOT / 'evidence.tar.gz') as archive: + prior = [json.load(archive.extractfile(f'evidence/41908407297/api/{report}.json')) for report in prior_ids] + assert all(r['branch']['name'] == 'v3' and r['branch']['head']['uuid'] == branch['head']['uuid'] for r in prior) + earliest = min(r['start_time'] for r in prior) + start = int(datetime.fromisoformat(earliest.replace('Z', '+00:00')).timestamp() * 1000) + (active / 'window.json').write_text(json.dumps(dict(head_created=branch['head']['created'], + earliest_actual_v3_report_start=earliest, start_time_ms=start), indent=2) + '\n') + groups = {} + for item in plan: + if item['role'] != 'full-v3': + continue + for file in (OUT / str(item['run']) / 'api').glob('*.json'): + report = json.loads(file.read_text()) + assert report['branch']['name'] == 'v3' + for iteration in report['results']: + for result in iteration: + for measure in result['measures']: + key = (report['testbed']['uuid'], measure['measure']['uuid']) + groups.setdefault(key, set()).add(result['benchmark']['uuid']) + queries = [] + for (testbed, measure), ids in sorted(groups.items()): + ids = sorted(ids) + for offset in range(0, len(ids), 8): + query = dict(branches=branch['uuid'], heads=branch['head']['uuid'], testbeds=testbed, + measures=measure, benchmarks=','.join(ids[offset:offset + 8]), start_time=start) + queries.append('https://api.bencher.dev/v0/projects/minigraf/perf?' + urlencode(query)) + (active / 'queries.json').write_text(json.dumps(queries, indent=2) + '\n') + with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool: + list(pool.map(lambda pair: get(pair[1], active / f'{pair[0]:02d}.json'), enumerate(queries))) + probes = OUT / 'head-probes' + probes.mkdir(exist_ok=True) + for role in ('c-probe', 'main-probe', 'v3-probe'): + for stage in ('before', 'after'): + filename = f'{stage}-{role}.json' + shutil.copyfile(ledger.parent / filename, probes / filename) + orchestration = OUT / 'orchestration' + orchestration.mkdir(exist_ok=True) + for filename in ('accumulate.py', 'probes.py'): + shutil.copyfile(ledger.parent / filename, orchestration / filename) + print('Collected active-head comparisons for', sum(map(len, groups.values())), 'IDs') + + +if __name__ == '__main__': + main() diff --git a/docs/benchmarks/840-regression/comparison-inventory.json b/docs/benchmarks/840-regression/comparison-inventory.json new file mode 100644 index 0000000000..9dfd552011 --- /dev/null +++ b/docs/benchmarks/840-regression/comparison-inventory.json @@ -0,0 +1,5901 @@ +{ + "aggregation/count_distinct_scale/10k": { + "baseline": 19689356.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 34814606.860637486, + "value": 9788100.0 + }, + "aggregation/count_distinct_scale/1k": { + "baseline": 1781493.3333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3161981.1141229686, + "value": 1006400.0 + }, + "aggregation/count_scale/10k": { + "baseline": 19556210.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 35821799.10953746, + "value": 8630200.0 + }, + "aggregation/count_scale/1k": { + "baseline": 1595489.3333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3014543.836106753, + "value": 869370.0 + }, + "aggregation/grouped_count_scale/10k": { + "baseline": 23255426.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 42605256.47085299, + "value": 9910600.0 + }, + "aggregation/grouped_count_scale/1k": { + "baseline": 1981023.3333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3590245.030125015, + "value": 1019400.0000000001 + }, + "aggregation/sum_scale/10k": { + "baseline": 18722376.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 33718049.325273156, + "value": 8714500.0 + }, + "aggregation/sum_scale/1k": { + "baseline": 1720661.6666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3121702.3782427697, + "value": 896560.0 + }, + "aggregation/v3_controls/hash_avg/10k": { + "baseline": 8986800.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 56952256.10104735, + "value": 8986900.0 + }, + "aggregation/v3_controls/hash_exact_float_sum/10k": { + "baseline": 9003600.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 79904262.69649069, + "value": 8909900.0 + }, + "aggregation/v3_controls/hash_mixed_extrema/10k": { + "baseline": 9817150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 54557342.67356314, + "value": 9761600.0 + }, + "aggregation/v3_controls/hash_mixed_sum_distinct/10k": { + "baseline": 9706600.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 55301349.65007605, + "value": 9731500.0 + }, + "aggregation/v3_controls/skew_avg_committed/100k": { + "baseline": 48144000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 383350865.62571025, + "value": 47808000.0 + }, + "aggregation/v3_controls/skew_count_committed/100k": { + "baseline": 46850500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 201497746.39475614, + "value": 46846000.0 + }, + "aggregation/v3_controls/skew_count_pending/100k": { + "baseline": 129600000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4766260756.434577, + "value": 121780000.0 + }, + "aggregation/v3_controls/tuple_float_constant_fallback/100k": { + "baseline": 59024500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 330001760.6168365, + "value": 58783000.0 + }, + "aggregation/v3_controls/tuple_hidden_with_fallback/100k": { + "baseline": 510050000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6668373501.709183, + "value": 501240000.0 + }, + "aggregation/v3_controls/tuple_repeated_constant/100k": { + "baseline": 61007000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 193325499.58909613, + "value": 61311000.0 + }, + "aggregation/with_grouped_sum/10k": { + "baseline": 88916733.33333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 149716507.8526269, + "value": 48971000.0 + }, + "aggregation/with_grouped_sum/1k": { + "baseline": 6603086.666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11395375.512682967, + "value": 4333400.0 + }, + "binding/browser/aggregate/1k": { + "baseline": 1561249.9999999271, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 123542366.80897866, + "value": 1812500.0 + }, + "binding/browser/cursor_batch_1/1k": { + "baseline": 1232500.0000004366, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 136307634.9970423, + "value": 1429999.9999995634 + }, + "binding/browser/cursor_batch_256/1k": { + "baseline": 573750.000000291, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 66043840.94252181, + "value": 684999.9999998545 + }, + "binding/browser/indexeddb_reopen/1k": { + "baseline": 2324999.9999970896, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 181506301.5252935, + "value": 2699999.9999970896 + }, + "binding/browser/indexeddb_write/1k": { + "baseline": 7000000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 337796248.9759495, + "value": 7849999.999991269 + }, + "binding/browser/predicate/1k": { + "baseline": 1780000.000000291, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 146503358.92557418, + "value": 2084999.9999998545 + }, + "binding/browser/prepared_execute/1k": { + "baseline": 111249.99999992724, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 13205268.188464118, + "value": 160000.00000058208 + }, + "binding/browser/set_roundtrip/2": { + "baseline": 59999.99999985448, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-browser-ubuntu24-v1", + "upper_limit": 5573270.816131966, + "value": 82499.99999970896 + }, + "binding/c/aggregate/1k": { + "baseline": 598971.75, + "independent_runs": [ + 41960942244, + 41965724402, + 41970954512 + ], + "report": "01a1aa14-152a-74d1-8c9a-591746bd353a", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-c-ubuntu24-v1", + "upper_limit": 14133086.781408457, + "value": 449278.0 + }, + "binding/c/cursor_batch_1/1k": { + "baseline": 780590.5, + "independent_runs": [ + 41960942244, + 41965724402, + 41970954512 + ], + "report": "01a1aa14-152a-74d1-8c9a-591746bd353a", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-c-ubuntu24-v1", + "upper_limit": 9591348.591388939, + "value": 573590.5 + }, + "binding/c/cursor_batch_256/1k": { + "baseline": 704413.0, + "independent_runs": [ + 41960942244, + 41965724402, + 41970954512 + ], + "report": "01a1aa14-152a-74d1-8c9a-591746bd353a", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-c-ubuntu24-v1", + "upper_limit": 8663370.750284132, + "value": 507792.0 + }, + "binding/c/predicate/1k": { + "baseline": 778897.5, + "independent_runs": [ + 41960942244, + 41965724402, + 41970954512 + ], + "report": "01a1aa14-152a-74d1-8c9a-591746bd353a", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-c-ubuntu24-v1", + "upper_limit": 5470415.3010558905, + "value": 566830.0 + }, + "binding/c/prepared_execute/1k": { + "baseline": 243786.75, + "independent_runs": [ + 41960942244, + 41965724402, + 41970954512 + ], + "report": "01a1aa14-152a-74d1-8c9a-591746bd353a", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-c-ubuntu24-v1", + "upper_limit": 4319058.705573755, + "value": 154578.5 + }, + "binding/c/set_roundtrip/2": { + "baseline": 10483.5, + "independent_runs": [ + 41960942244, + 41965724402, + 41970954512 + ], + "report": "01a1aa14-152a-74d1-8c9a-591746bd353a", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-c-ubuntu24-v1", + "upper_limit": 1063793.8894373674, + "value": 6194.0 + }, + "binding/node/aggregate/1k": { + "baseline": 1822762.2124999983, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e6-37a1-7475-a2e6-402006843f6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-node-ubuntu24-v1", + "upper_limit": 113811854.41631638, + "value": 1188005.825000002 + }, + "binding/node/cursor_batch_1/1k": { + "baseline": 1412191.7125000008, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e6-37a1-7475-a2e6-402006843f6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-node-ubuntu24-v1", + "upper_limit": 89849500.67914182, + "value": 869541.3250000002 + }, + "binding/node/cursor_batch_256/1k": { + "baseline": 947436.6374999989, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e6-37a1-7475-a2e6-402006843f6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-node-ubuntu24-v1", + "upper_limit": 45877664.86450887, + "value": 590473.6250000007 + }, + "binding/node/predicate/1k": { + "baseline": 1952348.8249999986, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e6-37a1-7475-a2e6-402006843f6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-node-ubuntu24-v1", + "upper_limit": 119814252.90971933, + "value": 1223864.999999998 + }, + "binding/node/prepared_execute/1k": { + "baseline": 288239.6875000001, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e6-37a1-7475-a2e6-402006843f6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-node-ubuntu24-v1", + "upper_limit": 14811180.514561798, + "value": 178523.44999999978 + }, + "binding/node/set_roundtrip/2": { + "baseline": 19593.19999999991, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e6-37a1-7475-a2e6-402006843f6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-node-ubuntu24-v1", + "upper_limit": 3334943.472622972, + "value": 9528.899999999396 + }, + "binding/python/aggregate/1k": { + "baseline": 8105848.050000001, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7388-700a-aeb7-13730e791aa3", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-python-ubuntu24-v1", + "upper_limit": 66425902.11956917, + "value": 8269807.6 + }, + "binding/python/cursor_batch_1/1k": { + "baseline": 14052281.325, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7388-700a-aeb7-13730e791aa3", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-python-ubuntu24-v1", + "upper_limit": 207567714.8282726, + "value": 14384134.425 + }, + "binding/python/cursor_batch_256/1k": { + "baseline": 1157803.1625, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7388-700a-aeb7-13730e791aa3", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-python-ubuntu24-v1", + "upper_limit": 6318534.767958162, + "value": 1142358.95 + }, + "binding/python/predicate/1k": { + "baseline": 8791105.962499999, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7388-700a-aeb7-13730e791aa3", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-python-ubuntu24-v1", + "upper_limit": 72263428.76682438, + "value": 8918386.875 + }, + "binding/python/prepared_execute/1k": { + "baseline": 381252.9875, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7388-700a-aeb7-13730e791aa3", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-python-ubuntu24-v1", + "upper_limit": 2138022.2751434604, + "value": 378179.525 + }, + "binding/python/set_roundtrip/2": { + "baseline": 90964.6125, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7388-700a-aeb7-13730e791aa3", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "binding-python-ubuntu24-v1", + "upper_limit": 8776095.936753022, + "value": 102068.375 + }, + "checkpoint/100k": { + "baseline": 1631976393.3333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 5544293594.980179, + "value": 327050000.0 + }, + "checkpoint/10k": { + "baseline": 167330236.66666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 600183331.5605175, + "value": 27502000.0 + }, + "checkpoint/1k": { + "baseline": 18903826.333333332, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 75704101.28214636, + "value": 3390500.0 + }, + "checkpoint/1m": { + "baseline": 18354605366.666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 59577266608.01547, + "value": 4341400000.0 + }, + "checkpoint/after_100k_facts/100k": { + "baseline": 1156250000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 68666251144.52009, + "value": 1229800000.0 + }, + "checkpoint/after_100k_facts/10k": { + "baseline": 1181900000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 35750108017.65137, + "value": 1196600000.0 + }, + "checkpoint/after_1_fact/100k": { + "baseline": 70047089.66666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 202412324.15514815, + "value": 1196000.0 + }, + "checkpoint/after_1_fact/10k": { + "baseline": 6718828.666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 16207916.33002074, + "value": 1066400.0 + }, + "checkpoint/after_1_fact/1m": { + "baseline": 1206600.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 39082770.50737877, + "value": 1210600.0 + }, + "checkpoint/set_storage/inline_unique/10k": { + "baseline": 29479500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2914849781.6647277, + "value": 35008000.0 + }, + "checkpoint/set_storage/long_repeated/10k": { + "baseline": 77530000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4313375968.09594, + "value": 85299000.0 + }, + "checkpoint/set_storage/long_unique/10k": { + "baseline": 97835000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2973005730.654735, + "value": 100550000.0 + }, + "checkpoint/storage_v3/multivalue_inline_unique/10k": { + "baseline": 21715000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2548447015.070115, + "value": 24474000.0 + }, + "checkpoint/storage_v3/multivalue_long_repeated/10k": { + "baseline": 31906500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3312026972.1055317, + "value": 35990000.0 + }, + "checkpoint/storage_v3/multivalue_long_unique/10k": { + "baseline": 54552500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 809594938.2802799, + "value": 54090000.0 + }, + "checkpoint/storage_v3/rebuild_damaged_avet/100k_history": { + "baseline": 397545000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 31103706810.894623, + "value": 426500000.0 + }, + "checkpoint/storage_v3/verify/100k_history": { + "baseline": 188515000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 25905166722.222458, + "value": 219300000.0 + }, + "concurrent/readers/100k_16t": { + "baseline": 41947.63333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f32e-775a-ac89-4dfadf7b54ff", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 75094.0576485576, + "value": 61234.0 + }, + "concurrent/readers/100k_4t": { + "baseline": 10580.633333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f32e-775a-ac89-4dfadf7b54ff", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 18800.37316221075, + "value": 15369.0 + }, + "concurrent/readers/100k_8t": { + "baseline": 21329.8, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f32e-775a-ac89-4dfadf7b54ff", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 37882.96362972396, + "value": 30902.0 + }, + "concurrent/readers/10k_16t": { + "baseline": 41827.73333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f32e-775a-ac89-4dfadf7b54ff", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 74162.52906474959, + "value": 60961.0 + }, + "concurrent/readers/10k_4t": { + "baseline": 10766.763333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f32e-775a-ac89-4dfadf7b54ff", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 19006.093814053223, + "value": 15376.0 + }, + "concurrent/readers/10k_8t": { + "baseline": 21070.866666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f32e-775a-ac89-4dfadf7b54ff", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 37804.6467525075, + "value": 31884.0 + }, + "concurrent/readers_plus_writer/100k_16t": { + "baseline": 46313.9, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7323-7295-8c04-2ca4f4e1f3c4", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 76383.83943959423, + "value": 32828.0 + }, + "concurrent/readers_plus_writer/100k_4t": { + "baseline": 14392.880000000001, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7323-7295-8c04-2ca4f4e1f3c4", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 24763.3270348016, + "value": 8434.2 + }, + "concurrent/readers_plus_writer/100k_8t": { + "baseline": 24557.033333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7323-7295-8c04-2ca4f4e1f3c4", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 40763.42020364579, + "value": 16458.0 + }, + "concurrent/readers_plus_writer/10k_16t": { + "baseline": 45020.23333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7323-7295-8c04-2ca4f4e1f3c4", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 75150.55125837342, + "value": 31235.0 + }, + "concurrent/readers_plus_writer/10k_4t": { + "baseline": 13906.689999999999, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7323-7295-8c04-2ca4f4e1f3c4", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 23488.453016149255, + "value": 8354.2 + }, + "concurrent/readers_plus_writer/10k_8t": { + "baseline": 23416.533333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-7323-7295-8c04-2ca4f4e1f3c4", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 38301.386788933356, + "value": 15937.0 + }, + "concurrent/serialized_writers/100k_16t": { + "baseline": 116441.93333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 209779.0893550347, + "value": 43932.0 + }, + "concurrent/serialized_writers/100k_2t": { + "baseline": 12183.236666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21756.15549767561, + "value": 4582.200000000001 + }, + "concurrent/serialized_writers/100k_4t": { + "baseline": 31238.933333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 58203.44381294352, + "value": 13880.0 + }, + "concurrent/serialized_writers/100k_8t": { + "baseline": 58466.933333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 106083.08688384661, + "value": 21655.0 + }, + "concurrent/serialized_writers/10k_16t": { + "baseline": 113817.2, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 205909.61083169412, + "value": 43304.0 + }, + "concurrent/serialized_writers/10k_2t": { + "baseline": 11987.983333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21870.24479614888, + "value": 4499.1 + }, + "concurrent/serialized_writers/10k_4t": { + "baseline": 30274.466666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 56669.70693311983, + "value": 13698.0 + }, + "concurrent/serialized_writers/10k_8t": { + "baseline": 56778.26666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-f464-7225-ab41-0405bf038929", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 103467.1412517601, + "value": 21073.0 + }, + "concurrent_btree_scan/100k_2t": { + "baseline": 22663.9, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-08d5-7106-b75c-a4d98467b778", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 38951.49776983736, + "value": 19284.0 + }, + "concurrent_btree_scan/100k_4t": { + "baseline": 35511.96666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-08d5-7106-b75c-a4d98467b778", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 64951.673513388654, + "value": 27388.0 + }, + "concurrent_btree_scan/100k_8t": { + "baseline": 71383.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-08d5-7106-b75c-a4d98467b778", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 131405.89396869644, + "value": 54496.0 + }, + "concurrent_btree_scan/10k_2t": { + "baseline": 24861.033333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-08d5-7106-b75c-a4d98467b778", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 43475.84479695286, + "value": 19322.0 + }, + "concurrent_btree_scan/10k_4t": { + "baseline": 38270.86666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-08d5-7106-b75c-a4d98467b778", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 71360.28878518763, + "value": 27375.0 + }, + "concurrent_btree_scan/10k_8t": { + "baseline": 77280.2, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-08d5-7106-b75c-a4d98467b778", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 144770.1454135114, + "value": 54300.0 + }, + "concurrent_file/readers/100k_16t": { + "baseline": 142044.1, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-1534-76fb-bf8c-6aa8ecd85188", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 244325.45278981832, + "value": 122730.0 + }, + "concurrent_file/readers/100k_4t": { + "baseline": 35677.53333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-1534-76fb-bf8c-6aa8ecd85188", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 60652.00594026522, + "value": 30963.0 + }, + "concurrent_file/readers/100k_8t": { + "baseline": 71851.13333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-1534-76fb-bf8c-6aa8ecd85188", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 121910.48065051393, + "value": 62356.0 + }, + "concurrent_file/readers/10k_16t": { + "baseline": 153349.93333333332, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-1534-76fb-bf8c-6aa8ecd85188", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 266957.345920314, + "value": 123260.0 + }, + "concurrent_file/readers/10k_4t": { + "baseline": 40348.76666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-1534-76fb-bf8c-6aa8ecd85188", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 72055.7031431509, + "value": 31157.0 + }, + "concurrent_file/readers/10k_8t": { + "baseline": 77477.06666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-1534-76fb-bf8c-6aa8ecd85188", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 134560.9609840044, + "value": 62398.0 + }, + "concurrent_file/readers_plus_writer/100k_16t": { + "baseline": 309397.36666666664, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-510f-7266-be06-79a92bbbdec0", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 677506.0688299752, + "value": 354110.0 + }, + "concurrent_file/readers_plus_writer/100k_4t": { + "baseline": 206554.26666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-510f-7266-be06-79a92bbbdec0", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 515105.77025762986, + "value": 214770.0 + }, + "concurrent_file/readers_plus_writer/100k_8t": { + "baseline": 238428.73333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-510f-7266-be06-79a92bbbdec0", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 563965.0193313917, + "value": 267710.0 + }, + "concurrent_file/readers_plus_writer/10k_16t": { + "baseline": 328220.3333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-510f-7266-be06-79a92bbbdec0", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 722461.3140461927, + "value": 338550.0 + }, + "concurrent_file/readers_plus_writer/10k_4t": { + "baseline": 205670.2, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-510f-7266-be06-79a92bbbdec0", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 509877.69779367244, + "value": 226040.0 + }, + "concurrent_file/readers_plus_writer/10k_8t": { + "baseline": 248932.73333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-510f-7266-be06-79a92bbbdec0", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 588728.7812810838, + "value": 272220.0 + }, + "concurrent_file/serialized_writers/100k_16t": { + "baseline": 3584592.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 9252149.862620153, + "value": 10804000.0 + }, + "concurrent_file/serialized_writers/100k_2t": { + "baseline": 417968.5333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1016900.9723351388, + "value": 1551700.0 + }, + "concurrent_file/serialized_writers/100k_4t": { + "baseline": 859359.7333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2112232.0949456147, + "value": 3236100.0 + }, + "concurrent_file/serialized_writers/100k_8t": { + "baseline": 1800923.2333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4862390.634625617, + "value": 2021399.9999999998 + }, + "concurrent_file/serialized_writers/10k_16t": { + "baseline": 3388913.6666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8209697.076624043, + "value": 7743400.0 + }, + "concurrent_file/serialized_writers/10k_2t": { + "baseline": 450308.43333333335, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1165141.8443026468, + "value": 544540.0 + }, + "concurrent_file/serialized_writers/10k_4t": { + "baseline": 870975.3666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2146349.598529378, + "value": 1138700.0 + }, + "concurrent_file/serialized_writers/10k_8t": { + "baseline": 1742668.8333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4236708.402829729, + "value": 2368700.0 + }, + "disjunction/or_join_scale/10k": { + "baseline": 85389100.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 146930062.1667182, + "value": 25259000.0 + }, + "disjunction/or_join_scale/1k": { + "baseline": 7080630.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11970846.177917574, + "value": 1791400.0 + }, + "disjunction/or_join_short_circuit/10k": { + "baseline": 140696233.33333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 252511919.70915186, + "value": 21725000.0 + }, + "disjunction/or_join_short_circuit/1k": { + "baseline": 10985926.666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 20034720.537868537, + "value": 1672500.0 + }, + "disjunction/or_rule_body/10k": { + "baseline": 164794766.66666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 342650367.1766092, + "value": 14379000.0 + }, + "disjunction/or_rule_body/1k": { + "baseline": 12400236.666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 24934312.08075893, + "value": 1371100.0 + }, + "disjunction/or_scale/10k": { + "baseline": 76849366.66666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 128974785.03557235, + "value": 25320000.0 + }, + "disjunction/or_scale/1k": { + "baseline": 6620843.333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11286802.266610883, + "value": 1781600.0 + }, + "disjunction/or_selectivity/match_0pct": { + "baseline": 37391633.333333336, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 56865495.84829129, + "value": 21251000.0 + }, + "disjunction/or_selectivity/match_100pct": { + "baseline": 114458033.33333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 203023458.33591154, + "value": 23168000.0 + }, + "disjunction/or_selectivity/match_25pct": { + "baseline": 56748233.333333336, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 93865702.06941906, + "value": 21581000.0 + }, + "disjunction/or_selectivity/match_50pct": { + "baseline": 77562433.33333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 133865020.7050979, + "value": 18832000.0 + }, + "disjunction/or_selectivity/match_75pct": { + "baseline": 99579333.33333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 177337385.85030943, + "value": 19822000.0 + }, + "disjunction/or_short_circuit/10k": { + "baseline": 127659100.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 230176645.6373986, + "value": 22529000.0 + }, + "disjunction/or_short_circuit/1k": { + "baseline": 10232296.666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 18580244.826566443, + "value": 1707000.0 + }, + "expr/binding_into_agg/100k": { + "baseline": 297136333.3333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 482732904.4448254, + "value": 135240000.0 + }, + "expr/binding_into_agg/10k": { + "baseline": 22269200.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 38236979.36678728, + "value": 10519000.0 + }, + "expr/binding_into_agg/1k": { + "baseline": 2090800.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3487732.694706279, + "value": 1074900.0 + }, + "expr/binding_scale/100k": { + "baseline": 279317333.3333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 443006444.1731538, + "value": 135680000.0 + }, + "expr/binding_scale/10k": { + "baseline": 22420566.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 37933779.74062627, + "value": 10599000.0 + }, + "expr/binding_scale/1k": { + "baseline": 2137263.3333333335, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3583243.4714119704, + "value": 1055100.0 + }, + "expr/filter_scale/100k": { + "baseline": 280249666.6666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 443973585.770507, + "value": 134270000.0 + }, + "expr/filter_scale/10k": { + "baseline": 22781066.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 39040483.67089416, + "value": 10228000.0 + }, + "expr/filter_scale/1k": { + "baseline": 1897346.6666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3207301.8948234296, + "value": 1018400.0 + }, + "expr/keyword_equality/dynamic_eq/10k": { + "baseline": 17766500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 512582555.75505745, + "value": 17081000.0 + }, + "expr/keyword_equality/dynamic_neq/10k": { + "baseline": 19255500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 434680456.0015998, + "value": 17960000.0 + }, + "expr/keyword_equality/literal_eq/10k": { + "baseline": 11681000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 366735640.56407464, + "value": 10975000.0 + }, + "expr/keyword_equality/literal_neq/10k": { + "baseline": 12617000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 267881438.79063132, + "value": 12076000.0 + }, + "expr/keyword_equality/nested_literal_eq/10k": { + "baseline": 12001000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 292075157.4635868, + "value": 11400000.0 + }, + "expr/keyword_equality/prepared_keyword_eq/10k": { + "baseline": 11895000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 60963110.26428982, + "value": 11132000.0 + }, + "fact_log/attribute_only_storage/10k": { + "baseline": 3561500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 257778417.33555084, + "value": 2960400.0 + }, + "fact_log/filter_attribute_entity_tx/100k": { + "baseline": 1819150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 72857644.466896, + "value": 1608700.0 + }, + "fact_log/storage_mixed/100k": { + "baseline": 174935000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 5795714597.12848, + "value": 155480000.0 + }, + "fact_log/tx_mixed_default/100k": { + "baseline": 272500000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 12313483462.607748, + "value": 233710000.0 + }, + "fact_log/tx_mixed_default/10k": { + "baseline": 24708500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1007449022.9898496, + "value": 21418000.0 + }, + "fact_log/tx_mixed_window_4096/100k": { + "baseline": 757075000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10325356501.536514, + "value": 750830000.0 + }, + "insert/batch_100/100k": { + "baseline": 418657.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 623174.854182455, + "value": 234460.0 + }, + "insert/batch_100/10k": { + "baseline": 398770.6666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 581070.5078180677, + "value": 226880.0 + }, + "insert/batch_100/1k": { + "baseline": 397135.3333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 587852.3136689191, + "value": 233470.0 + }, + "insert/batch_100/1m": { + "baseline": 411995.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 608441.0288541969, + "value": 210860.0 + }, + "insert/explicit_tx/100k": { + "baseline": 4812.23, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 7308.823494539594, + "value": 3186.6 + }, + "insert/explicit_tx/10k": { + "baseline": 4619.916666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6918.482370116439, + "value": 3204.2999999999997 + }, + "insert/explicit_tx/1k": { + "baseline": 4618.496666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6933.850084666557, + "value": 3103.7 + }, + "insert/explicit_tx/1m": { + "baseline": 4527.069999999999, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6936.132652953025, + "value": 3028.7000000000003 + }, + "insert/single_fact/100k": { + "baseline": 4758.83, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 7194.937120407247, + "value": 2953.3999999999996 + }, + "insert/single_fact/10k": { + "baseline": 4559.503333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6849.846034391572, + "value": 2846.2000000000003 + }, + "insert/single_fact/1k": { + "baseline": 4541.8933333333325, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6919.098016535425, + "value": 2852.1 + }, + "insert/single_fact/1m": { + "baseline": 4420.063333333331, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6721.098534121742, + "value": 2593.6 + }, + "insert_file/batch_100/100k": { + "baseline": 778729.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1850820.1947021212, + "value": 656350.0 + }, + "insert_file/batch_100/10k": { + "baseline": 773820.6666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1914379.3956190054, + "value": 655500.0 + }, + "insert_file/batch_100/1k": { + "baseline": 767624.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1747334.9373174615, + "value": 676630.0 + }, + "insert_file/batch_100/1m": { + "baseline": 809792.3333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2278141.492789923, + "value": 675390.0 + }, + "insert_file/chunked/execute/65535": { + "baseline": 231960000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3677754260.1327124, + "value": 298220000.0 + }, + "insert_file/chunked/execute/65537": { + "baseline": 294805000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1852304005.5799856, + "value": 337310000.0 + }, + "insert_file/chunked/retract/65535": { + "baseline": 201230000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3917174530.1271167, + "value": 256079999.99999997 + }, + "insert_file/chunked/retract/65537": { + "baseline": 414225000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2164688484.1474175, + "value": 476110000.0 + }, + "insert_file/chunked/statement_rollback/65537": { + "baseline": 283185000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1421675423.547848, + "value": 343600000.0 + }, + "insert_file/chunked/write_tx/65535": { + "baseline": 257275000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3352976563.303229, + "value": 346400000.0 + }, + "insert_file/chunked/write_tx/65537": { + "baseline": 355555000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 9355969607.466642, + "value": 442440000.0 + }, + "insert_file/explicit_tx/100k": { + "baseline": 202388.27333333335, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 583031.5615024453, + "value": 214190.0 + }, + "insert_file/explicit_tx/10k": { + "baseline": 223399.49333333332, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 766124.3290578438, + "value": 208210.0 + }, + "insert_file/explicit_tx/1k": { + "baseline": 211000.01333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 563453.1860770173, + "value": 208250.0 + }, + "insert_file/explicit_tx/1m": { + "baseline": 241339.79333333336, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1046725.5604677281, + "value": 207220.0 + }, + "insert_file/set_storage/inline_unique/10k": { + "baseline": 50989000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1765616223.8420374, + "value": 69232000.0 + }, + "insert_file/set_storage/long_repeated/10k": { + "baseline": 193795000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6487193636.706385, + "value": 248010000.0 + }, + "insert_file/set_storage/long_unique/10k": { + "baseline": 194290000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8574461640.642755, + "value": 253190000.0 + }, + "insert_file/set_storage/scalar/10k": { + "baseline": 31204000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1098021902.9370885, + "value": 40693000.0 + }, + "insert_file/single_fact/100k": { + "baseline": 266285.7833333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1309990.5904221158, + "value": 233460.0 + }, + "insert_file/single_fact/10k": { + "baseline": 225043.93000000002, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 715007.4665153591, + "value": 209950.0 + }, + "insert_file/single_fact/1k": { + "baseline": 223729.31999999998, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 717551.7896261486, + "value": 213150.0 + }, + "insert_file/single_fact/1m": { + "baseline": 215173.77999999997, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 605710.1260283012, + "value": 210340.0 + }, + "log_writer/append/100k": { + "baseline": 667005000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 30430397501.322315, + "value": 615960000.0 + }, + "log_writer/append/10k": { + "baseline": 30008500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2053103226.0091176, + "value": 25100000.0 + }, + "log_writer/finish/100k": { + "baseline": 438870000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8752882390.84821, + "value": 400090000.0 + }, + "log_writer/finish/10k": { + "baseline": 35193000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e8-3821-7122-870f-13482be89335", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 915662349.3491106, + "value": 31434000.0 + }, + "memory_meter/count/limited/10k": { + "baseline": 4079500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e7-b8c7-722d-97cb-e2cb3d1cb892", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 77020072.89848925, + "value": 3198600.0 + }, + "memory_meter/count/unlimited/10k": { + "baseline": 4028950.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e7-b8c7-722d-97cb-e2cb3d1cb892", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 51470645.37350717, + "value": 3198900.0 + }, + "memory_meter/hash_groups/limited/10k": { + "baseline": 16278500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e7-b8c7-722d-97cb-e2cb3d1cb892", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 95945263.29426779, + "value": 13076000.0 + }, + "memory_meter/hash_groups/unlimited/10k": { + "baseline": 16207500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e7-b8c7-722d-97cb-e2cb3d1cb892", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 53973405.09105452, + "value": 13198000.0 + }, + "negation/not_join_scale/10k": { + "baseline": 29128966.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 50260255.756653026, + "value": 24227000.0 + }, + "negation/not_join_scale/1k": { + "baseline": 2728536.6666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4669762.440159027, + "value": 2337500.0 + }, + "negation/not_pushdown_selectivity/excl_0pct": { + "baseline": 130044433.33333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 231781894.66556084, + "value": 68471000.0 + }, + "negation/not_pushdown_selectivity/excl_100pct": { + "baseline": 63513800.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 115035639.80874114, + "value": 27761000.0 + }, + "negation/not_pushdown_selectivity/excl_25pct": { + "baseline": 130228100.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 237221005.1525284, + "value": 58851000.0 + }, + "negation/not_pushdown_selectivity/excl_50pct": { + "baseline": 129667166.66666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 240934930.24267775, + "value": 49245000.0 + }, + "negation/not_pushdown_selectivity/excl_75pct": { + "baseline": 128562833.33333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 244585955.09695712, + "value": 40808000.0 + }, + "negation/not_rule_body/10k": { + "baseline": 1417629400.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3149485451.038362, + "value": 30088000.0 + }, + "negation/not_rule_body/1k": { + "baseline": 20458420.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 43323103.94586372, + "value": 2900300.0 + }, + "negation/not_scale/10k": { + "baseline": 25076100.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 43121754.38415296, + "value": 20419000.0 + }, + "negation/not_scale/1k": { + "baseline": 2304563.3333333335, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4065381.4716826733, + "value": 1917500.0 + }, + "negation/not_selectivity/excl_0pct": { + "baseline": 20595600.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 34741573.8646623, + "value": 19535000.0 + }, + "negation/not_selectivity/excl_100pct": { + "baseline": 44943166.666666664, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 79982392.69053629, + "value": 25749000.0 + }, + "negation/not_selectivity/excl_25pct": { + "baseline": 28619500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 51119488.65219487, + "value": 21593000.0 + }, + "negation/not_selectivity/excl_50pct": { + "baseline": 35922733.333333336, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 64019762.36681299, + "value": 22908000.0 + }, + "negation/not_selectivity/excl_75pct": { + "baseline": 40762033.333333336, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 72107510.1860354, + "value": 24269000.0 + }, + "open/checkpointed/100k": { + "baseline": 10470946.5, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21921235.063397154, + "value": 22897.0 + }, + "open/checkpointed/10k": { + "baseline": 967423.1666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2015464.7442633947, + "value": 22738.0 + }, + "open/checkpointed/1k": { + "baseline": 108805.16666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 218333.34444847325, + "value": 22333.0 + }, + "open/checkpointed/1m": { + "baseline": 144844119.06666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 300088410.4014503, + "value": 22752.0 + }, + "open/storage_v3/migrate_v7/multi_checkpoint": { + "baseline": 1974330.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 577488130.6919575, + "value": 1982600.0 + }, + "open/storage_v3/read_only_v7/multi_checkpoint": { + "baseline": 582865.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 32783123.202088166, + "value": 811750.0 + }, + "open/storage_v3/read_only_v8/10k": { + "baseline": 13550.95, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2054040.045434268, + "value": 19333.0 + }, + "open/storage_v3/read_only_wal/10k_plus_1k": { + "baseline": 1816500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 275495263.3167805, + "value": 2623500.0 + }, + "open/wal_replay/100k": { + "baseline": 1962566266.6666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6145615544.537551, + "value": 278350000.0 + }, + "open/wal_replay/10k": { + "baseline": 4140631.3666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 22677927.148895204, + "value": 26195.0 + }, + "open/wal_replay/1k": { + "baseline": 266394.4, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-91fe-708a-895c-346483052a19", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1878504.0356882364, + "value": 22953.0 + }, + "point_query_chain_depth/attr_scan/1": { + "baseline": 22323.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 155744.1537523386, + "value": 21861.0 + }, + "point_query_chain_depth/attr_scan/2000": { + "baseline": 732025.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10460191.355206674, + "value": 713500.0 + }, + "point_query_chain_depth/churned_attr/1": { + "baseline": 22305.5, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 145527.10274234577, + "value": 21900.0 + }, + "point_query_chain_depth/churned_attr/2000": { + "baseline": 725055.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11743326.72620036, + "value": 707690.0 + }, + "point_query_chain_depth/churned_attr_as_of/1": { + "baseline": 23863.5, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 152598.37355855812, + "value": 23453.0 + }, + "point_query_chain_depth/churned_attr_as_of/2000": { + "baseline": 644690.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 876247.3742809183, + "value": 643730.0 + }, + "point_query_chain_depth/sibling_attr/1": { + "baseline": 22356.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 80245.34357022957, + "value": 21639.0 + }, + "point_query_chain_depth/sibling_attr/2000": { + "baseline": 28596.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 136104.78091614065, + "value": 28499.0 + }, + "prepared/count_slots/10k": { + "baseline": 9322300.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 219488183.5140144, + "value": 8752100.0 + }, + "prepared/cursor_drain/10k": { + "baseline": 10958000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 250785280.50523674, + "value": 10536000.0 + }, + "prepared/prepare_value_lookup/10k": { + "baseline": 2847.65, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 130507.4357493968, + "value": 3103.0 + }, + "prepared/threshold_filter/10k": { + "baseline": 10822000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 258367859.64793402, + "value": 10291000.0 + }, + "prepared/threshold_filter/1k": { + "baseline": 968695.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 32562493.41230481, + "value": 1018500.0 + }, + "prepared/time_entity_slots/10k": { + "baseline": 4757.15, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 134953.0403248542, + "value": 5288.2 + }, + "prepared/unprepared_count_slots/10k": { + "baseline": 9114550.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 256412312.46120444, + "value": 8983000.0 + }, + "prepared/unprepared_cursor_drain/10k": { + "baseline": 10400000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 20323887.46918221, + "value": 10705000.0 + }, + "prepared/unprepared_time_entity_slots/10k": { + "baseline": 10279.95, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 434278.0421208101, + "value": 11231.0 + }, + "prepared/unprepared_value_lookup/10k": { + "baseline": 4021350.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 74287986.55262613, + "value": 4068500.0 + }, + "prepared/value_lookup/10k": { + "baseline": 4046100.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 64636946.270173475, + "value": 4139599.9999999995 + }, + "prepared/value_lookup/1k": { + "baseline": 369205.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 22171434.4427117, + "value": 401620.0 + }, + "query/cursor_lifecycle/drain/100k": { + "baseline": 43230000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 210282105.73123387, + "value": 42922000.0 + }, + "query/cursor_lifecycle/drain/10k": { + "baseline": 4236550.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 22898971.712878767, + "value": 4169600.0 + }, + "query/cursor_lifecycle/first_batch/10k": { + "baseline": 381720.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2366497.493836442, + "value": 385540.0 + }, + "query/cursor_lifecycle/held_checkpoint_close/10k": { + "baseline": 23085.5, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 382275.09367623396, + "value": 25979.0 + }, + "query/cursor_lifecycle/held_checkpoint_drain/10k": { + "baseline": 4380250.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8432504.04991607, + "value": 4549100.0 + }, + "query/cursor_lifecycle/iterator/10k": { + "baseline": 4297850.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 41209198.11454161, + "value": 4352500.0 + }, + "query/cursor_lifecycle/open/10k": { + "baseline": 5875.700000000001, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 18335.692044639836, + "value": 5157.9 + }, + "query/cursor_lifecycle/staged_read/10k_plus_200": { + "baseline": 4679950.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 16891844.85791033, + "value": 4552300.0 + }, + "query/distinct/bag/100k": { + "baseline": 122690000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 508618957.1348638, + "value": 138810000.0 + }, + "query/distinct/bag/10k": { + "baseline": 8922750.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 38005253.55552009, + "value": 8629500.0 + }, + "query/distinct/float_set_tuple/100k": { + "baseline": 389630000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2005018349.1502154, + "value": 383190000.0 + }, + "query/distinct/float_set_tuple/10k": { + "baseline": 35318000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 273491299.26037306, + "value": 33772000.0 + }, + "query/distinct/scalar/100k": { + "baseline": 122290000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1539200599.7665713, + "value": 136560000.0 + }, + "query/distinct/scalar/10k": { + "baseline": 9383150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11064697.598944763, + "value": 8959500.0 + }, + "query/distinct/sorted_page/100k": { + "baseline": 126320000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 953310622.431851, + "value": 137380000.0 + }, + "query/distinct/sorted_page/10k": { + "baseline": 9953150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 177473883.75061193, + "value": 9056300.0 + }, + "query/join_3pattern/100k": { + "baseline": 807315168.5652174, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1826308698.5380936, + "value": 20069.0 + }, + "query/join_3pattern/10k": { + "baseline": 53845148.1, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 114583745.78891957, + "value": 19908.0 + }, + "query/join_3pattern/1k": { + "baseline": 4672081.333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10184698.600703537, + "value": 19868.0 + }, + "query/join_3pattern/1m": { + "baseline": 9966932579.695652, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 24543319722.063057, + "value": 20309.0 + }, + "query/operator_paths/anti_committed/10k": { + "baseline": 173860000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1160735476.1020088, + "value": 181060000.0 + }, + "query/operator_paths/entity_committed/10k": { + "baseline": 166230000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 309575041.2215208, + "value": 172020000.0 + }, + "query/operator_paths/entity_pending/10k": { + "baseline": 27614500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 599616347.1820302, + "value": 27197000.0 + }, + "query/operator_paths/hash_committed/10k": { + "baseline": 33009000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 110746118.50859295, + "value": 33738000.0 + }, + "query/operator_paths/hash_pending/10k": { + "baseline": 44196500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1090890964.4579124, + "value": 44736000.0 + }, + "query/operator_paths/union_committed/10k": { + "baseline": 195440000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 890112122.8427547, + "value": 197170000.0 + }, + "query/operator_paths/value_committed/10k": { + "baseline": 162750000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 774723060.5995697, + "value": 175330000.0 + }, + "query/operator_paths/value_pending/10k": { + "baseline": 32543500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1059390189.5195482, + "value": 33545000.0 + }, + "query/operator_paths/windows_committed/10k": { + "baseline": 174210000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 482953165.707891, + "value": 184750000.0 + }, + "query/ordered_groups/avg/100k": { + "baseline": 121735000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 758517779.2725252, + "value": 113560000.0 + }, + "query/ordered_groups/avg/10k": { + "baseline": 12308500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 117887636.1304663, + "value": 11213000.0 + }, + "query/ordered_groups/avg/1k": { + "baseline": 1296400.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10117633.305939743, + "value": 1222600.0 + }, + "query/ordered_groups/avg/1m": { + "baseline": 1236100000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 9395740807.994263, + "value": 1128500000.0 + }, + "query/ordered_groups/avg_limit/100k": { + "baseline": 1241000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 13315063.087505022, + "value": 1128500.0 + }, + "query/ordered_groups/avg_limit/10k": { + "baseline": 1236150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 16204680.266016502, + "value": 1136500.0 + }, + "query/ordered_groups/avg_limit/1k": { + "baseline": 1201700.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 16693990.993556673, + "value": 1106100.0 + }, + "query/ordered_groups/avg_limit/1m": { + "baseline": 1233650.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 14327668.188504307, + "value": 1129900.0 + }, + "query/ordered_groups/count/100k": { + "baseline": 102545000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 463664238.4619082, + "value": 98424000.0 + }, + "query/ordered_groups/count/10k": { + "baseline": 10086500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 19183396.84675036, + "value": 9661000.0 + }, + "query/ordered_groups/count/1k": { + "baseline": 1134400.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 20155184.315932572, + "value": 1052700.0 + }, + "query/ordered_groups/count/1m": { + "baseline": 1026550000.0000001, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8276501123.319227, + "value": 975140000.0 + }, + "query/ordered_groups/count_limit/100k": { + "baseline": 1035899.9999999999, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4895189.571348637, + "value": 985290.0 + }, + "query/ordered_groups/count_limit/10k": { + "baseline": 1042450.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 9505320.702885972, + "value": 987660.0 + }, + "query/ordered_groups/count_limit/1k": { + "baseline": 1000210.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4357791.927073315, + "value": 958630.0 + }, + "query/ordered_groups/count_limit/1m": { + "baseline": 1034550.0000000001, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 7181846.960076758, + "value": 981760.0 + }, + "query/ordered_groups/distinct/100k": { + "baseline": 117550000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 839788476.9238164, + "value": 106740000.0 + }, + "query/ordered_groups/distinct/10k": { + "baseline": 11471500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 84522338.3148135, + "value": 10597000.0 + }, + "query/ordered_groups/distinct/1k": { + "baseline": 1243150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8217437.58250861, + "value": 1147900.0 + }, + "query/ordered_groups/distinct/1m": { + "baseline": 1159150000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4384413427.484219, + "value": 1072400000.0 + }, + "query/ordered_groups/distinct_limit/100k": { + "baseline": 1187850.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 17810361.510880202, + "value": 1079200.0 + }, + "query/ordered_groups/distinct_limit/10k": { + "baseline": 1183950.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 16703807.347637735, + "value": 1093300.0 + }, + "query/ordered_groups/distinct_limit/1k": { + "baseline": 1149300.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 17523714.324150644, + "value": 1058400.0 + }, + "query/ordered_groups/distinct_limit/1m": { + "baseline": 1180850.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 17417432.553745337, + "value": 1078100.0 + }, + "query/ordered_groups/max/100k": { + "baseline": 107800000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1254560329.7721665, + "value": 101000000.0 + }, + "query/ordered_groups/max/10k": { + "baseline": 10633500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 113456000.72236012, + "value": 9902100.0 + }, + "query/ordered_groups/max/1k": { + "baseline": 1146900.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 18072641.40577188, + "value": 1084900.0 + }, + "query/ordered_groups/max/1m": { + "baseline": 1076200000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 9897433305.939743, + "value": 998230000.0 + }, + "query/ordered_groups/max_limit/100k": { + "baseline": 1083000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 20324315.148581065, + "value": 999050.0 + }, + "query/ordered_groups/max_limit/10k": { + "baseline": 1083850.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21510518.374066714, + "value": 1004999.9999999999 + }, + "query/ordered_groups/max_limit/1k": { + "baseline": 1053450.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 18392686.716987807, + "value": 973820.0 + }, + "query/ordered_groups/max_limit/1m": { + "baseline": 1085700.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 19720555.358797707, + "value": 997470.0 + }, + "query/ordered_groups/min/100k": { + "baseline": 110170000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 253515041.2215208, + "value": 101510000.0 + }, + "query/ordered_groups/min/10k": { + "baseline": 10829000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 13585635.408106169, + "value": 9874200.0 + }, + "query/ordered_groups/min/1k": { + "baseline": 1150450.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 15347122.351746775, + "value": 1081900.0 + }, + "query/ordered_groups/min/1m": { + "baseline": 1068350000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10192813200.831388, + "value": 1006999999.9999999 + }, + "query/ordered_groups/min_limit/100k": { + "baseline": 1095300.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 15043875.16501722, + "value": 1008599.9999999999 + }, + "query/ordered_groups/min_limit/10k": { + "baseline": 1085850.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 20906058.58428336, + "value": 1002899.9999999999 + }, + "query/ordered_groups/min_limit/1k": { + "baseline": 1054550.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21922280.03936374, + "value": 971110.0 + }, + "query/ordered_groups/min_limit/1m": { + "baseline": 1081250.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21177122.125093978, + "value": 1002699.9999999999 + }, + "query/ordered_groups/sum/100k": { + "baseline": 107540000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 669893623.2536585, + "value": 102440000.0 + }, + "query/ordered_groups/sum/10k": { + "baseline": 10809500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 37548863.45862985, + "value": 9996200.0 + }, + "query/ordered_groups/sum/1k": { + "baseline": 1150200.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11790812.675289813, + "value": 1084700.0 + }, + "query/ordered_groups/sum/1m": { + "baseline": 1074150000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6504721753.969154, + "value": 1006000000.0 + }, + "query/ordered_groups/sum_distinct/100k": { + "baseline": 112385000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 335672468.05659974, + "value": 105590000.0 + }, + "query/ordered_groups/sum_distinct/10k": { + "baseline": 11286000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 111076201.77344334, + "value": 10480000.0 + }, + "query/ordered_groups/sum_distinct/1k": { + "baseline": 1210900.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8267886.644751795, + "value": 1130000.0 + }, + "query/ordered_groups/sum_distinct/1m": { + "baseline": 1114950000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2906763015.2690105, + "value": 1041600000.0000001 + }, + "query/ordered_groups/sum_distinct_limit/100k": { + "baseline": 1133250.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 13455410.274234578, + "value": 1062400.0 + }, + "query/ordered_groups/sum_distinct_limit/10k": { + "baseline": 1137850.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 15775584.017043762, + "value": 1069900.0 + }, + "query/ordered_groups/sum_distinct_limit/1k": { + "baseline": 1101250.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 9619253.411048064, + "value": 1046899.9999999999 + }, + "query/ordered_groups/sum_distinct_limit/1m": { + "baseline": 1129800.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 12487137.881397419, + "value": 1061500.0 + }, + "query/ordered_groups/sum_limit/100k": { + "baseline": 1080250.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 17316832.553745337, + "value": 1002000.0 + }, + "query/ordered_groups/sum_limit/10k": { + "baseline": 1081100.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 16518258.28539455, + "value": 1012300.0 + }, + "query/ordered_groups/sum_limit/1k": { + "baseline": 1049950.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 15742816.725205854, + "value": 985870.0 + }, + "query/ordered_groups/sum_limit/1m": { + "baseline": 1086050.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 16936703.596610475, + "value": 1009600.0 + }, + "query/ordered_values/index/100k": { + "baseline": 470725.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3792470.6667679343, + "value": 454430.0 + }, + "query/ordered_values/index/10k": { + "baseline": 474080.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2260379.744452798, + "value": 460030.0 + }, + "query/ordered_values/index/1k": { + "baseline": 453985.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3930102.24962188, + "value": 441600.0 + }, + "query/ordered_values/index/1m": { + "baseline": 469455.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3052422.377395481, + "value": 458710.0 + }, + "query/ordered_values/uuid_ties/100k": { + "baseline": 214860000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 633868582.0321379, + "value": 202580000.0 + }, + "query/ordered_values/uuid_ties/10k": { + "baseline": 19647000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 41148756.18322812, + "value": 18562000.0 + }, + "query/ordered_values/uuid_ties/1k": { + "baseline": 1919600.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 7598268.940698709, + "value": 1761500.0 + }, + "query/ordered_values/uuid_ties/1m": { + "baseline": 2212150000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8524845084.5631275, + "value": 2105900000.0 + }, + "query/pagination/committed_asc_offset0_k10/100k": { + "baseline": 488915.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1936148.5892557392, + "value": 468590.0 + }, + "query/pagination/committed_asc_offset_half_k10/100k": { + "baseline": 18665000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 23626943.734591104, + "value": 17822000.0 + }, + "query/pagination/committed_compound_ties/100k": { + "baseline": 3226850000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11524322578.39957, + "value": 3322400000.0 + }, + "query/pagination/committed_compound_ties/10k": { + "baseline": 318045000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 651597884.3808465, + "value": 331890000.0 + }, + "query/pagination/committed_desc_k10/100k": { + "baseline": 42711500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 337395825.12654954, + "value": 40190000.0 + }, + "query/pagination/committed_desc_k1000/100k": { + "baseline": 44003000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 367080669.8300431, + "value": 40946000.0 + }, + "query/pagination/pending_asc_offset0_k10/100k": { + "baseline": 119720000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 527702040.3997131, + "value": 136190000.0 + }, + "query/pagination/pending_asc_offset_half_k10/100k": { + "baseline": 188355000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1128367674.164204, + "value": 201670000.0 + }, + "query/pagination/pending_desc_k10/100k": { + "baseline": 116210000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3451738843.808465, + "value": 136770000.0 + }, + "query/pagination/pending_desc_k1000/100k": { + "baseline": 119600000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2120917306.285079, + "value": 140180000.0 + }, + "query/partition_windows/entity/100k": { + "baseline": 351370000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2269988244.041894, + "value": 318410000.0 + }, + "query/partition_windows/entity/10k": { + "baseline": 33118500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 105066684.15157205, + "value": 29754000.0 + }, + "query/partition_windows/entity/1k": { + "baseline": 3245250.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 14078827.153857246, + "value": 2848400.0 + }, + "query/partition_windows/entity/1m": { + "baseline": 3582650000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21969408172.068153, + "value": 3272000000.0 + }, + "query/partition_windows/entity_limit/100k": { + "baseline": 2968150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 31830122.72287159, + "value": 2570700.0 + }, + "query/partition_windows/entity_limit/10k": { + "baseline": 2807350.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 26652246.28011837, + "value": 2400700.0 + }, + "query/partition_windows/entity_limit/1k": { + "baseline": 2671550.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 23098218.374066714, + "value": 2259700.0 + }, + "query/partition_windows/entity_limit/1m": { + "baseline": 3042650.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 36094708.543192975, + "value": 2697800.0 + }, + "query/partition_windows/value/100k": { + "baseline": 356485000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2641735753.3200145, + "value": 320540000.0 + }, + "query/partition_windows/value/10k": { + "baseline": 33830500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 189580400.55799857, + "value": 30101000.0 + }, + "query/partition_windows/value/1k": { + "baseline": 3415250.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10940864.664129842, + "value": 3040500.0 + }, + "query/partition_windows/value/1m": { + "baseline": 3641550000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 25722199618.930416, + "value": 3313400000.0 + }, + "query/partition_windows/value_limit/100k": { + "baseline": 3130200.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 34500710.94424821, + "value": 2730300.0 + }, + "query/partition_windows/value_limit/10k": { + "baseline": 2982500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 23932929.10160689, + "value": 2570200.0 + }, + "query/partition_windows/value_limit/1k": { + "baseline": 2836800.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21526788.06695983, + "value": 2409600.0 + }, + "query/partition_windows/value_limit/1m": { + "baseline": 3204400.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 34685176.36057246, + "value": 2841400.0 + }, + "query/point_attribute/100k": { + "baseline": 282441304.34782606, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 486220079.54189944, + "value": 143970000.0 + }, + "query/point_attribute/10k": { + "baseline": 18503870.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 33231834.573754095, + "value": 8887900.0 + }, + "query/point_attribute/1k": { + "baseline": 1651663.3333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3011067.9851560583, + "value": 800270.0 + }, + "query/point_attribute/1m": { + "baseline": 3949700000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 7147128011.070749, + "value": 1915700000.0 + }, + "query/point_entity/100k": { + "baseline": 205781637.1347826, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 499516470.78292, + "value": 7184.599999999999 + }, + "query/point_entity/10k": { + "baseline": 5999.606666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11023.472658701803, + "value": 7138.7 + }, + "query/point_entity/1k": { + "baseline": 5859.2, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10884.262533192516, + "value": 7181.6 + }, + "query/point_entity/1m": { + "baseline": 2863309471.2173915, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 7793680452.890194, + "value": 7301.3 + }, + "query/predicate_pushdown/100k": { + "baseline": 640883000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1153325569.301722, + "value": 171120000.0 + }, + "query/predicate_pushdown/10k": { + "baseline": 48397500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 92008335.0117091, + "value": 11205000.0 + }, + "query/predicate_pushdown/1k": { + "baseline": 4162796.6666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 8079019.97290702, + "value": 1003300.0000000001 + }, + "query/regex_filter/10k": { + "baseline": 23431030.833333332, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 51884219.829697765, + "value": 14035.0 + }, + "query/regex_filter/1k": { + "baseline": 1917662.8333333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4266649.037490355, + "value": 14004.0 + }, + "query/set_queries/all_in/committed/10k": { + "baseline": 181550000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 550939144.6862267, + "value": 183330000.0 + }, + "query/set_queries/all_in/pending/10k": { + "baseline": 38902500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1243827836.8832068, + "value": 39188000.0 + }, + "query/set_queries/any_none/committed/10k": { + "baseline": 188035000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 345163218.2620517, + "value": 187760000.0 + }, + "query/set_queries/any_none/pending/10k": { + "baseline": 43407000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1258531887.8931997, + "value": 43200000.0 + }, + "query/set_queries/collect/committed/10k": { + "baseline": 186000000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2220396931.182353, + "value": 183650000.0 + }, + "query/set_queries/collect/pending/10k": { + "baseline": 39510000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1071594296.7949499, + "value": 40371000.0 + }, + "query/set_queries/in/committed/10k": { + "baseline": 181165000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 762815071.1104017, + "value": 182860000.0 + }, + "query/set_queries/in/pending/10k": { + "baseline": 37918000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1101427940.4473603, + "value": 38949000.0 + }, + "query/set_queries/not_in/committed/10k": { + "baseline": 182240000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1031283705.6967003, + "value": 183420000.0 + }, + "query/set_queries/not_in/pending/10k": { + "baseline": 38293500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1529908919.3262484, + "value": 38705000.0 + }, + "query/set_storage/exact_inline_unique/10k": { + "baseline": 20386.5, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 124862.98196722384, + "value": 17636.0 + }, + "query/set_storage/exact_long_unique/10k": { + "baseline": 48795.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 934226.2930837019, + "value": 44322.0 + }, + "query/set_storage/scan_long_unique/10k": { + "baseline": 57047000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 388945903.1359828, + "value": 59830000.0 + }, + "query/storage_v3/multivalue_long_scan/10k": { + "baseline": 14741500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 201365717.12878767, + "value": 13726000.0 + }, + "recursion/ancestors/w10_d4": { + "baseline": 235905.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-a076-736f-9c60-d3a45b0923a6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2267545.2957742466, + "value": 211870.0 + }, + "recursion/chain/depth_10": { + "baseline": 5921305.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-a076-736f-9c60-d3a45b0923a6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 12562734.483114593, + "value": 419540.0 + }, + "recursion/chain/depth_100": { + "baseline": 3689371933.3333335, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-a076-736f-9c60-d3a45b0923a6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 7864092442.068277, + "value": 17841000.0 + }, + "recursion/fanout/w10_d3": { + "baseline": 140851233.33333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-a076-736f-9c60-d3a45b0923a6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 285518529.89957154, + "value": 17030000.0 + }, + "recursion/operator_paths/ancestors_committed/10k": { + "baseline": 2875700.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9eb-a076-736f-9c60-d3a45b0923a6", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 32978158.656519372, + "value": 2467300.0 + }, + "retract/batch_100/10k": { + "baseline": 334150.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3024626.1583116213, + "value": 332510.0 + }, + "retract/single_fact/10k": { + "baseline": 3966.3499999999995, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 15847.448608937564, + "value": 3931.2 + }, + "temporal_metadata/tx_time/100k": { + "baseline": 278849000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 472810529.78204024, + "value": 127080000.0 + }, + "temporal_metadata/tx_time/10k": { + "baseline": 24941876.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 46169557.94091761, + "value": 8227100.0 + }, + "temporal_metadata/tx_time/1k": { + "baseline": 2379664.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4576966.905822257, + "value": 798780.0 + }, + "temporal_metadata/valid_from/100k": { + "baseline": 279431000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 474451846.3703128, + "value": 122010000.0 + }, + "temporal_metadata/valid_from/10k": { + "baseline": 24852886.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 45506445.25435001, + "value": 8386400.0 + }, + "temporal_metadata/valid_from/1k": { + "baseline": 2447498.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4561880.407301785, + "value": 806790.0 + }, + "temporal_metadata/valid_to/100k": { + "baseline": 279744666.6666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 471528286.2239855, + "value": 119860000.0 + }, + "temporal_metadata/valid_to/10k": { + "baseline": 24914346.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 46005436.06690152, + "value": 8205900.0 + }, + "temporal_metadata/valid_to/1k": { + "baseline": 2447172.6666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4551074.074631801, + "value": 805450.0 + }, + "time_travel/as_of_counter/100k": { + "baseline": 83283589.85, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 178073470.10772395, + "value": 7713.7 + }, + "time_travel/as_of_counter/10k": { + "baseline": 6725932.416666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 14880910.576378513, + "value": 7651.900000000001 + }, + "time_travel/as_of_counter/1k": { + "baseline": 664046.25, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1480462.0384330647, + "value": 7552.1 + }, + "time_travel/as_of_counter/1m": { + "baseline": 1378887298.98, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2934857771.702593, + "value": 7808.4 + }, + "time_travel/operator_paths/timestamp_irregular/10k": { + "baseline": 4459350.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 56642458.2754498, + "value": 4676700.0 + }, + "time_travel/operator_paths/timestamp_monotone/10k": { + "baseline": 3554050.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 24752576.28833644, + "value": 3163700.0 + }, + "time_travel/valid_at/100k": { + "baseline": 7506.416666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 13181.303936577486, + "value": 8580.6 + }, + "time_travel/valid_at/10k": { + "baseline": 7430.506666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 13087.96190396836, + "value": 8502.7 + }, + "time_travel/valid_at/1k": { + "baseline": 7341.42, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 12923.053055774271, + "value": 8429.8 + }, + "time_travel/valid_at/1m": { + "baseline": 7607.906666666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 13398.953218671413, + "value": 8703.900000000001 + }, + "udf/aggregate_sum_dispatch/10k": { + "baseline": 19403053.333333332, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 36635442.39099128, + "value": 5133400.0 + }, + "udf/aggregate_sum_dispatch/1k": { + "baseline": 1572928.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3120654.48393311, + "value": 496530.0 + }, + "udf/predicate_filter_dispatch/10k": { + "baseline": 38447743.333333336, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 80862925.43379614, + "value": 4979500.0 + }, + "udf/predicate_filter_dispatch/1k": { + "baseline": 3082067.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 6633477.464859569, + "value": 485520.0 + }, + "udf/v3/aggregate_fallible/10k": { + "baseline": 8573966.666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 28714853.346048675, + "value": 4956400.0 + }, + "udf/v3/aggregate_infallible/10k": { + "baseline": 9174666.666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 18613604.83222439, + "value": 5065400.0 + }, + "udf/v3/callback_reentry/1k": { + "baseline": 8976966.666666666, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 43620422.65784991, + "value": 5528100.0 + }, + "udf/v3/predicate_fallible/10k": { + "baseline": 10745700.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 38578630.454889126, + "value": 6187700.0 + }, + "udf/v3/predicate_infallible/10k": { + "baseline": 10785333.333333334, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 37868842.27662891, + "value": 6152500.0 + }, + "udf/v3/registry_snapshot_churn/32": { + "baseline": 10194.199999999999, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402, + 41971706654 + ], + "report": "01a1aa1d-240c-757f-a595-9b75dd430226", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 52140.545265291425, + "value": 6586.0 + }, + "window/many_partitions/100k": { + "baseline": 221950000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 10807429967.127691, + "value": 184470000.0 + }, + "window/many_partitions/10k": { + "baseline": 16729500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1113594728.8854449, + "value": 13045000.0 + }, + "window/many_partitions/1k": { + "baseline": 1385650.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 11061440.282452654, + "value": 1151200.0 + }, + "window/neighbors/lag/hundred/10k": { + "baseline": 51879500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 640421159.6306672, + "value": 42478000.0 + }, + "window/neighbors/lag/one/10k": { + "baseline": 50477500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 1259262126.4545553, + "value": 42250000.0 + }, + "window/neighbors/lag_lead/hundred/100k": { + "baseline": 674450000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 4500659946.451363, + "value": 556430000.0 + }, + "window/neighbors/lag_lead/one/100k": { + "baseline": 652245000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3874751792.0761123, + "value": 541440000.0 + }, + "window/neighbors/lead/hundred/10k": { + "baseline": 52327000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 171413649.63018653, + "value": 43519000.0 + }, + "window/neighbors/lead/one/10k": { + "baseline": 51445000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 577411035.8666573, + "value": 41592000.0 + }, + "window/rank/100k": { + "baseline": 355985666.6666667, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 498453579.30969954, + "value": 190150000.0 + }, + "window/rank/10k": { + "baseline": 25411366.666666668, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 39031195.01977421, + "value": 12836000.0 + }, + "window/rank/1k": { + "baseline": 2218803.3333333335, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3543127.412810543, + "value": 1181200.0 + }, + "window/row_number/100k": { + "baseline": 356653333.3333333, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 514485121.3871154, + "value": 195620000.0 + }, + "window/row_number/10k": { + "baseline": 25187133.333333332, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 38832650.84958584, + "value": 13335000.0 + }, + "window/row_number/1k": { + "baseline": 2216196.6666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3537587.4171026694, + "value": 1183400.0 + }, + "window/running_sum/100k": { + "baseline": 394806000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 568971333.3953856, + "value": 185350000.0 + }, + "window/running_sum/10k": { + "baseline": 25908933.333333332, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 39661122.833365336, + "value": 13605000.0 + }, + "window/running_sum/1k": { + "baseline": 2215966.6666666665, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ed-acab-77a8-824a-99b059685905", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3646472.6516460814, + "value": 1184700.0 + }, + "window_rows/10000/hundred/frame_bench_sum/two": { + "baseline": 32125500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2657269399.1395054, + "value": 34091000.0 + }, + "window_rows/10000/hundred/sum/prefix": { + "baseline": 30156000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3178784963.138867, + "value": 32820999.999999996 + }, + "window_rows/10000/hundred/sum/two": { + "baseline": 30556500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2849767531.8701797, + "value": 31746000.0 + }, + "window_rows/10000/hundred/sum/zero": { + "baseline": 31324500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2732000209.3216143, + "value": 31657000.0 + }, + "window_rows/10000/one/sum/prefix": { + "baseline": 30283500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2803183057.013996, + "value": 32002000.000000004 + }, + "window_rows/10000/one/sum/two": { + "baseline": 30662000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 3045318482.3049073, + "value": 31326000.0 + }, + "window_rows/10000/one/sum/zero": { + "baseline": 29901500.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 2764759488.3821306, + "value": 31514000.0 + }, + "window_rows/100000/hundred/frame_bench_sum/prefix": { + "baseline": 406600000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 20199242230.202297, + "value": 424690000.0 + }, + "window_rows/100000/hundred/max/prefix": { + "baseline": 408000000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 19401217961.85151, + "value": 424080000.0 + }, + "window_rows/100000/hundred/max/two": { + "baseline": 398720000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 21977661974.655094, + "value": 423940000.0 + }, + "window_rows/100000/hundred/min/prefix": { + "baseline": 405640000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 15627780723.56227, + "value": 429430000.0 + }, + "window_rows/100000/hundred/min/two": { + "baseline": 406220000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 17387094113.934006, + "value": 428700000.0 + }, + "window_rows/100000/hundred/sum/prefix": { + "baseline": 402800000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 20421486333.667004, + "value": 427810000.0 + }, + "window_rows/100000/hundred/sum/two": { + "baseline": 400170000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 23081766137.897564, + "value": 426580000.0 + }, + "window_rows/100000/hundred/sum/zero": { + "baseline": 398005000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 19156908952.162483, + "value": 428500000.0 + }, + "window_rows/100000/one/sum/prefix": { + "baseline": 375480000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 23161828283.405594, + "value": 413800000.0 + }, + "window_rows/100000/one/sum/two": { + "baseline": 385515000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 26317184284.05474, + "value": 417810000.0 + }, + "window_rows/100000/one/sum/zero": { + "baseline": 376115000.0, + "independent_runs": [ + 41908407297, + 41960942244, + 41965724402 + ], + "report": "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "testbed": "ubuntu-latest", + "upper_limit": 27107208552.405533, + "value": 418220000.0 + } +} diff --git a/docs/benchmarks/840-regression/completion-summary.json b/docs/benchmarks/840-regression/completion-summary.json new file mode 100644 index 0000000000..8b9130d56b --- /dev/null +++ b/docs/benchmarks/840-regression/completion-summary.json @@ -0,0 +1,142 @@ +{ + "runs": [ + { + "run": 41960942244, + "role": "full-v3", + "sha": "cdad3e7270d0e5e791cd48f47a4ed35ff50df6a4", + "workflow_sha": "25fd6fb9bdb9bc69021a217c1f755e1780c0eb07", + "first_report": "2026-11-05T03:23:00+00:00", + "last_report": "2026-11-05T04:06:07+00:00", + "selected": 453, + "reports": [ + "01a1a9c7-6874-73fc-8aa3-7f7c79d36fb1", + "01a1a9c0-4226-7233-adbc-b0d53a44e031", + "01a1a9dc-dea0-77b7-ba27-b86da6856a7e", + "01a1a9bd-b134-7228-ae60-c6d653aea55b", + "01a1a9c3-c278-734a-80a5-5c94f8d51652", + "01a1a9c3-e184-7316-b222-69c4e4eee0ec", + "01a1a9be-6ab1-73c5-bdfb-9c39e5f72446", + "01a1a9c0-6dcf-75b3-b2d8-602153f139c6", + "01a1a9c3-855f-72ff-8ba5-046ca03016ee", + "01a1a9bc-fe60-76f7-abd5-a9b56a9da101", + "01a1a9bf-137b-7582-84eb-a7c675327f8c", + "01a1a9bf-0c42-76ce-8aa8-747d5398bda9", + "01a1a9e2-aedc-7740-a63c-1f7be7141788", + "01a1a9c0-12ba-765d-8446-f4b92c3f5438", + "01a1a9c0-e687-7762-be36-526d09bf6386", + "01a1a9c5-c5f8-7057-be29-ff848258c1ba", + "01a1a9c4-f4c4-7585-837b-58417551fde8", + "01a1a9c1-939e-7036-a885-9a62cb69922f", + "01a1a9c0-fd66-715d-9de8-4c14a194968f", + "01a1a9bf-2efe-7627-9933-fec71e9b7f83", + "01a1a9c0-8ac4-73c5-8477-b90d75199f91", + "01a1a9be-0930-75b7-b999-5a24a037e1bd", + "01a1a9bd-935c-70dc-ba32-af403f3ff69b", + "01a1a9c3-256d-7138-a4a9-a638091dbe95", + "01a1a9bf-80ea-7418-a0ad-69bb4ae3a1b2", + "01a1a9be-d2dd-7165-8e2e-b465bb76aec5", + "01a1a9be-3455-7663-a634-fa70ad38648d", + "01a1a9bb-3696-7474-a433-fea9d395c0a4", + "01a1a9bb-71c7-756e-a675-aec2e82686c2", + "01a1a9bd-82ad-7346-9fce-97d0186c7fbc" + ], + "alerts": [] + }, + { + "run": 41965724402, + "role": "full-v3", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "workflow_sha": "25fd6fb9bdb9bc69021a217c1f755e1780c0eb07", + "first_report": "2026-11-05T04:09:59+00:00", + "last_report": "2026-11-05T04:56:53+00:00", + "selected": 453, + "reports": [ + "01a1a9ef-3db9-7278-88ca-5087cd7e1c6e", + "01a1a9ee-3069-71c9-b01b-13ef083655b8", + "01a1aa0a-9399-72a8-85ff-6791e5f7dd90", + "01a1a9e8-f32e-775a-ac89-4dfadf7b54ff", + "01a1a9e8-7323-7295-8c04-2ca4f4e1f3c4", + "01a1a9e8-f464-7225-ab41-0405bf038929", + "01a1a9e9-1534-76fb-bf8c-6aa8ecd85188", + "01a1a9e8-510f-7266-be06-79a92bbbdec0", + "01a1a9eb-ff8c-7374-9ccf-f0663c524097", + "01a1a9eb-08d5-7106-b75c-a4d98467b778", + "01a1a9ec-cd9c-70b2-9bac-59d341319432", + "01a1a9e8-e2aa-774b-bbb5-1321d9e4ab27", + "01a1aa11-2ad0-75b4-84a2-68464e44d805", + "01a1a9ea-44bf-72fb-95ca-88cc4b053495", + "01a1a9ec-d0a5-7413-b578-06186b6fbfbf", + "01a1a9ed-acab-77a8-824a-99b059685905", + "01a1a9ee-194c-73c9-b9ce-0f666e50fdc1", + "01a1a9e9-3eaa-74ca-8031-deba90a2b017", + "01a1a9e8-8a30-76e1-9c00-6ac45086112c", + "01a1a9e7-b8c7-722d-97cb-e2cb3d1cb892", + "01a1a9ea-4b8a-7666-a873-826fb9488ed6", + "01a1a9e9-5cd8-72d4-9b96-b91ee7c6d763", + "01a1a9e8-3821-7122-870f-13482be89335", + "01a1a9ee-63a8-717d-985d-5fbe2a5806ce", + "01a1a9eb-a076-736f-9c60-d3a45b0923a6", + "01a1a9e9-91fe-708a-895c-346483052a19", + "01a1a9e8-62ae-73d6-a1ba-fceab1ee1762", + "01a1a9e6-ef5a-76c2-a7a5-5ea3040f552f", + "01a1a9e6-37a1-7475-a2e6-402006843f6e", + "01a1a9e8-7388-700a-aeb7-13730e791aa3" + ], + "alerts": [ + "concurrent_file/serialized_writers/100k_16t", + "concurrent_file/serialized_writers/100k_2t", + "concurrent_file/serialized_writers/100k_4t" + ] + }, + { + "run": 41970954512, + "role": "c-probe", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "workflow_sha": "44a8edb949a0b5a1e0b5cbc096ee8e7478216817", + "first_report": "2026-11-05T05:00:04+00:00", + "last_report": "2026-11-05T05:00:04+00:00", + "selected": 6, + "reports": [ + "01a1aa14-152a-74d1-8c9a-591746bd353a" + ], + "alerts": [] + }, + { + "run": 41971233145, + "role": "main-probe", + "sha": "25fd6fb9bdb9bc69021a217c1f755e1780c0eb07", + "workflow_sha": "44a8edb949a0b5a1e0b5cbc096ee8e7478216817", + "first_report": "2026-11-05T05:04:29+00:00", + "last_report": "2026-11-05T05:04:29+00:00", + "selected": 4, + "reports": [ + "01a1aa18-1dc7-77c2-91f5-b12b7c03749d" + ], + "alerts": [] + }, + { + "run": 41971706654, + "role": "v3-probe", + "sha": "5a0ff549271084018c7d4e6050658e64e7e6432e", + "workflow_sha": "44a8edb949a0b5a1e0b5cbc096ee8e7478216817", + "first_report": "2026-11-05T05:09:58+00:00", + "last_report": "2026-11-05T05:09:58+00:00", + "selected": 10, + "reports": [ + "01a1aa1d-240c-757f-a595-9b75dd430226" + ], + "alerts": [] + } + ], + "native_numeric": 427, + "binding_numeric": 26, + "missing": 0, + "head": "01a1a84c-e947-745c-98e7-018e0c2245a3", + "temporary_probe_ref_deleted": true, + "model": { + "test": "t_test", + "upper_probability": 0.999, + "max_sample_size": 30 + }, + "minimum_independent_v3_runs": 3 +} diff --git a/docs/benchmarks/840-regression/completion-verification.txt b/docs/benchmarks/840-regression/completion-verification.txt new file mode 100644 index 0000000000..1babd65710 --- /dev/null +++ b/docs/benchmarks/840-regression/completion-verification.txt @@ -0,0 +1,10 @@ +Verification date: 2026-11-05 + +evidence.tar.gz: 746 internal checksums passed +live-notifications.tar.gz: 196 internal checksums passed +completion.tar.gz: 558 internal checksums passed +All eight derived outputs reproduced byte-for-byte from fresh extraction +Negative control: a missing numeric comparison is rejected +Negative control: substituting inherited main history for a v3 observation is rejected +Negative control: missing independently read notification delivery is rejected +All three verifiers refuse optimized Python diff --git a/docs/benchmarks/840-regression/completion.md b/docs/benchmarks/840-regression/completion.md new file mode 100644 index 0000000000..c04254aa8b --- /dev/null +++ b/docs/benchmarks/840-regression/completion.md @@ -0,0 +1,193 @@ +# Regression-history follow-up + +The production history fix is merged in [main #853](https://github.com/project-minigraf/minigraf/pull/853) +(`25fd6fb9bdb9bc69021a217c1f755e1780c0eb07`) and +[v3 #854](https://github.com/project-minigraf/minigraf/pull/854) +(`fc0a61545addd84eb0d6c300965082a6bf189101`). Both passed their required CI before +merge. Persistent main/v3 uploads now omit main start-point inheritance; ad hoc +comparison branches retain it. This preserves the existing v3 head and its independent observations. + +The [initial audit](README.md) is a frozen pre-fix snapshot. Its null comparisons +and notification-policy fixtures describe that snapshot, not the subsequent +live checks below. This follow-up completes the evidence for #840. + +## Fresh independent measurements + +All **427 native and 26 binding IDs** now have finite numeric comparison baselines +and upper limits on their intended v3 testbeds. Each ID is traced to at least +**three independent v3 workflow runs** on head +`01a1a84c-e947-745c-98e7-018e0c2245a3`. The original 244 native and 26 binding +comparison gaps are closed. [comparison-inventory.json](comparison-inventory.json) +records every ID's value, baseline, upper limit, testbed, source hash, report UUID +and independent run IDs; [completion-summary.json](completion-summary.json) +records the five new dispatches. + +| Run | Scope | IDs | Measured core SHA | Workflow SHA | Alerts | +| --- | --- | --- | --- | --- | --- | +| [41960942244](https://github.com/project-minigraf/minigraf/actions/runs/41960942244) | Full v3 | 453 | `cdad3e7270d0e5e791cd48f47a4ed35ff50df6a4` | `25fd6fb9bdb9bc69021a217c1f755e1780c0eb07` | 0 | +| [41965724402](https://github.com/project-minigraf/minigraf/actions/runs/41965724402) | Full v3 | 453 | `5a0ff549271084018c7d4e6050658e64e7e6432e` | `25fd6fb9bdb9bc69021a217c1f755e1780c0eb07` | 3 | +| [41970954512](https://github.com/project-minigraf/minigraf/actions/runs/41970954512) | C v3 | 6 | `5a0ff549271084018c7d4e6050658e64e7e6432e` | `44a8edb949a0b5a1e0b5cbc096ee8e7478216817` | 0 | +| [41971233145](https://github.com/project-minigraf/minigraf/actions/runs/41971233145) | UDF main | 4 | `25fd6fb9bdb9bc69021a217c1f755e1780c0eb07` | `44a8edb949a0b5a1e0b5cbc096ee8e7478216817` | 0 | +| [41971706654](https://github.com/project-minigraf/minigraf/actions/runs/41971706654) | UDF v3 | 10 | `5a0ff549271084018c7d4e6050658e64e7e6432e` | `44a8edb949a0b5a1e0b5cbc096ee8e7478216817` | 0 | + +Every selection is nonempty and disjoint within its run. The offline verifier +requires selected = raw executed = raw uploaded = report/API IDs, checks core, +binding and workflow provenance, and verifies every stored 0.999/30 model. +Native uses `ubuntu-latest`; bindings retain their four +`binding--ubuntu24-v1` testbeds. + +Two complete production-workflow v3 dispatches add fresh observations to the +retained original scheduled run. C needs one additional real C-only dispatch: +its original scheduled report belongs to the previous head and cannot count +here. All measurements use normal benchmark code and source pinning on +GitHub-hosted runners; none replay a log or upload a fixture. Existing inherited +main observations are excluded from the independent-v3 counts. + +The perf query starts at the earliest actual v3 report timestamp, 2026-11-04 +20:42:54 UTC. That report started one second before its upload created the head +at 20:42:55. A head-creation cutoff initially excluded its six B-tree observations; +`initial-query-window/` retains that diagnostic snapshot. The verifier derives +the corrected cutoff from the original API reports and still requires every +counted observation to have known, independent v3 provenance. + +The real main UDF probe advances Bencher's measured main version from 84 +(`e7923eb710b1e1d8aff517721643c0b0b1f2cb7a`) to 85 +(`25fd6fb9bdb9bc69021a217c1f755e1780c0eb07`); a subsequent v3 UDF upload +preserves the same v3 head. Six before/after branch API snapshots +and both actual reports verify preservation across a measured main advance. The disposable probe workflow and its exact source +revision are retained in the archive. + +These are initial numeric comparisons, not thirty completed independent runs. +With only a few observations, the limits can be wide; future independent nightly +runs continue to build the history. No latency percentage budget is claimed. + +Reported latency alerts (retained as real results): + +- Run 41965724402: `concurrent_file/serialized_writers/100k_16t` +- Run 41965724402: `concurrent_file/serialized_writers/100k_2t` +- Run 41965724402: `concurrent_file/serialized_writers/100k_4t` + +| File-writer case | Value (ms) | Baseline (ms) | Upper limit (ms) | +| --- | --- | --- | --- | +| 100k, 2 threads | 1.5517 | 0.417969 | 1.0169 | +| 100k, 4 threads | 3.2361 | 0.85936 | 2.11223 | +| 100k, 16 threads | 10.804 | 3.58459 | 9.25215 | + +The retained production job log records no alerts in the preceding run and +suppresses notification under the two-run policy. These are real threshold +crossings; this audit does not establish their performance cause. The verifier +checks the actual reporting-step outcome and timestamped suppression message. + +Expansion to Java/JVM, WASI and qualified Swift/iOS simulator or Android emulator +lanes is tracked separately in [#858](https://github.com/project-minigraf/minigraf/issues/858). +The #839 platform restriction applies only to that manual evidence task; mobile +nightly inclusion depends on measured agreement with physical-device workloads. +This follow-up verifies the existing matrix and does not claim coverage of those +additional runtimes. + +## Actual scheduled routes + +The initial archive retains actual main and v3 `schedule` events, measured and +workflow hashes, and exact nonempty/disjoint selected, executed and uploaded +sets. Main covers 177 native IDs; v3 covers 427 native and 26 binding IDs across +all four binding jobs. The history fix changes upload ancestry, not schedules, +selection, measured-ref resolution or notification routing. The fresh runs are +explicitly dispatches; they are not presented as new scheduled events. The +observed schedules were delayed relative to their nominal 02:00/14:00 UTC crons. + +## Live notification delivery + +[The live summary](live-summary.json) records 16 job scenarios: eight for each +unchanged production notification body. The temporary workflow used source +`437d81e3566c5bb052981d65304e938873ca5eed` and isolated Bencher branches +`audit-840-20261105` and `audit-840-20261105-other`, dedicated native/binding +testbeds, and IDs under `audit/840/20261105/`. No synthetic value was written to +main/v3 or their production testbeds. + +| Scenario | Actual result for both native and binding | +| --- | --- | +| First alert | Real Bencher alert; no issue/comment | +| Intervening different branch | Real alert; no issue/comment | +| Repeated same ID/branch/suite | Issue created; previous matching branch selected despite intervening other branch | +| Different suite | No previous matching job; no issue/comment | +| Reset first alert | No delivery after the intervening different suite | +| Different benchmark ID | No repeated-ID delivery | +| Previous-log lookup failure | Actual GitHub 404; warning and empty prior-alert set; no delivery | +| Failed upload | Actual Bencher 404; immediate comment; completeness audit fails as intended | + +The repeated-alert run created [#856](https://github.com/project-minigraf/minigraf/issues/856) +and [#857](https://github.com/project-minigraf/minigraf/issues/857). Failed uploads +added [the binding comment](https://github.com/project-minigraf/minigraf/issues/856#issuecomment-6648972167) +and [the native comment](https://github.com/project-minigraf/minigraf/issues/857#issuecomment-6648970917). +Independent GitHub reads matched all four exact message bodies. The expected +failed-upload workflow failure is retained, not classified as a failed experiment. + +For isolation, the driver redirected the notification body's hardcoded physical +workflow lookup from `main` to `audit/840-live-checks`. The script bodies themselves +are byte-checked against the production workflow. These checks establish real +GitHub delivery and prior-job log retrieval within that isolated scope; they do +not establish unmodified default-branch lookup or receipt in a human mailbox. + +## Cleanup and durable evidence + +After the original source/API evidence was committed and pushed, cleanup run +[41963024134](https://github.com/project-minigraf/minigraf/actions/runs/41963024134) +deleted exactly 34 synthetic reports, four thresholds, two branches, two testbeds +and four benchmarks. Guard tests rejected foreign production resource names. +All 46 resources returned 404 after deletion, verified again by an independent +reader. Both test issues are closed. The notification workflow ref is deleted +and returns 404; all five production v3 threshold models still have 0.999/30. +The history-probe workflow ref was also deleted and independently returned 404 +only after its source and evidence were pushed in `9e1a5e8ad`. Historical Actions +run records remain as an audit trail; real Bencher observations are retained. + +`completion.tar.gz` retains all five fresh runs, their exact workflow sources, +63 public report responses, complete active-head queries, six head snapshots, +the query-window diagnostic, production alert logs and probe-ref cleanup proof. +[completion-verification.txt](completion-verification.txt) records fresh-extraction +reproduction and rejection of missing comparisons, inherited-main substitutions +and missing delivery evidence. + +`live-notifications.tar.gz` preserves source, real CLI output, full job logs, +complete paginated metadata, Bencher API responses, notification receipts, +independent read-backs and cleanup receipts. Its internal checksums and the +adjacent `SHA256SUMS` make the retained evidence independent of Actions' 30-day +artifact expiry. `verify-live.py` reproduces the live summary entirely offline. + +The baseline model is a latency t-test with upper probability **0.999** and +maximum sample size **30**. The probability is not a latency percentage budget; +the maximum is not evidence of 30 independent runs. Fresh-run provenance must +be established separately, and inherited main history never counts as an +independent v3 run. + +## Offline reproduction + +From this directory: + +```sh +sha256sum -c SHA256SUMS +tar -xzf evidence.tar.gz +tar -xzf live-notifications.tar.gz +tar -xzf completion.tar.gz +(cd evidence && sha256sum -c SHA256SUMS) +(cd live-evidence && sha256sum -c SHA256SUMS) +(cd completion-evidence && sha256sum -c SHA256SUMS) +python3 audit.py +node notification-preview.cjs +python3 verify-live.py +python3 verify-completion.py +``` + +The verifiers reject optimized Python (`-O`) because assertions are part of the +evidence checks. `audit.py` reproduces the initial snapshot; `verify-live.py` +reproduces notification/cleanup findings; `verify-completion.py` reproduces the +fresh-run comparison inventory and history-preservation result. Each archive +contains its own complete file manifest. + +For a network refresh, stage the retained `runs.json`, six `head-probes/*.json` +files and two `orchestration/*.py` files together in one input directory. Run +`collect-completion.py INPUT/runs.json` from a fresh copy of the collector together +with `inventory.json` and `evidence.tar.gz`: it caches files in its adjacent +`completion-evidence/` directory for resumable collection. Preserve the frozen archives first. Later real observations can +legitimately change the active history and require review of the frozen verifier's +expected provenance. diff --git a/docs/benchmarks/840-regression/completion.tar.gz b/docs/benchmarks/840-regression/completion.tar.gz new file mode 100644 index 0000000000..96c51e07b1 Binary files /dev/null and b/docs/benchmarks/840-regression/completion.tar.gz differ diff --git a/docs/benchmarks/840-regression/live-checks.md b/docs/benchmarks/840-regression/live-checks.md index e49a33a5fd..c98d49464d 100644 --- a/docs/benchmarks/840-regression/live-checks.md +++ b/docs/benchmarks/840-regression/live-checks.md @@ -1,7 +1,9 @@ # Controlled live notification checks -Status: prepared protocol, not executed. This is the remaining delivery work for -#840. The real nightly histories must receive only real measurements; synthetic +Status: executed on 2026-11-05 with explicit user authorization. +[Live results](live-summary.json) verify 16 job scenarios, four independently +read-back deliveries, and cleanup of all 46 disposable Bencher resources. Test +issues #856/#857 are closed and the temporary notification workflow ref is deleted. The real nightly histories must receive only real measurements; synthetic values belong exclusively to disposable audit history. ## Scope and externally visible messages @@ -65,7 +67,7 @@ watchers; closing an issue does not retract delivered mail. temporary workflow ref. Never delete normal run history. Close the two test issues. Re-query to verify cleanup, and leave #840 open for any unmet gate. -Authorization needed: merge the separately reviewed workflow fixes after CI -passes; create and close the clearly scoped test issues/comments and disposable -test history. Repository issue #840 explicitly requires a separate review for -production changes, and external test notifications need explicit permission. +Authorization recorded: the user approved merging after CI and running the +isolated notification tests, including creating and closing their test issues. +The separately reviewed fixes #853/#854 passed CI and were merged before these +checks. The protocol above is retained to explain the experiment and its limits. diff --git a/docs/benchmarks/840-regression/live-notifications.tar.gz b/docs/benchmarks/840-regression/live-notifications.tar.gz new file mode 100644 index 0000000000..252e104cb5 Binary files /dev/null and b/docs/benchmarks/840-regression/live-notifications.tar.gz differ diff --git a/docs/benchmarks/840-regression/live-summary.json b/docs/benchmarks/840-regression/live-summary.json new file mode 100644 index 0000000000..49cf7d5b22 --- /dev/null +++ b/docs/benchmarks/840-regression/live-summary.json @@ -0,0 +1,151 @@ +{ + "scenarios": [ + { + "stage": "first", + "kind": "binding", + "run": 41961640029, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "first", + "kind": "native", + "run": 41961640029, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "other-branch", + "kind": "binding", + "run": 41961995516, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "other-branch", + "kind": "native", + "run": 41961995516, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "repeat", + "kind": "binding", + "run": 41962121762, + "writes": 1, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "repeat", + "kind": "native", + "run": 41962121762, + "writes": 1, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "other-suite", + "kind": "binding", + "run": 41962206753, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "other-suite", + "kind": "native", + "run": 41962206753, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "reset-first", + "kind": "binding", + "run": 41962254895, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "reset-first", + "kind": "native", + "run": 41962254895, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "different-id", + "kind": "binding", + "run": 41962338519, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "different-id", + "kind": "native", + "run": 41962338519, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "lookup-error", + "kind": "binding", + "run": 41962460219, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "lookup-error", + "kind": "native", + "run": 41962460219, + "writes": 0, + "upload_status": "alerts", + "audit_errors": [] + }, + { + "stage": "upload-failure", + "kind": "binding", + "run": 41962507803, + "writes": 1, + "upload_status": "error", + "audit_errors": [ + "Upload: missing audit/840/20261105/binding/primary, audit/840/20261105/binding/secondary", + "Bencher outcome: failure (no alerts)" + ] + }, + { + "stage": "upload-failure", + "kind": "native", + "run": 41962507803, + "writes": 1, + "upload_status": "error", + "audit_errors": [ + "Upload: missing audit/840/20261105/native/primary, audit/840/20261105/native/secondary", + "Bencher outcome: failure (no alerts)" + ] + } + ], + "deliveries": [ + "https://github.com/project-minigraf/minigraf/issues/856", + "https://github.com/project-minigraf/minigraf/issues/857", + "https://github.com/project-minigraf/minigraf/issues/856#issuecomment-6648972167", + "https://github.com/project-minigraf/minigraf/issues/857#issuecomment-6648970917" + ], + "synthetic_reports": 34, + "cleanup_resources": 46, + "test_issues_closed": [ + 856, + 857 + ], + "limitation": "Previous workflow lookup redirects main to audit/840-live-checks; GitHub delivery, not human mailbox receipt." +} diff --git a/docs/benchmarks/840-regression/live-verification.txt b/docs/benchmarks/840-regression/live-verification.txt new file mode 100644 index 0000000000..21a48c424e --- /dev/null +++ b/docs/benchmarks/840-regression/live-verification.txt @@ -0,0 +1,31 @@ +Verification date: 2026-11-05 + +Fresh temporary extraction: +- evidence.tar.gz: 746 internal SHA-256 checksums passed. +- live-notifications.tar.gz: 196 internal SHA-256 checksums passed. +- python3 audit.py, node notification-preview.cjs and python3 verify-live.py passed. +- inventory.json, missing-baselines.json, active-head-summary.json, summary.json, + notification-preview.json and live-summary.json reproduced byte-for-byte. +- Negative control: replacing the independently retrieved issue-856-comments.json + with an empty array causes verify-live.py to reject the evidence. + +Live verifier result: +- 16 native/binding notification scenarios. +- Four actual deliveries matched independent GitHub reads. +- 34 synthetic Bencher report responses retained. +- All 46 disposable resources deleted and independently verified absent. +- Test issues #856/#857 closed; temporary notification workflow ref absent. +- Five production v3 threshold models preserved at t_test / 0.999 / 30. + +Lightweight workflow regression checks: +node --test .github/bench/history.test.cjs .github/bench/report.test.cjs .github/bench/bindings-report.test.cjs +All three files passed. + +The live summary sorts artifact paths explicitly so a fresh archive extraction +reproduces the same output regardless of filesystem directory iteration order. + +Optimized-mode negative control: +- Before the guard, python3 -O verify-live.py accepted the missing-comment fixture. +- audit.py, verify-live.py and verify-completion.py now refuse Python -O explicitly. +- All three refusals passed; normal fresh-extraction verification and byte-for-byte + regeneration still pass. diff --git a/docs/benchmarks/840-regression/verify-completion.py b/docs/benchmarks/840-regression/verify-completion.py new file mode 100644 index 0000000000..af4b1b4e5a --- /dev/null +++ b/docs/benchmarks/840-regression/verify-completion.py @@ -0,0 +1,216 @@ +#!/usr/bin/env python3 +"""Verify real-run uploads and active numeric comparisons for every intended ID.""" +from datetime import datetime +import json +import math +from pathlib import Path +import re +import tarfile +from urllib.parse import parse_qs, urlparse + +if not __debug__: + raise RuntimeError('Evidence verification requires Python assertions; do not use -O') + +ROOT = Path(__file__).resolve().parent +BASE = ROOT / 'completion-evidence' +HEAD = '01a1a84c-e947-745c-98e7-018e0c2245a3' + + +def read(path): + return json.loads(path.read_text()) + + +def exact(expected, actual): + assert expected and len(expected) == len(set(expected)) + assert len(actual) == len(set(actual)) and set(actual) == set(expected) + + +def finite(value): + return type(value) in (int, float) and math.isfinite(value) + + +def model(measure): + m = measure['threshold']['model'] + assert (m['test'], m['upper_boundary'], m['max_sample_size']) == ('t_test', .999, 30) + + +def main(): + original = read(ROOT / 'inventory.json') + expected = original['41908407297'] + plan = read(BASE / 'runs.json') + assert [p['role'] for p in plan] == ['full-v3', 'full-v3', 'c-probe', 'main-probe', 'v3-probe'] + assert len({p['run'] for p in plan}) == 5 + provenance = {v['report']: dict(run=41908407297, sha=v['sha']) for v in expected.values() if v['head'] == HEAD} + observations = {} + summaries = [] + previous_end = None + for item in plan: + folder = BASE / str(item['run']) + metadata = read(folder / 'run.json') + assert metadata['event'] == 'workflow_dispatch' and metadata['conclusion'] == 'success' + jobs = [j for page in read(folder / 'jobs.json') for j in page['jobs']] + assert all(j['conclusion'] in ('success', 'skipped') for j in jobs) + paths = sorted(folder.glob('artifacts/*/bench-report.json')) + sorted(folder.glob('artifacts/*/binding-report.json')) + union = [] + report_ids = [] + alerted = [] + report_starts, report_ends = [], [] + sha_set = set() + branch = 'main' if item['role'] == 'main-probe' else 'v3' + for path in paths: + report = read(path) + binding = 'binding' in report + selected = report['selected'] + alerted.extend(report['alerts']) + assert report['ref'] == branch + assert not report['errors'] and report['uploadStatus'] in ('success', 'alerts') + assert report['workflowSha'] == metadata['head_sha'] + if binding: + assert report['historyBranch'] == branch + assert report['bindingSha'] == (path.parent / 'binding-sha.txt').read_text().strip() + executed = re.findall(r'^ok: (.+)$', (path.parent / 'binding-execution.txt').read_text(), re.M) + metrics = read(path.parent / 'binding/binding-results.json') + exact(selected, list(metrics)) + assert all(finite(m['latency']['value']) and m['latency']['value'] > 0 for m in metrics.values()) + output = (path.parent / 'binding-bencher.txt').read_text() + sha = report['coreSha'] + testbed = report['testbed'] + else: + selection = read(path.parent / 'bench-selection.json') + assert all(report[k] == v for k, v in selection.items()) + listed = re.findall(r'^(.+): benchmark\s*$', (path.parent / 'bench-list.txt').read_text(), re.M) + exact(selected, [name for name in listed if re.search(report['filter'], name)]) + logs = list(path.parent.glob('bench_*.txt')) + assert len(logs) == 1 + executed = re.findall(r'^Benchmarking (.+): Analyzing\s*$', logs[0].read_text(), re.M) + output = (path.parent / 'bencher_out.txt').read_text() + sha = report['sha'] + testbed = 'ubuntu-latest' + assert all(expected[name]['testbed'] == testbed for name in selected) + sha_set.add(sha) + uploaded = re.findall(r'^- (.+) \(Latency\): https://bencher.dev/console/projects/[^/\s]+/perf\?', output, re.M) + for actual in (executed, uploaded, report['executed'], report['uploaded']): + exact(selected, actual) + urls = re.findall(r'https://bencher.dev/console/projects/minigraf/perf\?\S+', output) + ids = {parse_qs(urlparse(url).query)['report'][0] for url in urls} + assert len(ids) == 1 + report_id = ids.pop() + report_ids.append(report_id) + api = read(folder / 'api' / (report_id + '.json')) + report_starts.append(datetime.fromisoformat(api['start_time'])) + report_ends.append(datetime.fromisoformat(api['end_time'])) + assert api['branch']['name'] == branch + assert api['branch']['head']['version']['hash'] == sha + assert api['testbed']['name'] == testbed + if branch == 'v3': + assert api['branch']['head']['uuid'] == HEAD + provenance[report_id] = dict(run=item['run'], sha=sha) + else: + assert sha != 'e7923eb710b1e1d8aff517721643c0b0b1f2cb7a' + assert api['branch']['head']['version']['number'] > 84 + api_ids = [] + for iteration in api['results']: + for row in iteration: + name = row['benchmark']['name'] + assert len(row['measures']) == 1 + measure = row['measures'][0] + assert measure['measure']['slug'] == 'latency' + model(measure) + api_ids.append(name) + assert finite(measure['metric']['value']) and measure['metric']['value'] > 0 + if binding: + assert measure['metric']['value'] == metrics[name]['latency']['value'] + observations[(report_id, name)] = measure + exact(selected, api_ids) + union.extend(selected) + assert len(report_ids) == len(set(report_ids)) + assert len(sha_set) == 1 + if item['role'] == 'full-v3': + assert len(paths) == 30 + exact(list(expected), union) + elif item['role'] == 'c-probe': + exact([name for name in expected if name.startswith('binding/c/')], union) + elif item['role'] == 'v3-probe': + exact([name for name in expected if name.startswith('udf/')], union) + else: + exact([name for name in original['41801278030'] if name.startswith('udf/')], union) + if item['run'] == 41965724402: + assert sorted(alerted) == [ + 'concurrent_file/serialized_writers/100k_16t', + 'concurrent_file/serialized_writers/100k_2t', + 'concurrent_file/serialized_writers/100k_4t'] + job = next(j for j in jobs if j['name'] == 'bench / concurrent_file/serialized_writers') + assert any(step['name'] == 'Report regression' and step['conclusion'] == 'success' for step in job['steps']) + log = (folder / f'notification-job-{job["id"]}.txt').read_text() + assert re.search(r'^2026-11-05T\S+ Alerts in previous run: none$', log, re.M) + assert re.search(r'^2026-11-05T\S+ .*Bencher flagged 3 benchmark\(s\).*Not reporting\.$', log, re.M) + if previous_end is not None: + assert previous_end < min(report_starts), 'Fresh reports must follow the preceding run' + previous_end = max(report_ends) + summaries.append(dict(**item, sha=sha_set.pop(), workflow_sha=metadata['head_sha'], + first_report=min(report_starts).isoformat(), last_report=previous_end.isoformat(), + selected=len(union), reports=report_ids, alerts=sorted(alerted))) + active = BASE / 'active-head' + branch = read(active / 'branch.json') + assert branch['head']['uuid'] == HEAD + prior_ids = {v['report'] for v in expected.values() if v['head'] == HEAD} + with tarfile.open(ROOT / 'evidence.tar.gz') as archive: + prior = [json.load(archive.extractfile(f'evidence/41908407297/api/{report}.json')) + for report in prior_ids] + assert all(r['branch']['name'] == 'v3' and r['branch']['head']['uuid'] == HEAD for r in prior) + earliest = min(r['start_time'] for r in prior) + start = int(datetime.fromisoformat(earliest).timestamp() * 1000) + assert read(active / 'window.json') == dict(head_created=branch['head']['created'], + earliest_actual_v3_report_start=earliest, start_time_ms=start) + inventory = {} + requested = [] + for index, query in enumerate(read(active / 'queries.json')): + params = parse_qs(urlparse(query).query) + ids = params['benchmarks'][0].split(',') + assert len(ids) <= 8 and params['heads'] == [HEAD] + assert params['branches'] == [branch['uuid']] and params['start_time'] == [str(start)] + requested.extend(ids) + data = read(active / f'{index:02d}.json') + for series in data['results']: + name = series['benchmark']['name'] + assert name in expected and name not in inventory + assert series['branch']['name'] == 'v3' and series['branch']['head']['uuid'] == HEAD + assert series['testbed']['name'] == expected[name]['testbed'] + real = [m for m in series['metrics'] if m['report'] in provenance] + assert len(real) == len(series['metrics']), 'Unattributed history cannot count as independent v3 runs' + independent = {provenance[m['report']]['run'] for m in real} + assert len(independent) >= 3, name + for metric in real: + assert metric['version']['hash'] == provenance[metric['report']]['sha'] + model(metric) + latest = max(real, key=lambda m: m['start_time']) + assert (latest['report'], name) in observations + model(latest) + boundary = latest['boundary'] + assert finite(boundary['baseline']) and finite(boundary['upper_limit']), name + stored = observations[(latest['report'], name)] + assert latest['metric']['value'] == stored['metric']['value'] + assert boundary == stored['boundary'] + inventory[name] = dict(testbed=series['testbed']['name'], independent_runs=sorted(independent), + report=latest['report'], sha=latest['version']['hash'], value=latest['metric']['value'], + baseline=boundary['baseline'], upper_limit=boundary['upper_limit']) + exact(list(expected), list(inventory)) + assert len(requested) == len(set(requested)) == 453 + for filename in ('before-c-probe.json', 'after-c-probe.json', 'before-main-probe.json', + 'after-main-probe.json', 'before-v3-probe.json', 'after-v3-probe.json'): + assert read(BASE / 'head-probes' / filename)['head']['uuid'] == HEAD + cleanup = read(BASE / 'branch-cleanup.json') + assert cleanup['ref'] == 'refs/heads/audit/840-history-probe' + assert cleanup['last_sha'] == summaries[-1]['workflow_sha'] + assert cleanup['readback_exit_code'] != 0 and 'HTTP 404' in cleanup['readback_error'] + summary = dict(runs=summaries, native_numeric=427, binding_numeric=26, missing=0, head=HEAD, + temporary_probe_ref_deleted=True, + model=dict(test='t_test', upper_probability=.999, max_sample_size=30), + minimum_independent_v3_runs=min(len(v['independent_runs']) for v in inventory.values())) + (ROOT / 'completion-summary.json').write_text(json.dumps(summary, indent=2) + '\n') + (ROOT / 'comparison-inventory.json').write_text(json.dumps(inventory, indent=2, sort_keys=True) + '\n') + print('Verified all 453 numeric comparisons, at least three independent v3 runs per ID, and unchanged v3 head after main advanced') + + +if __name__ == '__main__': + main() diff --git a/docs/benchmarks/840-regression/verify-live.py b/docs/benchmarks/840-regression/verify-live.py new file mode 100644 index 0000000000..3102a54cdf --- /dev/null +++ b/docs/benchmarks/840-regression/verify-live.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +"""Offline verification of actual #840 notification delivery evidence.""" +import json +from pathlib import Path +import re +from urllib.parse import parse_qs, urlparse + +if not __debug__: + raise RuntimeError('Evidence verification requires Python assertions; do not use -O') + +ROOT = Path(__file__).resolve().parent +BASE = ROOT / 'live-evidence' +SESSION = 'audit-840-20261105' + + +def read(path): + return json.loads(path.read_text()) + + +def main(): + workflow = (BASE / 'sources/production-bench.yml').read_text() + bodies = [re.sub(r'^ ', '', body, flags=re.M) + for body in re.findall(r' script: \|\n((?: .*\n|\n)+)', workflow)] + assert len(bodies) == 2 + for name, body in zip(('native', 'binding'), bodies): + expected = 'module.exports = async function ({github, context, core}) {\n' + body + '\n};\n' + assert (BASE / 'sources' / (name + '.cjs')).read_text() == expected + runs = read(BASE / 'notifications/runs.json') + plan = read(BASE / 'sources/cleanup-plan.json') + reports = {r['uuid']: r for r in plan['reports']} + assert len(reports) == 34 + for report in reports.values(): + assert report['branch']['name'] in (SESSION, SESSION + '-other') + assert report['branch']['head']['start_point'] is None + assert report['testbed']['name'] in (SESSION + '-native', SESSION + '-binding') + for iteration in report['results']: + for row in iteration: + assert row['benchmark']['name'].startswith('audit/840/20261105/') + for measure in row['measures']: + model = measure['threshold']['model'] + assert (model['test'], model['upper_boundary'], model['max_sample_size']) == ('t_test', .999, 30) + summary = [] + delivered = [] + by_stage = {r['stage']: r['run'] for r in runs} + for entry in runs: + run, stage = entry['run'], entry['stage'] + folder = BASE / 'notifications' / str(run) + metadata = read(folder / 'run.json') + assert metadata['head_sha'] == '437d81e3566c5bb052981d65304e938873ca5eed' + assert metadata['event'] == 'workflow_dispatch' + assert metadata['conclusion'] == ('failure' if stage == 'upload-failure' else 'success') + jobs = [j for page in read(folder / 'jobs.json') for j in page['jobs']] + assert len(jobs) == 2 + artifacts = sorted((folder / 'artifacts').glob('audit-840-*')) + assert len(artifacts) == 2 + for artifact in artifacts: + kind = artifact.name.rsplit('-', 1)[-1] + suite = 'audit_840_native' if kind == 'native' else 'binding_audit_840' + if stage == 'other-suite': + suite += '_other' + job = next(j for j in jobs if j['name'] == 'bench / ' + suite) + if stage == 'seed': + assert len(list(artifact.glob('seed-*.txt'))) == 10 + continue + receipt = read(artifact / 'notification-receipt.json') + audit = read(artifact / 'bench-report.json') + output = (artifact / 'bencher_out.txt').read_text() + expected_writes = 1 if stage in ('repeat', 'upload-failure') else 0 + assert len(receipt['writes']) == expected_writes + assert next(s for s in job['steps'] if s['name'] == 'Report regression')['conclusion'] == 'success' + if stage == 'upload-failure': + assert 'Status: 404 Not Found' in output + assert '00000000-0000-0000-0000-000000000840' in output + assert audit['uploadStatus'] == 'error' and audit['errors'] and not audit['uploaded'] + assert next(s for s in job['steps'] if s['name'] == 'Verify reporting and summarize')['conclusion'] == 'failure' + else: + assert audit['uploadStatus'] == 'alerts' and not audit['errors'] + assert set(audit['selected']) == set(audit['executed']) == set(audit['uploaded']) + urls = re.findall(r'https://bencher.dev/console/projects/minigraf/perf\?\S+', output) + report_ids = {parse_qs(urlparse(url).query)['report'][0] for url in urls} + assert len(report_ids) == 1 + report = reports[report_ids.pop()] + assert report['testbed']['name'] == SESSION + '-' + kind + assert report['branch']['name'] == SESSION + ('-other' if stage == 'other-branch' else '') + assert report['counts']['alerts']['total'] == 1 + for measure in [m for rows in report['results'] for row in rows for m in row['measures']]: + assert measure['boundary']['baseline'] is not None + assert measure['boundary']['upper_limit'] is not None + if stage == 'repeat': + previous = next(r for r in receipt['records'] if r['name'] == 'listJobsForWorkflowRun') + assert previous['args']['run_id'] == by_stage['first'], 'Must skip the intervening other branch' + if stage == 'other-suite': + assert not any(r['name'] == 'downloadJobLogsForWorkflowRun' for r in receipt['records']) + if stage == 'lookup-error': + assert any(r['name'] == 'downloadJobLogsForWorkflowRun' and r.get('status') == 404 for r in receipt['records']) + assert any('Could not read' in w for w in receipt['warnings']) + for write in receipt['writes']: + issue = 857 if kind == 'native' else 856 + assert f'/issues/{issue}' in write['url'] + independent = read(BASE / f'issue-{issue}.json') + assert independent['title'] == write['title'] + if stage == 'repeat': + assert independent['body'] == write['body'] + else: + comments = [c for page in read(BASE / f'issue-{issue}-comments.json') for c in page] + assert any(c['html_url'] == write['url'] and c['body'] == write['body'] for c in comments) + delivered.append(write['url']) + summary.append(dict(stage=stage, kind=kind, run=run, writes=expected_writes, + upload_status=audit['uploadStatus'], audit_errors=audit['errors'])) + assert len(delivered) == 4 + cleanup = read(BASE / 'cleanup/audit-840-cleanup/cleanup-receipts.json') + assert read(BASE / 'cleanup/run.json')['conclusion'] == 'success' + assert len(cleanup) == 46 + planned = {kind + '/' + row['uuid'] for kind, rows in plan.items() for row in rows} + assert {r['resource'] for r in cleanup} == planned + assert all(r['delete_status'] in (200, 202, 204) and r['verification_status'] == 404 for r in cleanup) + independent = read(BASE / 'independent-cleanup.json') + assert {r['url'].split('/minigraf/')[1] for r in independent} == planned + assert all(r['status'] == 404 for r in independent) + for issue in (856, 857): + assert read(BASE / f'issue-{issue}-closed.json')['state'] == 'closed' + branch_cleanup = read(BASE / 'branch-cleanup.json') + assert branch_cleanup['ref'] == 'refs/heads/audit/840-live-checks' + assert branch_cleanup['readback_exit_code'] != 0 and 'HTTP 404' in branch_cleanup['readback_error'] + models = read(BASE / 'production-thresholds-after-cleanup.json') + assert len(models) == 5 + for model in models: + assert model['branch']['name'] == 'v3' + assert (model['model']['test'], model['model']['upper_boundary'], model['model']['max_sample_size']) == ('t_test', .999, 30) + result = dict(scenarios=summary, deliveries=delivered, synthetic_reports=len(reports), + cleanup_resources=len(cleanup), test_issues_closed=[856, 857], + limitation='Previous workflow lookup redirects main to audit/840-live-checks; GitHub delivery, not human mailbox receipt.') + (ROOT / 'live-summary.json').write_text(json.dumps(result, indent=2) + '\n') + print(f'Verified {len(summary)} live job scenarios, {len(delivered)} independently read-back deliveries, and {len(reports)} synthetic API reports') + + +if __name__ == '__main__': + main()