Skip to main content

sc_crawler.workload_profiles

Workload profile definitions for compound benchmark scoring.

Each workload profile is a weighted combination of benchmark scores that represents a specific real-world usage pattern. Scores are aggregated as a weighted average (geometric mean) of benchmark scores compared to their medians. A score of 1.0 represents a synthetic baseline server with the median performance of each component benchmark.

Weights within each workload sum to 1.0.

Classes:

  • BenchmarkEntry – A single benchmark component contributing to a workload profile score.
  • CompoundSource –
  • Workload – A named workload profile composed of weighted benchmark entries.

Attributes:

class BenchmarkEntry​

Bases: BaseModel

A single benchmark component contributing to a workload profile score.

Functions:

Attributes:

attr 
BenchmarkEntry.benchmark_id
​

benchmark_id: str

The benchmark ID of a BenchmarkScore.

attr 
BenchmarkEntry.weight
​

weight: float

Relative weight of this component. Weights within a workload sum to 1.0.

attr 
BenchmarkEntry.label
​

label: str

Human-readable description of what this component measures.

attr 
BenchmarkEntry.config_filter
​

config_filter: dict[str, Any] | None = None

Optional filter applied to the benchmark's config JSON column.

attr 
BenchmarkEntry.on_missing
​

on_missing: BenchmarkComponentMissingPolicy = BenchmarkComponentMissingPolicy.IGNORE

How to handle a missing or invalid measurement for this component.

attr 
BenchmarkEntry.penalty
​

penalty: float | None = None

Substituted normalized ratio when on_missing is PENALIZE.

meth 
BenchmarkEntry.effective_penalty
​

effective_penalty()

Return the penalty floor used when on_missing is PENALIZE.

class CompoundSource​

Bases: Json

Attributes:

attr 
CompoundSource.aggregation
​

aggregation: BenchmarkComponentAggregationMethod

How component benchmark scores are combined into one composite score.

attr 
CompoundSource.normalization
​

normalization: BenchmarkComponentNormalizationMethod

How each raw benchmark value is scaled to be comparable across benchmarks.

attr 
CompoundSource.components
​

components: list[BenchmarkEntry]

The components of the workload profile.

attr 
CompoundSource.impact_formula
​

impact_formula: str | None = None

Human-friendly explanation of how to read per-component impact on scores.

class Workload​

Bases: BaseModel

A named workload profile composed of weighted benchmark entries.

Attributes:

attr 
Workload.name
​

name: str

Short human-readable name, e.g. 'Web server'.

attr 
Workload.version
​

version: str

Workload profile version.

attr 
Workload.rationale
​

rationale: str

Explanation of which benchmarks were chosen and why.

attr 
Workload.benchmarks
​

benchmarks: list[BenchmarkEntry]

Ordered list of benchmark components with weights.

attr WORKLOADS​

WORKLOADS: dict[str, Workload] = {'web': Workload(name='Web Server', version='2.0', rationale='Primary workloads drivers are single-process static HTTP serving speed and throughput, text processing, TLS termination, and asset compression.', benchmarks=[BenchmarkEntry(benchmark_id='static_web:rps-extrapolated', weight=0.3, label='Static web RPS (1 KiB, 8 conn/vCPU)', config_filter={'size': '1k', 'connections_per_vcpus': 8.0}), BenchmarkEntry(benchmark_id='static_web:rps-extrapolated', weight=0.2, label='Static web RPS (64 KiB, 8 conn/vCPU)', config_filter={'size': '64k', 'connections_per_vcpus': 8.0}), BenchmarkEntry(benchmark_id='static_web:throughput-extrapolated', weight=0.2, label='Static web throughput (256 KiB, 8 conn/vCPU)', config_filter={'size': '256k', 'connections_per_vcpus': 8.0}), BenchmarkEntry(benchmark_id='openssl', weight=0.2, label='OpenSSL AES-256-CBC (16 kB blocks)', config_filter={'algo': 'AES-256-CBC', 'block_size': 16384}), BenchmarkEntry(benchmark_id='compression_text:compress', weight=0.05, label='Gzip compression (multi-core, level 5)', config_filter={'algo': 'gzip', 'compression_level': 5, 'cores': 'multi'}), BenchmarkEntry(benchmark_id='passmark:cpu_string_sorting_test', weight=0.05, label='PassMark string sorting')]), 'compute': Workload(name='Compute Heavy Applications', version='2.0', rationale='Number-crunching workload augmenting raw CPU performance stressing, general CPU performance benchmarks, memory bandwidth, and pure math computation speed like floating point, integer, SIMD (AVX/SSE/FMA) operations.', benchmarks=[BenchmarkEntry(benchmark_id='stress_ng:bestn', weight=0.15, label='stress-ng div16 best-N cores'), BenchmarkEntry(benchmark_id='stress_ng:best1', weight=0.1, label='stress-ng div16 single core'), BenchmarkEntry(benchmark_id='passmark:cpu_mark', weight=0.2, label='PassMark CPU Mark (composite)'), BenchmarkEntry(benchmark_id='bw_mem', weight=0.1, label='Memory bandwidth (read, 64 MB)', config_filter={'operation': 'rd', 'size': 64.0}), BenchmarkEntry(benchmark_id='passmark:cpu_floating_point_maths_test', weight=0.15, label='PassMark floating point'), BenchmarkEntry(benchmark_id='passmark:cpu_extended_instructions_test', weight=0.15, label='PassMark AVX/SSE/FMA (SIMD)'), BenchmarkEntry(benchmark_id='passmark:cpu_integer_maths_test', weight=0.1, label='PassMark integer math'), BenchmarkEntry(benchmark_id='passmark:cpu_physics_test', weight=0.05, label='PassMark physics simulation')]), 'cache': Workload(name='Cache Intensive', version='2.0', rationale='In-memory key-value store workload, mixing direct Redis performance metrics with memory speed and latency benchmarks, and single-core CPU performance profiles.', benchmarks=[BenchmarkEntry(benchmark_id='redis:rps-extrapolated', weight=0.5, label='Redis RPS (pipeline=1, SET)', config_filter={'operation': 'SET', 'pipeline': 1.0}), BenchmarkEntry(benchmark_id='redis:rps-extrapolated', weight=0.2, label='Redis RPS (pipeline=16, SET)', config_filter={'operation': 'SET', 'pipeline': 16.0}), BenchmarkEntry(benchmark_id='passmark:memory_mark', weight=0.1, label='PassMark Memory Mark (composite)'), BenchmarkEntry(benchmark_id='bw_mem', weight=0.1, label='Memory bandwidth (read, 16 MB ~ L3)', config_filter={'operation': 'rd', 'size': 16.0}), BenchmarkEntry(benchmark_id='passmark:cpu_single_threaded_test', weight=0.1, label='PassMark single-thread CPU')]), 'data_analysis': Workload(name='Data Analysis', version='2.0', rationale='Data analysis and ETL workloads are memory-bandwidth-bound and CPU-throughput-driven. The profile combines general CPU performance and memory bandwidth/latency as the primary drivers, supplemented by single-core compression speed as a proxy for serialisation-heavy ETL tasks.', benchmarks=[BenchmarkEntry(benchmark_id='passmark:cpu_mark', weight=0.7, label='PassMark CPU Mark (composite)'), BenchmarkEntry(benchmark_id='compression_text:compress', weight=0.1, label='Gzip compression (single-core, level 5)', config_filter={'algo': 'gzip', 'compression_level': 5, 'cores': 'single'}), BenchmarkEntry(benchmark_id='bw_mem', weight=0.1, label='Memory bandwidth (read, 64 MB)', config_filter={'operation': 'rd', 'size': 64.0}), BenchmarkEntry(benchmark_id='passmark:memory_mark', weight=0.1, label='PassMark Memory Mark (composite)')]), 'llm': Workload(name='LLM Inference', version='2.0', rationale='VRAM and memory-bandwidth-bound LLM inference workload, using direct LLM speed benchmarks at three model sizes, and supplementing with raw memory bandwidth and SIMD performance benchmarks.', benchmarks=[BenchmarkEntry(benchmark_id='llm_speed:text_generation', weight=0.15, label='LLM text generation (SmolLM-135M, 128 tok)', config_filter={'model': 'SmolLM-135M.Q4_K_M.gguf', 'tokens': 128}, on_missing=BenchmarkComponentMissingPolicy.REQUIRE), BenchmarkEntry(benchmark_id='llm_speed:prompt_processing', weight=0.15, label='LLM prompt processing (SmolLM-135M, 512 tok)', config_filter={'model': 'SmolLM-135M.Q4_K_M.gguf', 'tokens': 512}, on_missing=BenchmarkComponentMissingPolicy.REQUIRE), BenchmarkEntry(benchmark_id='llm_speed:text_generation', weight=0.15, label='LLM text generation (Llama 7B, 128 tok)', config_filter={'model': 'llama-7b.Q4_K_M.gguf', 'tokens': 128}, on_missing=BenchmarkComponentMissingPolicy.PENALIZE, penalty=0.0001), BenchmarkEntry(benchmark_id='llm_speed:prompt_processing', weight=0.15, label='LLM prompt processing (Llama 7B, 512 tok)', config_filter={'model': 'llama-7b.Q4_K_M.gguf', 'tokens': 512}, on_missing=BenchmarkComponentMissingPolicy.PENALIZE, penalty=0.0001), BenchmarkEntry(benchmark_id='llm_speed:text_generation', weight=0.15, label='LLM text generation (Llama-3.3 70B, 128 tok)', config_filter={'model': 'Llama-3.3-70B-Instruct-Q4_K_M.gguf', 'tokens': 128}, on_missing=BenchmarkComponentMissingPolicy.PENALIZE, penalty=0.01), BenchmarkEntry(benchmark_id='llm_speed:prompt_processing', weight=0.15, label='LLM prompt processing (Llama-3.3 70B, 512 tok)', config_filter={'model': 'Llama-3.3-70B-Instruct-Q4_K_M.gguf', 'tokens': 512}, on_missing=BenchmarkComponentMissingPolicy.PENALIZE, penalty=0.01), BenchmarkEntry(benchmark_id='bw_mem', weight=0.05, label='Memory bandwidth (read, 256 MB)', config_filter={'operation': 'rd', 'size': 256.0}), BenchmarkEntry(benchmark_id='passmark:cpu_extended_instructions_test', weight=0.025, label='PassMark AVX/SSE/FMA (SIMD)'), BenchmarkEntry(benchmark_id='passmark:cpu_floating_point_maths_test', weight=0.025, label='PassMark floating point')]), 'cicd': Workload(name='CI/CD Build', version='2.0', rationale='Build performance is mainly driven by multi-core compilation throughput, but also bundles single-core compilation speed and general CPU performance, multi-core compression and text/scripting processing.', benchmarks=[BenchmarkEntry(benchmark_id='geekbench:clang', weight=0.5, label='Geekbench Clang compilation (multi-core)', config_filter={'cores': 'multi'}), BenchmarkEntry(benchmark_id='geekbench:clang', weight=0.1, label='Geekbench Clang compilation (single-core)', config_filter={'cores': 'single'}), BenchmarkEntry(benchmark_id='stress_ng:bestn', weight=0.2, label='stress-ng div16 best-N cores'), BenchmarkEntry(benchmark_id='passmark:cpu_integer_maths_test', weight=0.05, label='PassMark integer math'), BenchmarkEntry(benchmark_id='passmark:cpu_compression_test', weight=0.05, label='PassMark compression'), BenchmarkEntry(benchmark_id='compression_text:compress', weight=0.05, label='Brotli compression (multi-core, level 0)', config_filter={'algo': 'brotli', 'compression_level': 0, 'cores': 'single'}), BenchmarkEntry(benchmark_id='passmark:cpu_string_sorting_test', weight=0.05, label='PassMark string sorting')])}

Workload profile definitions keyed by workload ID.