Llamabench6.2 Statistics v2

GPUs	GPUsx	Model	TG	PP	TTFT	Watts	LDU	System Data	Created
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Bielik-11B-v3.0-Instruct-Q8_0.gguf	TG = 43.32	PP = 295.76	TTFT = 391.09	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 17:41:35
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2954.48 ms Generated Tokens: 128					VRAM Model: 11184.92 MiB (51/51 layers) [ROCm0: 7736.1 \| ROCm1: 3448.8] KV Cache: 1800.00 MiB [q4_0] [ROCm0: 1260.0 \| ROCm1: 540.0] Compute Buf: 968.14 MiB [ROCm0: 428.0 \| ROCm1: 268.1 \| ROCm_Host: 272.1] RAM BPE/Meta: 133.34 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon VII (16116 MiB) [[max Watt:27W]]	1	Bielik-11B-v3.0-Instruct-Q8_0.gguf	TG = 37.79	PP = 287.32	TTFT = 472.98	308.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-23 13:43:08
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 3386.88 ms Generated Tokens: 128					VRAM Model: 11184.92 MiB (51/51 layers) [ROCm0: 11184.9] KV Cache: 3346.88 MiB [q8_0] [ROCm0: 3346.9] Compute Buf: 225.68 MiB [ROCm0: 146.6 \| ROCm_Host: 79.1] RAM BPE/Meta: 133.34 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32428 MiB free)	1	Bielik-11B-v3.0-Instruct-Q8_0.gguf	TG = 44.39	PP = 300.90	TTFT = 383.81	160 W	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-29 20:55:05
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2770.83 ms Generated Tokens: 123					VRAM Model: 11184.92 MiB (51/51 layers) [ROCm0: 11184.9] KV Cache: 337.50 MiB [q4_0] [ROCm0: 337.5] Compute Buf: 140.02 MiB [ROCm0: 112.0 \| ROCm_Host: 28.0] RAM BPE/Meta: 133.34 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11127 MiB free) [RPC0] 127.0.0.1:50052 (32732 MiB free)	2	Bielik-11B-v3.0-Instruct-Q8_0.gguf	TG = 37.43	PP = 359.90	TTFT = 572.42	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:41:46
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 3420.13 ms Generated Tokens: 128					VRAM Model: 11184.92 MiB (51/51 layers) [CUDA0: 2564.7 \| RPC0: 8620.2] KV Cache: 1600.00 MiB [f16] [CUDA0: 352.0 \| RPC0: 1248.0] Compute Buf: 264.03 MiB [RPC0: 116.0 \| CUDA0: 116.0 \| CPU: 32.0] RAM BPE/Meta: 133.34 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[Vulkan0] AMD Radeon VII (RADV VEGA20) (15572 MiB free) [Vulkan1] AMD Radeon Graphics (RADV VEGA20) (32751 MiB free)	2	Bielik-11B-v3.0-Instruct-Q8_0.gguf	TG = 27.94	PP = 56.86	TTFT = 235.69	N/A	VULKAN	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-01 15:02:36
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 4581.38 ms Generated Tokens: 128					VRAM Model: 11184.92 MiB (51/51 layers) [Vulkan0: 3757.5 \| Vulkan1: 7427.4] KV Cache: 1600.00 MiB [f16] [Vulkan0: 544.0 \| Vulkan1: 1056.0] Compute Buf: 372.17 MiB [Vulkan0: 168.0 \| Vulkan1: 156.1 \| Vulkan_Host: 48.1] RAM BPE/Meta: 133.34 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11066 MiB free)	1	Bielik-11B-v3.0-Instruct.Q4_K_M.gguf	TG = 63.40	PP = 2716.68	TTFT = 55.13	200 W	CUDA	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:47:36
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2018.92 ms Generated Tokens: 128					VRAM Model: 6341.25 MiB (51/51 layers) [CUDA0: 6341.2] KV Cache: 800.00 MiB [f16] [CUDA0: 800.0] Compute Buf: 136.02 MiB [CUDA0: 112.0 \| CUDA_Host: 24.0] RAM BPE/Meta: 70.59 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11294 MiB free)	1	Bielik-11B-v3.0-Instruct.Q4_K_M.gguf	TG = 30.04	PP = 1373.55	TTFT = 94.35	100 W	CUDA	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-29 19:57:50
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 4260.97 ms Generated Tokens: 128					VRAM Model: 6341.25 MiB (51/51 layers) [CUDA0: 6341.2] KV Cache: 950.00 MiB [f16] [CUDA0: 950.0] Compute Buf: 139.77 MiB [CUDA0: 114.3 \| CUDA_Host: 25.5] RAM BPE/Meta: 70.59 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Bielik-11B-v3.0-Instruct.Q4_K_M.gguf	TG = 48.51	PP = 501.29	TTFT = 251.94	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 17:36:25
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2593.56 ms Generated Tokens: 125.8					VRAM Model: 6341.24 MiB (51/51 layers) [ROCm0: 4328.1 \| ROCm1: 2013.1] KV Cache: 225.00 MiB [q4_0] [ROCm0: 157.5 \| ROCm1: 67.5] Compute Buf: 380.14 MiB [ROCm0: 176.0 \| ROCm1: 156.1 \| ROCm_Host: 48.0] RAM BPE/Meta: 70.59 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32428 MiB free)	1	Bielik-11B-v3.0-Instruct.Q4_K_M.gguf	TG = 50.14	PP = 509.49	TTFT = 247.66	160 W	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-29 20:53:16
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2552.62 ms Generated Tokens: 128					VRAM Model: 6341.25 MiB (51/51 layers) [ROCm0: 6341.2] KV Cache: 337.50 MiB [q4_0] [ROCm0: 337.5] Compute Buf: 140.02 MiB [ROCm0: 112.0 \| ROCm_Host: 28.0] RAM BPE/Meta: 70.59 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11011 MiB free) [RPC0] 127.0.0.1:50052 (32732 MiB free)	2	Bielik-11B-v3.0-Instruct.Q4_K_M.gguf	TG = 47.69	PP = 597.77	TTFT = 359.39	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:17:46
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2684.09 ms Generated Tokens: 128					VRAM Model: 6341.25 MiB (51/51 layers) [CUDA0: 1514.1 \| RPC0: 4827.2] KV Cache: 1600.00 MiB [f16] [CUDA0: 352.0 \| RPC0: 1248.0] Compute Buf: 264.03 MiB [RPC0: 116.0 \| CUDA0: 116.0 \| CPU: 32.0] RAM BPE/Meta: 70.59 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[Vulkan0] AMD Radeon VII (RADV VEGA20) (15698 MiB free) [Vulkan1] AMD Radeon Graphics (RADV VEGA20) (32751 MiB free)	2	Bielik-11B-v3.0-Instruct.Q4_K_M.gguf	TG = 27.79	PP = 188.96	TTFT = 729.17	N/A	VULKAN	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-03 18:11:29
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 4605.68 ms Generated Tokens: 128					VRAM Model: 6341.25 MiB (51/51 layers) [Vulkan0: 2128.8 \| Vulkan1: 4212.5] KV Cache: 13600.00 MiB [q8_0] [Vulkan0: 4624.0 \| Vulkan1: 8976.0] Compute Buf: 2981.10 MiB [Vulkan0: 1288.3 \| Vulkan1: 652.4 \| Vulkan_Host: 1040.4] RAM BPE/Meta: 70.59 MiB RAM Layers: 0.00 MiB (0/51 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11062 MiB free)	1	Bielik-4.5B-v3.0-Instruct.Q8_0.gguf	TG = 80.13	PP = 5415.43	TTFT = 50.63	N/A	CUDA	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:53:38
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1597.37 ms Generated Tokens: 128					VRAM Model: 4759.56 MiB (61/61 layers) [CUDA0: 4759.6] KV Cache: 240.00 MiB [f16] [CUDA0: 240.0] Compute Buf: 96.52 MiB [CUDA0: 80.5 \| CUDA_Host: 16.0] RAM BPE/Meta: 66.41 MiB RAM Layers: 0.00 MiB (0/61 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Bielik-4.5B-v3.0-Instruct.Q8_0.gguf	TG = 41.35	PP = 780.28	TTFT = 292.35	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 17:54:21
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1114.35 ms Generated Tokens: 76.8					VRAM Model: 4759.56 MiB (61/61 layers) [ROCm0: 3207.0 \| ROCm1: 1552.6] KV Cache: 135.03 MiB [q4_0] [ROCm0: 92.3 \| ROCm1: 42.8] Compute Buf: 357.14 MiB [ROCm0: 160.5 \| ROCm1: 124.5 \| ROCm_Host: 72.0] RAM BPE/Meta: 66.41 MiB RAM Layers: 0.00 MiB (0/61 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32428 MiB free)	1	Bielik-4.5B-v3.0-Instruct.Q8_0.gguf	TG = 70.92	PP = 646.84	TTFT = 340.81	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-29 20:55:41
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1804.83 ms Generated Tokens: 128					VRAM Model: 4759.56 MiB (61/61 layers) [ROCm0: 4759.6] KV Cache: 101.28 MiB [q4_0] [ROCm0: 101.3] Compute Buf: 106.52 MiB [ROCm0: 86.5 \| ROCm_Host: 20.0] RAM BPE/Meta: 66.41 MiB RAM Layers: 0.00 MiB (0/61 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (10979 MiB free) [RPC0] 127.0.0.1:50052 (32732 MiB free)	2	Bielik-4.5B-v3.0-Instruct.Q8_0.gguf	TG = 56.89	PP = 796.45	TTFT = 502.65	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:14:37
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2250.28 ms Generated Tokens: 128					VRAM Model: 4759.56 MiB (61/61 layers) [CUDA0: 1161.5 \| RPC0: 3598.1] KV Cache: 480.00 MiB [f16] [CUDA0: 112.0 \| RPC0: 368.0] Compute Buf: 201.03 MiB [RPC0: 92.5 \| CUDA0: 84.5 \| CPU: 24.0] RAM BPE/Meta: 66.41 MiB RAM Layers: 0.00 MiB (0/61 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	Bielik-7B-v3.0-Instruct-GGUF.Q8_0.gguf	TG = 59.52	PP = 185.65	TTFT = 71.09	N/A	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-08 09:53:07
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 300 Avg Time: 5040.62 ms Generated Tokens: 300					VRAM Model: 7444.61 MiB (41/41 layers) [ROCm0: 7444.6] KV Cache: 10880.00 MiB [q8_0] [ROCm0: 10880.0] Compute Buf: 680.18 MiB [ROCm0: 408.1 \| ROCm_Host: 272.1] RAM BPE/Meta: 133.34 MiB RAM Layers: 0.00 MiB (0/41 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:225W]]	1	Bielik-7B-v3.0-Instruct-GGUF.Q8_0.gguf	TG = 58.86	PP = 167.73	TTFT = 71.55	299.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-08 19:26:06
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 1 Max Time: 30 Context Limit: ∞ Avg Time: 7968.43 ms Generated Tokens: 470					VRAM Model: 7444.61 MiB (41/41 layers) [ROCm0: 7444.6] KV Cache: 21760.00 MiB [q8_0] [ROCm0: 21760.0] Compute Buf: 1320.18 MiB [ROCm0: 792.1 \| ROCm_Host: 528.1] RAM BPE/Meta: 133.34 MiB RAM Layers: 0.00 MiB (0/41 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	FINAL-Bench_Darwin-35B-A3B-Opus-Q8_0.gguf	TG = 58.03	PP = 645.62	TTFT = 723.84	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-02 22:10:38
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2205.82 ms Generated Tokens: 128					VRAM Model: 34678.80 MiB (41/41 layers) [ROCm0: 23914.4 \| ROCm1: 10764.4] KV Cache: 1360.00 MiB [q8_0] [ROCm0: 952.0 \| ROCm1: 408.0] Compute Buf: 3684.04 MiB [ROCm0: 1236.1 \| ROCm1: 1021.1 \| ROCm_Host: 1032.1 \| ROCm0: 358.8 \| CPU: 36.0] SSM/RS State: 62.81 MiB [ROCm0: 44.0 \| ROCm1: 18.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11111 MiB free) [RPC0] 127.0.0.1:50052 (32732 MiB free)	2	GLM-4.7-Flash-MXFP4_MOE.gguf	TG = 46.93	PP = 810.10	TTFT = 304.27	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:42:52
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2727.57 ms Generated Tokens: 128					VRAM Model: 15852.01 MiB (48/48 layers) [CUDA0: 4015.3 \| RPC0: 11836.7] KV Cache: 423.00 MiB [f16] [CUDA0: 99.0 \| RPC0: 324.0] Compute Buf: 438.90 MiB [RPC0: 88.0 \| CUDA0: 326.9 \| CPU: 24.0] RAM BPE/Meta: 321.41 MiB RAM Layers: 0.00 MiB (0/48 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	GLM-4.7-Flash-Q8_0.gguf	TG = 70.95	PP = 616.02	TTFT = 246.40	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 17:56:50
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 102.83 ms Generated Tokens: 7.2					VRAM Model: 30037.26 MiB (48/48 layers) [ROCm0: 20698.0 \| ROCm1: 9339.2] KV Cache: 2944.49 MiB [q4_0] [ROCm0: 2067.4 \| ROCm1: 877.1] Compute Buf: 4591.59 MiB [ROCm0: 1881.0 \| ROCm1: 1118.5 \| ROCm_Host: 1592.0] RAM BPE/Meta: 321.41 MiB RAM Layers: 0.00 MiB (0/48 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11112 MiB free) [RPC0] 127.0.0.1:50052 (32732 MiB free)	2	GLM-4.7-Flash-Q8_0.gguf	TG = 45.91	PP = 692.57	TTFT = 347.59	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:44:42
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2788.18 ms Generated Tokens: 128					VRAM Model: 30037.25 MiB (48/48 layers) [CUDA0: 7406.9 \| RPC0: 22630.4] KV Cache: 423.00 MiB [f16] [CUDA0: 99.0 \| RPC0: 324.0] Compute Buf: 438.90 MiB [RPC0: 88.0 \| CUDA0: 326.9 \| CPU: 24.0] RAM BPE/Meta: 321.41 MiB RAM Layers: 0.00 MiB (0/48 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	GLM-4.7-Flash-UD-Q4_K_XL.gguf	TG = 59.01	PP = 912.12	TTFT = 195.31	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 17:58:21
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1756.38 ms Generated Tokens: 103.6					VRAM Model: 16529.38 MiB (48/48 layers) [ROCm0: 11293.1 \| ROCm1: 5236.3] KV Cache: 2944.49 MiB [q4_0] [ROCm0: 2067.4 \| ROCm1: 877.1] Compute Buf: 4591.59 MiB [ROCm0: 1881.0 \| ROCm1: 1118.5 \| ROCm_Host: 1592.0] RAM BPE/Meta: 170.16 MiB RAM Layers: 0.00 MiB (0/48 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	GLM-4.7-Flash-UD-Q4_K_XL.gguf	TG = 64.92	PP = 942.26	TTFT = 191.84	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:17:03
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1971.56 ms Generated Tokens: 128					VRAM Model: 16529.37 MiB (48/48 layers) [ROCm0: 16529.4] KV Cache: 211.50 MiB [f16] [ROCm0: 211.5] Compute Buf: 326.51 MiB [ROCm0: 310.5 \| ROCm_Host: 16.0] RAM BPE/Meta: 170.16 MiB RAM Layers: 0.00 MiB (0/48 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11104 MiB free) [RPC0] 127.0.0.1:50052 (32732 MiB free)	2	GLM-4.7-Flash-UD-Q4_K_XL.gguf	TG = 47.62	PP = 173.79	TTFT = 116.21	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:47:56
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2446.58 ms Generated Tokens: 116.4					VRAM Model: 16529.38 MiB (48/48 layers) [CUDA0: 4196.3 \| RPC0: 12333.0] KV Cache: 423.00 MiB [f16] [CUDA0: 99.0 \| RPC0: 324.0] Compute Buf: 438.90 MiB [RPC0: 88.0 \| CUDA0: 326.9 \| CPU: 24.0] RAM BPE/Meta: 170.16 MiB RAM Layers: 0.00 MiB (0/48 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 3050 Laptop GPU (3299 MiB free) [[max Watt:69W]]	1	Llama-3.2-3B-Instruct-Q4_K_M.gguf	TG = 63.23	PP = 372.00	TTFT = 26.37	122.4 W	CUDA	OS: Windows-10-10.0.19045-SP0 CUDA: 591.59 MMAP: false	2026-04-08 17:45:33
		Server Environment: Host Build: b8468-3306dbaef User Host Data: f17w10 / Doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2024.61 ms Generated Tokens: 128					VRAM Model: 2226.58 MiB (29/29 warstw) [CPU: 308.2 \| CUDA0: 1918.3] KV Cache: 126.00 MiB [q4_0] [CUDA0: 126.0] Bufor Oblicz: 276.51 MiB [CUDA0: 256.5 \| CUDA_Host: 20.0] RAM BPE/Meta: 0 MiB RAM Warstwy: 0.00 MiB (0/29 warstw) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11059 MiB free)	1	NVIDIA-Nemotron3-Nano-4B-Q4_K_M.gguf	TG = 135.53	PP = 4508.39	TTFT = 104.29	N/A	CUDA	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:53:59
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 944.44 ms Generated Tokens: 128					VRAM Model: 2428.65 MiB (43/43 layers) [CUDA0: 2428.7] KV Cache: 64.00 MiB [f16] [CUDA0: 64.0] Compute Buf: 296.52 MiB [CUDA0: 276.3 \| CUDA_Host: 20.3] SSM/RS State: 81.09 MiB [CUDA0: 81.1] RAM BPE/Meta: 269.50 MiB RAM Layers: 0.00 MiB (0/43 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	NVIDIA-Nemotron3-Nano-4B-Q4_K_M.gguf	TG = 91.72	PP = 868.68	TTFT = 541.07	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:07:48
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1395.63 ms Generated Tokens: 128					VRAM Model: 2428.65 MiB (43/43 layers) [ROCm0: 1389.8 \| ROCm1: 1038.9] KV Cache: 8704.00 MiB [q8_0] [ROCm0: 6528.0 \| ROCm1: 2176.0] Compute Buf: 22149.75 MiB [ROCm0: 9413.9 \| ROCm1: 4531.5 \| ROCm_Host: 8204.3] SSM/RS State: 81.09 MiB [ROCm0: 54.1 \| ROCm1: 27.0] RAM BPE/Meta: 269.50 MiB RAM Layers: 0.00 MiB (0/43 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Nemotron-3-Nano-30B-A3B-IQ4_NL.gguf	TG = 93.63	PP = 765.51	TTFT = 613.47	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:02:35
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1367.12 ms Generated Tokens: 128					VRAM Model: 17098.29 MiB (53/53 layers) [ROCm0: 10952.8 \| ROCm1: 6145.5] KV Cache: 3264.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 544.0] Compute Buf: 21894.94 MiB [ROCm0: 9314.1 \| ROCm1: 4378.3 \| ROCm_Host: 8202.5] SSM/RS State: 47.61 MiB [ROCm0: 33.1 \| ROCm1: 14.5] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	Nemotron-3-Nano-30B-A3B-IQ4_NL.gguf	TG = 104.73	PP = 778.66	TTFT = 603.41	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:17:44
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1222.13 ms Generated Tokens: 128					VRAM Model: 17098.29 MiB (53/53 layers) [ROCm0: 17098.3] KV Cache: 24.00 MiB [f16] [ROCm0: 24.0] Compute Buf: 285.08 MiB [ROCm0: 266.6 \| ROCm_Host: 18.5] SSM/RS State: 47.62 MiB [ROCm0: 47.6] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11100 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	Nemotron-3-Nano-30B-A3B-IQ4_NL.gguf	TG = 80.77	PP = 915.43	TTFT = 513.22	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:49:24
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1584.74 ms Generated Tokens: 128					VRAM Model: 17098.29 MiB (53/53 layers) [CUDA0: 4696.3 \| RPC0: 12402.0] KV Cache: 48.00 MiB [f16] [CUDA0: 8.0 \| RPC0: 40.0] Compute Buf: 389.33 MiB [RPC0: 96.0 \| CUDA0: 266.8 \| CPU: 26.5] SSM/RS State: 190.47 MiB [CUDA0: 41.4 \| RPC0: 149.1] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Nemotron-3-Nano-30B-A3B-Q4_K_M.gguf	TG = 87.93	PP = 734.42	TTFT = 638.89	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:03:56
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1455.69 ms Generated Tokens: 128					VRAM Model: 23197.47 MiB (53/53 layers) [ROCm0: 14667.2 \| ROCm1: 8530.2] KV Cache: 3264.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 544.0] Compute Buf: 21894.94 MiB [ROCm0: 9314.1 \| ROCm1: 4378.3 \| ROCm_Host: 8202.5] SSM/RS State: 47.61 MiB [ROCm0: 33.1 \| ROCm1: 14.5] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	Nemotron-3-Nano-30B-A3B-Q4_K_M.gguf	TG = 97.48	PP = 745.46	TTFT = 630.17	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:18:37
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1313.08 ms Generated Tokens: 128					VRAM Model: 23197.47 MiB (53/53 layers) [ROCm0: 23197.5] KV Cache: 24.00 MiB [f16] [ROCm0: 24.0] Compute Buf: 285.08 MiB [ROCm0: 266.6 \| ROCm_Host: 18.5] SSM/RS State: 47.62 MiB [ROCm0: 47.6] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 3050 Laptop GPU (3299 MiB free) [RPC0] 192.168.0.222:50052 (27365 MiB free)	2	Nemotron-3-Nano-30B-A3B-Q4_K_M.gguf	TG = 20.72	PP = 135.97	TTFT = 3459.82	N/A	CUDA + RPC	OS: Windows-10-10.0.19045-SP0 CUDA: 591.59 MMAP: false	2026-03-28 13:56:23
		Server Environment: Host Build: b8468-3306dbaef User Host Data: f17w10 / Doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 6179.98 ms Generated Tokens: 128					VRAM Model: 23428.46 MiB (53/53 layers) [CUDA0: 2574.1 \| CUDA_Host: 231.0 \| RPC0: 20623.4] KV Cache: 6.75 MiB [q4_0] [RPC0: 6.8] Compute Buf: 364.52 MiB [RPC0: 79.5 \| CUDA0: 266.5 \| CPU: 18.5] SSM/RS State: 47.62 MiB [CUDA0: 4.1 \| RPC0: 43.5] RAM BPE/Meta: 0 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11096 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	Nemotron-3-Nano-30B-A3B-Q4_K_M.gguf	TG = 78.32	PP = 877.17	TTFT = 535.84	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:51:23
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1634.28 ms Generated Tokens: 128					VRAM Model: 23197.47 MiB (53/53 layers) [CUDA0: 6541.5 \| RPC0: 16655.9] KV Cache: 48.00 MiB [f16] [CUDA0: 8.0 \| RPC0: 40.0] Compute Buf: 389.33 MiB [RPC0: 96.0 \| CUDA0: 266.8 \| CPU: 26.5] SSM/RS State: 190.47 MiB [CUDA0: 41.4 \| RPC0: 149.1] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Nemotron-3-Nano-30B-A3B-UD-Q8_K_XL.gguf	TG = 76.36	PP = 656.73	TTFT = 714.76	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:04:58
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1676.22 ms Generated Tokens: 128					VRAM Model: 37894.69 MiB (53/53 layers) [ROCm0: 25184.8 \| ROCm1: 12709.9] KV Cache: 3264.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 544.0] Compute Buf: 6242.17 MiB [ROCm0: 3090.6 \| ROCm1: 1093.1 \| ROCm_Host: 2058.5] SSM/RS State: 47.61 MiB [ROCm0: 33.1 \| ROCm1: 14.5] RAM BPE/Meta: 672.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11126 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	Nemotron-3-Nano-30B-A3B-UD-Q8_K_XL.gguf	TG = 66.50	PP = 776.82	TTFT = 604.57	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 16:10:40
		Server Environment: No data Build: No data User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1924.86 ms Generated Tokens: 128					VRAM Model: 37894.70 MiB (53/53 layers) [CUDA0: 9982.0 \| RPC0: 27912.7] KV Cache: 24.00 MiB [f16] [CUDA0: 4.0 \| RPC0: 20.0] Compute Buf: 373.47 MiB [RPC0: 84.5 \| CUDA0: 270.5 \| CPU: 18.5] SSM/RS State: 190.47 MiB [CUDA0: 41.4 \| RPC0: 149.1] RAM BPE/Meta: 672.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11108 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	Nemotron-3-Nano-30B-A3B-UD-Q8_K_XL.gguf	TG = 66.32	PP = 769.75	TTFT = 610.64	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:53:15
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1930.13 ms Generated Tokens: 128					VRAM Model: 37894.70 MiB (53/53 layers) [CUDA0: 9982.0 \| RPC0: 27912.7] KV Cache: 48.00 MiB [f16] [CUDA0: 8.0 \| RPC0: 40.0] Compute Buf: 389.33 MiB [RPC0: 96.0 \| CUDA0: 266.8 \| CPU: 26.5] SSM/RS State: 190.47 MiB [CUDA0: 41.4 \| RPC0: 149.1] RAM BPE/Meta: 672.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Qwen3-VL-30B-A3B-Instruct-Q5_K_M.gguf	TG = 63.70	PP = 716.64	TTFT = 196.32	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:15:19
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2009.63 ms Generated Tokens: 128					VRAM Model: 20509.43 MiB (49/49 layers) [ROCm0: 13894.4 \| ROCm1: 6615.0] KV Cache: 13056.00 MiB [q8_0] [ROCm0: 8976.0 \| ROCm1: 4080.0] Compute Buf: 5832.96 MiB [ROCm0: 2424.1 \| ROCm1: 1340.8 \| ROCm_Host: 2068.1] RAM BPE/Meta: 204.02 MiB RAM Layers: 0.00 MiB (0/49 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	Qwen3-VL-30B-A3B-Instruct-Q5_K_M.gguf	TG = 74.18	PP = 735.59	TTFT = 191.47	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:28:45
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1725.52 ms Generated Tokens: 128					VRAM Model: 20509.43 MiB (49/49 layers) [ROCm0: 20509.4] KV Cache: 384.00 MiB [f16] [ROCm0: 384.0] Compute Buf: 328.77 MiB [ROCm0: 300.8 \| ROCm_Host: 28.0] RAM BPE/Meta: 204.02 MiB RAM Layers: 0.00 MiB (0/49 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 4070 (11049 MiB free) [RPC0] 127.0.0.1:50052 (32732 MiB free)	2	Qwen3-VL-30B-A3B-Instruct-Q5_K_M.gguf	TG = 59.39	PP = 820.84	TTFT = 279.30	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:12:22
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2155.43 ms Generated Tokens: 128					VRAM Model: 20509.43 MiB (49/49 layers) [CUDA0: 4951.8 \| RPC0: 15557.6] KV Cache: 768.00 MiB [f16] [CUDA0: 176.0 \| RPC0: 592.0] Compute Buf: 448.79 MiB [RPC0: 112.0 \| CUDA0: 300.8 \| CPU: 36.0] RAM BPE/Meta: 204.02 MiB RAM Layers: 0.00 MiB (0/49 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Qwen3.5-35B-A3B-Q4_K_M.gguf	TG = 54.89	PP = 714.86	TTFT = 654.72	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:13:30
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2331.91 ms Generated Tokens: 128					VRAM Model: 20470.32 MiB (41/41 layers) [ROCm0: 14050.7 \| ROCm1: 6419.6] KV Cache: 2720.00 MiB [q8_0] [ROCm0: 1904.0 \| ROCm1: 816.0] Compute Buf: 5977.21 MiB [ROCm0: 2388.1 \| ROCm1: 1533.1 \| ROCm_Host: 2056.1] SSM/RS State: 62.81 MiB [ROCm0: 44.0 \| ROCm1: 18.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	Qwen3.5-35B-A3B-Q4_K_M.gguf	TG = 61.48	PP = 730.12	TTFT = 640.03	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:27:56
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2081.84 ms Generated Tokens: 128					VRAM Model: 20470.32 MiB (41/41 layers) [ROCm0: 20470.3] KV Cache: 80.00 MiB [f16] [ROCm0: 80.0] Compute Buf: 513.02 MiB [ROCm0: 497.0 \| ROCm_Host: 16.0] SSM/RS State: 62.81 MiB [ROCm0: 62.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11046 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	Qwen3.5-35B-A3B-Q4_K_M.gguf	TG = 50.66	PP = 833.31	TTFT = 560.42	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:09:09
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2526.74 ms Generated Tokens: 128					VRAM Model: 20470.32 MiB (41/41 layers) [CUDA0: 4909.2 \| RPC0: 15561.1] KV Cache: 160.00 MiB [f16] [CUDA0: 48.0 \| RPC0: 112.0] Compute Buf: 609.04 MiB [RPC0: 92.0 \| CUDA0: 493.0 \| CPU: 24.0] SSM/RS State: 251.25 MiB [CUDA0: 50.2 \| RPC0: 201.0] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Qwen3.5-35B-A3B-Q8_0.gguf	TG = 59.42	PP = 631.67	TTFT = 739.64	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:14:21
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2154.15 ms Generated Tokens: 128					VRAM Model: 34667.78 MiB (41/41 layers) [ROCm0: 23906.7 \| ROCm1: 10761.1] KV Cache: 2720.00 MiB [q8_0] [ROCm0: 1904.0 \| ROCm1: 816.0] Compute Buf: 5977.21 MiB [ROCm0: 2388.1 \| ROCm1: 1533.1 \| ROCm_Host: 2056.1] SSM/RS State: 62.81 MiB [ROCm0: 44.0 \| ROCm1: 18.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11030 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	Qwen3.5-35B-A3B-Q8_0.gguf	TG = 49.72	PP = 726.29	TTFT = 643.84	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:10:58
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2574.23 ms Generated Tokens: 128					VRAM Model: 34667.78 MiB (41/41 layers) [CUDA0: 8194.6 \| RPC0: 26473.1] KV Cache: 160.00 MiB [f16] [CUDA0: 48.0 \| RPC0: 112.0] Compute Buf: 609.04 MiB [RPC0: 92.0 \| CUDA0: 493.0 \| CPU: 24.0] SSM/RS State: 251.25 MiB [CUDA0: 50.2 \| RPC0: 201.0] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	Qwen3.5-35B-A3B-heretic-v2.i1-Q6_K.gguf	TG = 54.38	PP = 681.80	TTFT = 687.01	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:12:31
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2353.77 ms Generated Tokens: 128					VRAM Model: 26784.89 MiB (41/41 layers) [ROCm0: 18470.9 \| ROCm1: 8314.0] KV Cache: 2720.00 MiB [q8_0] [ROCm0: 1904.0 \| ROCm1: 816.0] Compute Buf: 5977.21 MiB [ROCm0: 2388.1 \| ROCm1: 1533.1 \| ROCm_Host: 2056.1] SSM/RS State: 62.81 MiB [ROCm0: 44.0 \| ROCm1: 18.8] RAM BPE/Meta: 397.85 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:211W]] [ROCm1] AMD Radeon VII (16340 MiB) [[max Watt:199W]]	2	Qwen3.5-35B-A3B-heretic-v2.i1-Q6_K.gguf	TG = 32.57	PP = 504.08	TTFT = 56662.46	438.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-14 12:49:59
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 4064.81 ms Generated Tokens: 128					VRAM Model: 26784.89 MiB (41/41 layers) [ROCm0: 18470.9 \| ROCm1: 8314.0] KV Cache: 5317.81 MiB [q8_0] [ROCm0: 3722.5 \| ROCm1: 1595.3] Compute Buf: 5867.65 MiB [ROCm0: 2345.4 \| ROCm1: 1511.1 \| ROCm_Host: 2011.1] SSM/RS State: 125.63 MiB [ROCm0: 87.9 \| ROCm1: 37.7] RAM BPE/Meta: 397.85 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:179W]] [ROCm1] AMD Radeon VII (16340 MiB) [[max Watt:145W]]	2	Qwen3.5-35B-A3B-heretic-v2.i1-Q6_K.gguf	TG = 51.94	PP = 636.24	TTFT = 778.00	342.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-14 17:46:40
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2464.43 ms Generated Tokens: 128					VRAM Model: 26784.89 MiB (41/41 layers) [ROCm0: 18470.9 \| ROCm1: 8314.0] KV Cache: 5317.81 MiB [q8_0] [ROCm0: 3722.5 \| ROCm1: 1595.3] Compute Buf: 5867.65 MiB [ROCm0: 2345.4 \| ROCm1: 1511.1 \| ROCm_Host: 2011.1] SSM/RS State: 125.63 MiB [ROCm0: 87.9 \| ROCm1: 37.7] RAM BPE/Meta: 397.85 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	Qwen3.5-35B-A3B-heretic-v2.i1-Q6_K.gguf	TG = 58.28	PP = 693.10	TTFT = 674.69	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:26:33
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2198.32 ms Generated Tokens: 128					VRAM Model: 26784.88 MiB (41/41 layers) [ROCm0: 26784.9] KV Cache: 80.00 MiB [f16] [ROCm0: 80.0] Compute Buf: 513.02 MiB [ROCm0: 497.0 \| ROCm_Host: 16.0] SSM/RS State: 62.81 MiB [ROCm0: 62.8] RAM BPE/Meta: 397.85 MiB RAM Layers: 0.00 MiB (0/41 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11129 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	Qwen3.5-35B-A3B-heretic-v2.i1-Q6_K.gguf	TG = 46.34	PP = 781.76	TTFT = 597.65	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:03:02
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2762.12 ms Generated Tokens: 128					VRAM Model: 26784.88 MiB (41/41 layers) [CUDA0: 6331.1 \| RPC0: 20453.8] KV Cache: 160.00 MiB [f16] [CUDA0: 48.0 \| RPC0: 112.0] Compute Buf: 609.04 MiB [RPC0: 92.0 \| CUDA0: 493.0 \| CPU: 24.0] SSM/RS State: 251.25 MiB [CUDA0: 50.2 \| RPC0: 201.0] RAM BPE/Meta: 397.85 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:211W]]	1	Qwen3.6-27B-UD-Q6_K_XL.gguf	TG = 16.54	PP = 143.69	TTFT = 3448.35	504.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-23 13:26:59
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 7741.11 ms Generated Tokens: 128					VRAM Model: 23150.10 MiB (65/65 layers) [ROCm0: 23150.1] KV Cache: 1071.00 MiB [q8_0] [ROCm0: 1071.0] Compute Buf: 578.29 MiB [ROCm0: 495.0 \| ROCm_Host: 83.3] SSM/RS State: 149.62 MiB [ROCm0: 149.6] RAM BPE/Meta: 1288.28 MiB RAM Layers: 0.00 MiB (0/65 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:245W]] [ROCm1] AMD Radeon VII (16340 MiB) [[max Watt:270W]]	2	Qwen3.6-27B-UD-Q8_K_XL.gguf	TG = 14.96	PP = 132.46	TTFT = 3744.43	584.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-23 13:09:07
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 8562.06 ms Generated Tokens: 128					VRAM Model: 31253.22 MiB (65/65 layers) [ROCm0: 18871.9 \| ROCm1: 12381.3] KV Cache: 1071.00 MiB [q8_0] [ROCm0: 736.3 \| ROCm1: 334.7] Compute Buf: 1405.90 MiB [ROCm0: 470.6 \| ROCm1: 662.1 \| ROCm_Host: 273.1] SSM/RS State: 149.63 MiB [ROCm0: 102.9 \| ROCm1: 46.8] RAM BPE/Meta: 2425.00 MiB RAM Layers: 0.00 MiB (0/65 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:177W]] [ROCm1] AMD Radeon VII (16340 MiB) [[max Watt:156W]]	2	Qwen3.6-35B-A3B-Q8_0.gguf	TG = 49.91	PP = 596.94	TTFT = 830.90	401.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-23 13:06:38
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2565.23 ms Generated Tokens: 128					VRAM Model: 34667.78 MiB (41/41 layers) [ROCm0: 23906.7 \| ROCm1: 10761.1] KV Cache: 334.69 MiB [q8_0] [ROCm0: 234.3 \| ROCm1: 100.4] Compute Buf: 1273.90 MiB [ROCm0: 376.6 \| ROCm1: 636.1 \| ROCm_Host: 261.1] SSM/RS State: 62.81 MiB [ROCm0: 44.0 \| ROCm1: 18.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:202W]]	1	Qwen3.6-35B-A3B-UD-Q6_K.gguf	TG = 60.11	PP = 667.46	TTFT = 743.22	475.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-23 13:24:55
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2129.59 ms Generated Tokens: 128					VRAM Model: 27187.46 MiB (41/41 layers) [ROCm0: 27187.5] KV Cache: 334.69 MiB [q8_0] [ROCm0: 334.7] Compute Buf: 564.29 MiB [ROCm0: 493.0 \| ROCm_Host: 71.3] SSM/RS State: 62.81 MiB [ROCm0: 62.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:174W]]	1	Qwen3.6-35B-A3B-UD-Q6_K.gguf	TG = 59.91	PP = 649.84	TTFT = 780.80	274.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-17 16:20:29
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2136.60 ms Generated Tokens: 128					VRAM Model: 27187.46 MiB (41/41 layers) [ROCm0: 27187.5] KV Cache: 584.38 MiB [q8_0] [ROCm0: 584.4] Compute Buf: 611.29 MiB [ROCm0: 493.0 \| ROCm_Host: 118.3] SSM/RS State: 62.81 MiB [ROCm0: 62.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:187W]] [ROCm1] AMD Radeon VII (16332 MiB) [[max Watt:140W]]	2	Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf	TG = 50.14	PP = 490.98	TTFT = 1009.52	402.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-23 13:04:37
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2553.13 ms Generated Tokens: 128					VRAM Model: 36144.11 MiB (41/41 layers) [ROCm0: 24659.8 \| ROCm1: 11484.4] KV Cache: 334.69 MiB [q8_0] [ROCm0: 234.3 \| ROCm1: 100.4] Compute Buf: 1273.90 MiB [ROCm0: 376.6 \| ROCm1: 636.1 \| ROCm_Host: 261.1] SSM/RS State: 62.81 MiB [ROCm0: 44.0 \| ROCm1: 18.8] RAM BPE/Meta: 515.31 MiB RAM Layers: 0.00 MiB (0/41 layers)
No data	0	Ternary-Bonsai-8B-Q2_0.gguf	TG = 4.38	PP = 5.07	TTFT = 6515.15	123.0 W	No data	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-26 00:20:33
		Server Environment: Native Host Build: b8846-d104cf1b6 User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: 5 Context Limit: ∞ Avg Time: 5065.29 ms Generated Tokens: 23.2					VRAM Model: 0 MiB (0/0 layers) KV Cache: 4608.00 MiB [f16] [CPU: 4608.0] Compute Buf: 312.23 MiB [CPU: 312.2] RAM BPE/Meta: 2075.42 MiB RAM Layers: 0.00 MiB (0/0 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon VII (16340 MiB) [[max Watt:23W]]	1	Ternary-Bonsai-8B-Q2_0.gguf	TG = 80.01	PP = 499.48	TTFT = 307.82	286.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-26 00:28:34
		Server Environment: Native Host Build: b8846-d104cf1b6 User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1599.79 ms Generated Tokens: 128					VRAM Model: 1918.05 MiB (37/37 layers) [ROCm0: 1918.0] KV Cache: 9216.00 MiB [f16] [ROCm0: 9216.0] Compute Buf: 448.24 MiB [ROCm0: 304.2 \| ROCm_Host: 144.0] RAM BPE/Meta: 157.37 MiB RAM Layers: 0.00 MiB (0/37 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:155W]]	1	gemma-4-26B-A4B-it-UD-Q4_K_M.gguf	TG = 10.04	PP = 268.05	TTFT = 1764.79	250.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 22:12:20
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: 5 Context Limit: ∞ Avg Time: 5040.16 ms Generated Tokens: 51.6					VRAM Model: 0 MiB (0/31 layers) KV Cache: 10699.37 MiB [q8_0] [CPU: 10380.6 \| CPU: 318.8] Compute Buf: 4575.65 MiB [ROCm0: 3582.1 \| ROCm_Host: 993.5] RAM BPE/Meta: Inc. below RAM Layers: 16071.72 MiB (31/31 layers + BPE) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:173W]]	1	gemma-4-26B-A4B-it-UD-Q4_K_M.gguf	TG = 74.49	PP = 837.44	TTFT = 564.34	268.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 22:51:45
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1718.33 ms Generated Tokens: 128					VRAM Model: 16071.80 MiB (31/31 layers) [ROCm0: 16071.8] KV Cache: 1360.00 MiB [q8_0] [ROCm0: 1041.2 \| ROCm0: 318.8] Compute Buf: 640.52 MiB [ROCm0: 528.5 \| ROCm_Host: 112.0] RAM BPE/Meta: 748.00 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:156W]] [ROCm1] AMD Radeon VII (16340 MiB) [[max Watt:149W]]	2	gemma-4-26B-A4B-it-uncensored-Q8_0.gguf	TG = 62.51	PP = 724.50	TTFT = 651.49	327.3 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-08 19:17:21
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / root / PL ISP: Bydgoszcz	Test Settings: Rounds: 2 Max Time: 30 Context Limit: ∞ Avg Time: 30002.07 ms Generated Tokens: 1876.5					VRAM Model: 25600.56 MiB (31/31 layers) [ROCm0: 17389.5 \| ROCm1: 8211.1] KV Cache: 2879.37 MiB [q8_0] [ROCm0: 1632.0 \| ROCm1: 1088.0 \| ROCm0: 114.8 \| ROCm1: 44.6] Compute Buf: 6075.49 MiB [ROCm0: 2430.7 \| ROCm1: 1573.7 \| ROCm_Host: 2071.1] RAM BPE/Meta: 748.00 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	gemma-4-26B-A4B-it-uncensored-Q8_0.gguf	TG = 75.79	PP = 736.27	TTFT = 640.54	N/A	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-08 09:55:31
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 300 Avg Time: 3958.41 ms Generated Tokens: 300					VRAM Model: 25600.56 MiB (31/31 layers) [ROCm0: 25600.6] KV Cache: 1519.38 MiB [q8_0] [ROCm0: 1360.0 \| ROCm0: 159.4] Compute Buf: 798.52 MiB [ROCm0: 528.5 \| ROCm_Host: 270.0] RAM BPE/Meta: 748.00 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [[max Watt:243W]]	1	gemma-4-26B-A4B-it-uncensored-Q8_0.gguf	TG = 75.97	PP = 736.97	TTFT = 640.73	320.9 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-08 18:25:39
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / root / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1684.85 ms Generated Tokens: 128					VRAM Model: 25600.56 MiB (31/31 warstw) [ROCm0: 25600.6] KV Cache: 2879.38 MiB [q8_0] [ROCm0: 2720.0 \| ROCm0: 159.4] Bufor Oblicz: 1345.66 MiB [ROCm0: 819.6 \| ROCm_Host: 526.0] RAM BPE/Meta: 748.00 MiB RAM Warstwy: 0.00 MiB (0/31 warstw) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon VII (16340 MiB free) [ROCm1] AMD Radeon Graphics (32732 MiB free)	2	gemma-4-31B-it-UD-Q8_K_XL.gguf	TG = 17.41	PP = 120.35	TTFT = 4008.50	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-03 16:22:10
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 4879.83 ms Generated Tokens: 79.2					VRAM Model: 33382.72 MiB (61/61 layers) [ROCm0: 10729.5 \| ROCm1: 22653.2] KV Cache: 6077.50 MiB [q8_0] [ROCm0: 1632.0 \| ROCm1: 3808.0 \| ROCm0: 229.5 \| ROCm1: 408.0] Compute Buf: 3558.24 MiB [ROCm0: 1418.6 \| ROCm1: 1082.6 \| ROCm_Host: 1057.1] RAM BPE/Meta: 1428.00 MiB RAM Layers: 0.00 MiB (0/61 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:171W]]	1	gemma-4-31B-it-UD-Q8_K_XL.gguf	TG = 1.08	PP = 58.58	TTFT = 8049.66	266.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 23:18:16
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: 10 Context Limit: ∞ Avg Time: 10555.33 ms Generated Tokens: 12.4					VRAM Model: 0 MiB (0/61 layers) KV Cache: 4791.88 MiB [q8_0] [CPU: 4154.4 \| CPU: 637.5] Compute Buf: 2175.52 MiB [ROCm0: 1950.5 \| ROCm_Host: 225.0] RAM BPE/Meta: Inc. below RAM Layers: 33382.61 MiB (61/61 layers + BPE) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:188W]]	1	gemma-4-E4B-it-Q4_K_M.gguf	TG = 73.22	PP = 1214.60	TTFT = 388.61	287.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 21:54:06
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1748.19 ms Generated Tokens: 128					VRAM Model: 2883.51 MiB (43/43 layers) [ROCm0: 2883.5] KV Cache: 8347.00 MiB [q8_0] [ROCm0: 8304.5 \| ROCm0: 42.5] Compute Buf: 2534.53 MiB [ROCm0: 1529.5 \| ROCm_Host: 1005.0] RAM BPE/Meta: 2208.00 MiB RAM Layers: 0.00 MiB (0/43 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:198W]]	1	gemma-4-E4B-it-Q4_K_M.gguf	TG = 73.26	PP = 1217.48	TTFT = 387.52	293.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 22:53:23
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1747.28 ms Generated Tokens: 128					VRAM Model: 2883.51 MiB (43/43 layers) [ROCm0: 2883.5] KV Cache: 875.50 MiB [q8_0] [ROCm0: 833.0 \| ROCm0: 42.5] Compute Buf: 643.02 MiB [ROCm0: 517.0 \| ROCm_Host: 126.0] RAM BPE/Meta: 2208.00 MiB RAM Layers: 0.00 MiB (0/43 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:169W]]	1	gemma-4-E4B-it-Q4_K_M.gguf	TG = 12.15	PP = 665.72	TTFT = 710.11	264.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-12 00:05:49
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 10535.13 ms Generated Tokens: 128					VRAM Model: 0 MiB (0/43 layers) KV Cache: 89.25 MiB [q8_0] [CPU: 68.0 \| CPU: 21.2] Compute Buf: 972.53 MiB [ROCm0: 928.5 \| ROCm_Host: 44.0] RAM BPE/Meta: Inc. below RAM Layers: 4731.51 MiB (43/43 layers + BPE) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon VII (16340 MiB free) [ROCm1] AMD Radeon Graphics (32732 MiB free)	2	gemma-4-E4B-it-UD-Q8_K_XL.gguf	TG = 56.87	PP = 760.57	TTFT = 633.78	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-03 16:18:23
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2250.75 ms Generated Tokens: 128					VRAM Model: 8242.46 MiB (43/43 layers) [ROCm0: 1669.1 \| ROCm1: 6573.3] KV Cache: 1109.25 MiB [q8_0] [ROCm0: 544.0 \| ROCm1: 544.0 \| ROCm0: 13.8 \| ROCm1: 7.4] Compute Buf: 3435.25 MiB [ROCm0: 1297.1 \| ROCm1: 1096.1 \| ROCm_Host: 1042.1] RAM BPE/Meta: 680.00 MiB RAM Layers: 0.00 MiB (0/43 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon VII (16332 MiB free) [ROCm1] AMD Radeon Graphics (32732 MiB free)	2	google_gemma-4-26B-A4B-it-IQ4_NL.gguf	TG = 64.97	PP = 791.07	TTFT = 611.32	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-03 17:20:46
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: 5 Context Limit: ∞ Avg Time: 3039.82 ms Generated Tokens: 186.6					VRAM Model: 14001.81 MiB (31/31 layers) [ROCm0: 4915.4 \| ROCm1: 9086.5] KV Cache: 1519.37 MiB [q8_0] [ROCm0: 272.0 \| ROCm1: 1088.0 \| ROCm0: 63.8 \| ROCm1: 95.6] Compute Buf: 3387.49 MiB [ROCm0: 1278.7 \| ROCm1: 1061.7 \| ROCm_Host: 1047.1] RAM BPE/Meta: 577.50 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	google_gemma-4-26B-A4B-it-IQ4_NL.gguf	TG = 75.79	PP = 832.76	TTFT = 565.59	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-03 22:22:40
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 1000 Avg Time: 13194.61 ms Generated Tokens: 1000					VRAM Model: 14001.81 MiB (31/31 layers) [ROCm0: 14001.8] KV Cache: 1519.38 MiB [q8_0] [ROCm0: 1360.0 \| ROCm0: 159.4] Compute Buf: 798.52 MiB [ROCm0: 528.5 \| ROCm_Host: 270.0] RAM BPE/Meta: 577.50 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[Vulkan0] AMD Radeon VII (RADV VEGA20) (15612 MiB free) [Vulkan1] AMD Radeon Graphics (RADV VEGA20) (32751 MiB free)	2	google_gemma-4-26B-A4B-it-IQ4_NL.gguf	TG = 37.92	PP = 493.11	TTFT = 956.44	N/A	VULKAN	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-03 21:48:12
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 5000 Avg Time: 131858.88 ms Generated Tokens: 5000					VRAM Model: 14001.74 MiB (31/31 layers) [Vulkan0: 4915.3 \| Vulkan1: 9086.4] KV Cache: 1519.37 MiB [q8_0] [Vulkan0: 272.0 \| Vulkan1: 1088.0 \| Vulkan0: 63.8 \| Vulkan1: 95.6] Compute Buf: 3487.67 MiB [Vulkan0: 1290.0 \| Vulkan1: 1150.6 \| Vulkan_Host: 1047.1] RAM BPE/Meta: 577.50 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[Vulkan0] AMD Radeon Graphics (RADV VEGA20) (32751 MiB free)	1	google_gemma-4-26B-A4B-it-IQ4_NL.gguf	TG = 74.64	PP = 474.21	TTFT = 993.24	N/A	VULKAN	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-03 22:15:37
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 1000 Avg Time: 13397.26 ms Generated Tokens: 1000					VRAM Model: 14001.74 MiB (31/31 layers) [Vulkan0: 14001.7] KV Cache: 1519.38 MiB [q8_0] [Vulkan0: 1360.0 \| Vulkan0: 159.4] Compute Buf: 797.44 MiB [Vulkan0: 527.4 \| Vulkan_Host: 270.0] RAM BPE/Meta: 577.50 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:128W]]	1	google_gemma-4-26B-A4B-it-Q8_0.gguf	TG = 8.58	PP = 245.90	TTFT = 1915.95	223.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 22:40:13
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: 5 Context Limit: ∞ Avg Time: 5079.10 ms Generated Tokens: 44.6					VRAM Model: 0 MiB (0/31 layers) KV Cache: 10699.37 MiB [q8_0] [CPU: 10380.6 \| CPU: 318.8] Compute Buf: 4575.65 MiB [ROCm0: 3582.1 \| ROCm_Host: 993.5] RAM BPE/Meta: Inc. below RAM Layers: 25600.47 MiB (31/31 layers + BPE) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:179W]]	1	google_gemma-4-26B-A4B-it-Q8_0.gguf	TG = 75.73	PP = 744.75	TTFT = 631.90	274.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 22:59:35
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1690.12 ms Generated Tokens: 128					VRAM Model: 25600.56 MiB (31/31 layers) [ROCm0: 25600.6] KV Cache: 1360.00 MiB [q8_0] [ROCm0: 1041.2 \| ROCm0: 318.8] Compute Buf: 640.52 MiB [ROCm0: 528.5 \| ROCm_Host: 112.0] RAM BPE/Meta: 748.00 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	google_gemma-4-26B-A4B-it-Q8_0.gguf	TG = 75.57	PP = 741.64	TTFT = 635.62	N/A	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-08 09:58:38
		Server Environment: Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 300 Avg Time: 3970.04 ms Generated Tokens: 300					VRAM Model: 25600.56 MiB (31/31 layers) [ROCm0: 25600.6] KV Cache: 1519.38 MiB [q8_0] [ROCm0: 1360.0 \| ROCm0: 159.4] Compute Buf: 798.52 MiB [ROCm0: 528.5 \| ROCm_Host: 270.0] RAM BPE/Meta: 748.00 MiB RAM Layers: 0.00 MiB (0/31 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:87W]]	1	gpt-oss-20b-Q4_K_M.gguf	TG = 13.99	PP = 364.90	TTFT = 540.81	182.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 23:02:09
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 9153.52 ms Generated Tokens: 128					VRAM Model: 0 MiB (0/25 layers) KV Cache: 1268.62 MiB [q8_0] [CPU: 1249.5 \| CPU: 19.1] Compute Buf: 1096.00 MiB [ROCm0: 985.2 \| ROCm_Host: 110.8] RAM BPE/Meta: Inc. below RAM Layers: 11073.83 MiB (25/25 layers + BPE) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon VII (16340 MiB) [[max Watt:24W]]	1	gpt-oss-20b-Q4_K_M.gguf	TG = 82.34	PP = 862.71	TTFT = 249.70	275.0 W	ROCM	OS: Linux-6.17.0-22-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-23 13:12:15
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1554.50 ms Generated Tokens: 128					VRAM Model: 10694.15 MiB (25/25 layers) [ROCm0: 10694.1] KV Cache: 411.18 MiB [q8_0] [ROCm0: 401.6 \| ROCm0: 9.6] Compute Buf: 474.18 MiB [ROCm0: 398.4 \| ROCm_Host: 75.8] RAM BPE/Meta: 379.71 MiB RAM Layers: 0.00 MiB (0/25 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:242W]]	1	gpt-oss-20b-Q4_K_M.gguf	TG = 59.92	PP = 642.60	TTFT = 4759.14	337.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-14 20:25:08
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 20 Max Time: ∞ Context Limit: 128 Avg Time: 2367.81 ms Generated Tokens: 128					VRAM Model: 10694.15 MiB (25/25 layers) [ROCm0: 10694.1] KV Cache: 6400.50 MiB [q8_0] [ROCm0: 6381.4 \| ROCm0: 19.1] Compute Buf: 1285.60 MiB [ROCm0: 772.3 \| ROCm_Host: 513.3] RAM BPE/Meta: 379.71 MiB RAM Layers: 0.00 MiB (0/25 layers) SSM/RS State: 0 MiB
[CUDA0] NVIDIA GeForce RTX 3050 Laptop GPU (3299 MiB free) [RPC0] 192.168.0.222:50052 (27365 MiB free)	2	gpt-oss-20b-Q4_K_M.gguf	TG = 28.10	PP = 64.49	TTFT = 1542.54	N/A	CUDA + RPC	OS: Windows-10-10.0.19045-SP0 CUDA: 591.59 MMAP: false	2026-03-28 13:41:12
		Server Environment: Host Build: b8468-3306dbaef User Host Data: f17w10 / Doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 3499.02 ms Generated Tokens: 106					VRAM Model: 11073.86 MiB (25/25 layers) [CUDA0: 1008.2 \| CUDA_Host: 379.7 \| RPC0: 9685.9] KV Cache: 32.06 MiB [q4_0] [CUDA0: 2.2 \| RPC0: 24.8 \| RPC0: 5.1] Compute Buf: 516.17 MiB [RPC0: 97.0 \| CUDA0: 398.4 \| CPU: 20.8] RAM BPE/Meta: 0 MiB RAM Layers: 0.00 MiB (0/25 layers) SSM/RS State: 0 MiB
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	nvidia_Nemotron-3-Nano-30B-A3B-Q8_0.gguf	TG = 84.38	PP = 656.95	TTFT = 711.84	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:06:02
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1516.95 ms Generated Tokens: 128					VRAM Model: 31665.15 MiB (53/53 layers) [ROCm0: 20483.5 \| ROCm1: 11181.6] KV Cache: 3264.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 544.0] Compute Buf: 6242.17 MiB [ROCm0: 3090.6 \| ROCm1: 1093.1 \| ROCm_Host: 2058.5] SSM/RS State: 47.61 MiB [ROCm0: 33.1 \| ROCm1: 14.5] RAM BPE/Meta: 357.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	nvidia_Nemotron-3-Nano-30B-A3B-Q8_0.gguf	TG = 93.42	PP = 664.88	TTFT = 705.16	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:42:20
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1370.10 ms Generated Tokens: 128					VRAM Model: 31665.15 MiB (53/53 layers) [ROCm0: 31665.2] KV Cache: 6.75 MiB [q4_0] [ROCm0: 6.8] Compute Buf: 285.08 MiB [ROCm0: 266.6 \| ROCm_Host: 18.5] SSM/RS State: 47.62 MiB [ROCm0: 47.6] RAM BPE/Meta: 357.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11090 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	nvidia_Nemotron-3-Nano-30B-A3B-Q8_0.gguf	TG = 73.28	PP = 789.68	TTFT = 593.66	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:55:25
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1746.81 ms Generated Tokens: 128					VRAM Model: 31665.15 MiB (53/53 layers) [CUDA0: 8471.6 \| RPC0: 23193.6] KV Cache: 48.00 MiB [f16] [CUDA0: 8.0 \| RPC0: 40.0] Compute Buf: 389.33 MiB [RPC0: 96.0 \| CUDA0: 266.8 \| CPU: 26.5] SSM/RS State: 190.47 MiB [CUDA0: 41.4 \| RPC0: 149.1] RAM BPE/Meta: 357.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf	TG = 84.52	PP = 723.37	TTFT = 648.90	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:08:27
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1514.40 ms Generated Tokens: 128					VRAM Model: 23341.83 MiB (53/53 layers) [ROCm0: 14756.1 \| ROCm1: 8585.8] KV Cache: 3264.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 544.0] Compute Buf: 21894.94 MiB [ROCm0: 9314.1 \| ROCm1: 4378.3 \| ROCm_Host: 8202.5] SSM/RS State: 47.61 MiB [ROCm0: 33.1 \| ROCm1: 14.5] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32520 MiB) [[max Watt:169W]]	1	nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf	TG = 11.87	PP = 234.64	TTFT = 2009.42	264.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 22:44:19
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: 5 Context Limit: ∞ Avg Time: 5053.82 ms Generated Tokens: 61					VRAM Model: 0 MiB (0/53 layers) KV Cache: 3114.19 MiB [q8_0] [CPU: 3114.2] Compute Buf: 3043.48 MiB [ROCm0: 2013.1 \| ROCm_Host: 1030.3] SSM/RS State: 95.23 MiB [CPU: 95.2] RAM BPE/Meta: Inc. below RAM Layers: 23572.77 MiB (53/53 layers + BPE)
[ROCm0] AMD Radeon Graphics (32732 MiB) [[max Watt:191W]]	1	nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf	TG = 12.02	PP = 232.52	TTFT = 2030.64	286.0 W	ROCM	OS: Linux-6.17.0-20-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-04-11 23:04:41
		Server Environment: Native Host Build: b8651-d3416a4aa User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 10650.31 ms Generated Tokens: 128					VRAM Model: 0 MiB (0/53 layers) KV Cache: 312.38 MiB [q8_0] [CPU: 312.4] Compute Buf: 1259.45 MiB [ROCm0: 1108.1 \| ROCm_Host: 151.3] SSM/RS State: 95.23 MiB [CPU: 95.2] RAM BPE/Meta: Inc. below RAM Layers: 23572.77 MiB (53/53 layers + BPE)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf	TG = 96.39	PP = 743.60	TTFT = 630.18	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:19:24
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1327.97 ms Generated Tokens: 128					VRAM Model: 23341.83 MiB (53/53 layers) [ROCm0: 23341.8] KV Cache: 24.00 MiB [f16] [ROCm0: 24.0] Compute Buf: 285.08 MiB [ROCm0: 266.6 \| ROCm_Host: 18.5] SSM/RS State: 47.62 MiB [ROCm0: 47.6] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11046 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	nvidia_Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf	TG = 75.20	PP = 869.88	TTFT = 538.94	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:22:25
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1702.74 ms Generated Tokens: 128					VRAM Model: 23341.83 MiB (53/53 layers) [CUDA0: 6585.9 \| RPC0: 16756.0] KV Cache: 48.00 MiB [f16] [CUDA0: 8.0 \| RPC0: 40.0] Compute Buf: 389.33 MiB [RPC0: 96.0 \| CUDA0: 266.8 \| CPU: 26.5] SSM/RS State: 190.47 MiB [CUDA0: 41.4 \| RPC0: 149.1] RAM BPE/Meta: 231.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	nvidia_Nemotron-Cascade-2-30B-A3B-Q8_0.gguf	TG = 85.21	PP = 648.49	TTFT = 722.34	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:09:11
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1502.30 ms Generated Tokens: 128					VRAM Model: 31665.15 MiB (53/53 layers) [ROCm0: 20483.5 \| ROCm1: 11181.6] KV Cache: 3264.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 544.0] Compute Buf: 6242.17 MiB [ROCm0: 3090.6 \| ROCm1: 1093.1 \| ROCm_Host: 2058.5] SSM/RS State: 47.61 MiB [ROCm0: 33.1 \| ROCm1: 14.5] RAM BPE/Meta: 357.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	nvidia_Nemotron-Cascade-2-30B-A3B-Q8_0.gguf	TG = 91.76	PP = 661.61	TTFT = 709.25	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:41:21
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1396.11 ms Generated Tokens: 128					VRAM Model: 31665.15 MiB (53/53 layers) [ROCm0: 31665.2] KV Cache: 6.75 MiB [q4_0] [ROCm0: 6.8] Compute Buf: 285.08 MiB [ROCm0: 266.6 \| ROCm_Host: 18.5] SSM/RS State: 47.62 MiB [ROCm0: 47.6] RAM BPE/Meta: 357.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11000 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	nvidia_Nemotron-Cascade-2-30B-A3B-Q8_0.gguf	TG = 72.76	PP = 782.02	TTFT = 599.79	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:20:35
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 1759.17 ms Generated Tokens: 128					VRAM Model: 31665.15 MiB (53/53 layers) [CUDA0: 8471.6 \| RPC0: 23193.6] KV Cache: 48.00 MiB [f16] [CUDA0: 8.0 \| RPC0: 40.0] Compute Buf: 389.33 MiB [RPC0: 96.0 \| CUDA0: 266.8 \| CPU: 26.5] SSM/RS State: 190.47 MiB [CUDA0: 41.4 \| RPC0: 149.1] RAM BPE/Meta: 357.00 MiB RAM Layers: 0.00 MiB (0/53 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	omnicoder-9b-bf16.gguf	TG = 25.30	PP = 200.43	TTFT = 2332.04	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:10:22
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 5064.08 ms Generated Tokens: 128					VRAM Model: 15140.05 MiB (33/33 layers) [ROCm0: 9500.0 \| ROCm1: 5640.1] KV Cache: 4352.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 1632.0] Compute Buf: 6053.21 MiB [ROCm0: 2440.1 \| ROCm1: 1549.1 \| ROCm_Host: 2064.1] SSM/RS State: 50.25 MiB [ROCm0: 37.7 \| ROCm1: 12.6] RAM BPE/Meta: 1940.00 MiB RAM Layers: 0.00 MiB (0/33 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free)	1	omnicoder-9b-bf16.gguf	TG = 27.42	PP = 210.59	TTFT = 2216.90	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 20:24:52
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 4668.69 ms Generated Tokens: 128					VRAM Model: 15140.05 MiB (33/33 layers) [ROCm0: 15140.0] KV Cache: 128.00 MiB [f16] [ROCm0: 128.0] Compute Buf: 525.02 MiB [ROCm0: 501.0 \| ROCm_Host: 24.0] SSM/RS State: 50.25 MiB [ROCm0: 50.2] RAM BPE/Meta: 1940.00 MiB RAM Layers: 0.00 MiB (0/33 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11122 MiB free) [RPC0] 127.0.0.1:50052 (32586 MiB free)	2	omnicoder-9b-bf16.gguf	TG = 25.54	PP = 256.80	TTFT = 1820.93	N/A	CUDA + RPC	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 18:57:51
		Server Environment: Host Build: b8465-568aec82d User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 5012.64 ms Generated Tokens: 128					VRAM Model: 15140.05 MiB (33/33 layers) [CUDA0: 4823.4 \| RPC0: 10316.7] KV Cache: 256.00 MiB [f16] [CUDA0: 64.0 \| RPC0: 192.0] Compute Buf: 629.04 MiB [RPC0: 104.0 \| CUDA0: 493.0 \| CPU: 32.0] SSM/RS State: 201.00 MiB [CUDA0: 41.9 \| RPC0: 159.1] RAM BPE/Meta: 1940.00 MiB RAM Layers: 0.00 MiB (0/33 layers)
[CUDA0] NVIDIA GeForce RTX 4070 (11062 MiB free)	1	omnicoder-9b-q8_0.gguf	TG = 49.22	PP = 3040.54	TTFT = 153.63	N/A	CUDA	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 CUDA: 580.126.09 / ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-27 19:54:39
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Bydgoszcz	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2600.57 ms Generated Tokens: 128					VRAM Model: 8045.05 MiB (33/33 layers) [CUDA0: 8045.1] KV Cache: 128.00 MiB [f16] [CUDA0: 128.0] Compute Buf: 525.02 MiB [CUDA0: 501.0 \| CUDA_Host: 24.0] SSM/RS State: 50.25 MiB [CUDA0: 50.2] RAM BPE/Meta: 1030.62 MiB RAM Layers: 0.00 MiB (0/33 layers)
[ROCm0] AMD Radeon Graphics (32732 MiB free) [ROCm1] AMD Radeon VII (16340 MiB free)	2	omnicoder-9b-q8_0.gguf	TG = 50.62	PP = 417.33	TTFT = 1120.01	N/A	ROCM	OS: Linux-6.17.0-19-generic-x86_64-with-glibc2.39 ROCM: 7.2.1.70201 / VULKAN: 1.3.275 MMAP: true	2026-03-31 18:11:14
		Server Environment: Host Build: b8530-a970515bd User Host Data: LianLi / doman / PL ISP: Warsaw	Test Settings: Rounds: 5 Max Time: ∞ Context Limit: 128 Avg Time: 2528.64 ms Generated Tokens: 128					VRAM Model: 8045.05 MiB (33/33 layers) [ROCm0: 5048.3 \| ROCm1: 2996.8] KV Cache: 4352.00 MiB [q8_0] [ROCm0: 2720.0 \| ROCm1: 1632.0] Compute Buf: 6053.21 MiB [ROCm0: 2440.1 \| ROCm1: 1549.1 \| ROCm_Host: 2064.1] SSM/RS State: 50.25 MiB [ROCm0: 37.7 \| ROCm1: 12.6] RAM BPE/Meta: 1030.62 MiB RAM Layers: 0.00 MiB (0/33 layers)

File Already Exists

Server Log:

Downloads

Llamabench6.2 Statistics v2