vasilito
cb424d7448
verify-patch-sanity.py validates every active recipe .patch has internally- consistent hunk line counts — catching the 'malformed patch at line N' failure at commit/CI/preflight time instead of hours into a cook. This cycle hit that class three times (qtwaylandscanner, sddm, xwayland), each only discovered when cookbook tried to apply the patch. Running it across the repo found 29 latent malformed patches (validated against GNU patch: e.g. relibc/P3-sysv-ipc reproduces 'malformed patch at line 22'). They were harmless only because they sit in vendored recipes (baked, not re- applied) — but would fail on any version-bump re-derivation. --fix recounts the hunk headers (body untouched) and repaired all 29. Wired into build-preflight.sh (Phase 1.0D) and redbear-ci.yml, with a unit test (test-patch-sanity.sh). Skips archived/legacy trees and unvalidatable formats (empty placeholders, bare-@@ git hunks).
88 lines
2.6 KiB
ReStructuredText
88 lines
2.6 KiB
ReStructuredText
==================================
|
|
Benchmarking tips
|
|
==================================
|
|
|
|
|
|
Introduction
|
|
============
|
|
|
|
For benchmarking a patch we want to reduce all possible sources of
|
|
noise as much as possible. How to do that is very OS dependent.
|
|
|
|
Note that low noise is required, but not sufficient. It does not
|
|
exclude measurement bias.
|
|
See `"Producing Wrong Data Without Doing Anything Obviously Wrong!" by Mytkowicz, Diwan, Hauswith and Sweeney (ASPLOS 2009) <https://users.cs.northwestern.edu/~robby/courses/322-2013-spring/mytkowicz-wrong-data.pdf>`_
|
|
for example.
|
|
|
|
General
|
|
================================
|
|
|
|
* Use a high resolution timer, e.g. perf under linux.
|
|
|
|
* Run the benchmark multiple times to be able to recognize noise.
|
|
|
|
* Disable as many processes or services as possible on the target system.
|
|
|
|
* Disable frequency scaling, turbo boost and address space
|
|
randomization (see OS specific section).
|
|
|
|
* Static link if the OS supports it. That avoids any variation that
|
|
might be introduced by loading dynamic libraries. This can be done
|
|
by passing ``-DLLVM_BUILD_STATIC=ON`` to cmake.
|
|
|
|
* Try to avoid storage. On some systems you can use tmpfs. Putting the
|
|
program, inputs and outputs on tmpfs avoids touching a real storage
|
|
system, which can have a pretty big variability.
|
|
|
|
To mount it (on linux and freebsd at least)::
|
|
|
|
mount -t tmpfs -o size=<XX>g none dir_to_mount
|
|
|
|
Linux
|
|
=====
|
|
|
|
* Disable address space randomization::
|
|
|
|
echo 0 > /proc/sys/kernel/randomize_va_space
|
|
|
|
* Set scaling_governor to performance::
|
|
|
|
for i in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
|
|
do
|
|
echo performance > $i
|
|
done
|
|
|
|
* Use https://github.com/lpechacek/cpuset to reserve cpus for just the
|
|
program you are benchmarking. If using perf, leave at least 2 cores
|
|
so that perf runs in one and your program in another::
|
|
|
|
cset shield -c N1,N2 -k on
|
|
|
|
This will move all threads out of N1 and N2. The ``-k on`` means
|
|
that even kernel threads are moved out.
|
|
|
|
* Disable the SMT pair of the cpus you will use for the benchmark. The
|
|
pair of cpu N can be found in
|
|
``/sys/devices/system/cpu/cpuN/topology/thread_siblings_list`` and
|
|
disabled with::
|
|
|
|
echo 0 > /sys/devices/system/cpu/cpuX/online
|
|
|
|
|
|
* Run the program with::
|
|
|
|
cset shield --exec -- perf stat -r 10 <cmd>
|
|
|
|
This will run the command after ``--`` in the isolated cpus. The
|
|
particular perf command runs the ``<cmd>`` 10 times and reports
|
|
statistics.
|
|
|
|
With these in place you can expect perf variations of less than 0.1%.
|
|
|
|
Linux Intel
|
|
-----------
|
|
|
|
* Disable turbo mode::
|
|
|
|
echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo
|