Fast PCH generator and optimized PCH files

Ported update_pch.sh to Python with improved performance
and features. The new script is invoked from the same
update_pch.sh which calls it for each library in
parallel, although it can be invoked directly.

The ported script (update_pch) updates all PCH files
in ~15 seconds where the old script took ~4500 seconds.
In addition, the new script supports 3-tiered headers
(system, module, and local) and is very flexible to
support other improvement. It has a per-library
optimal configuration settings that can be updated
using another new scripts (update_pch_autotune.sh)
which finds optimal per-PCH settings.

PCH files have been generated using the new scripts
which builds significantly faster (2-3x, depending
on module and configuration) and the intermediate
binaries are noticably smaller (by several GBs).

The new script stamps each generated PCH file with
the command that generated it to make it trivial
for users to update them, and also adds the command
to invoke another script (update_pch_bisect) that
helps find missing headers or conflicting headers
that may break the build after updating the PCH.

Finally update_pch has built-in unit-tests for
makefile parsing and other core functionality.

Change-Id: Ib933b50e50374d7e2e7e3e95ba8799b0cc8a27fa
Reviewed-on: https://gerrit.libreoffice.org/19965
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Caolán McNamara <caolanm@redhat.com>
Tested-by: Caolán McNamara <caolanm@redhat.com>

1 files changed, 3 insertions, 6 deletions

diff --git a/bin/update_pch_bisect b/bin/update_pch_bisect
index f88b58dfcf40..86cb78396d2c 100644
--- a/bin/update_pch_bisect
+++ b/bin/update_pch_bisect
@@ -23,9 +23,6 @@ header that compiles fine, however,
 it contains one or more required
 include without which it wouldn't
 compile, which it identifies.
-This mode is used to find source
-files that miss one or more includes
-and rely on pch file to compile.
 
 Usage: ./bin/update_pch_bisect ./vcl/inc/pch/precompiled_vcl.hxx "/opt/lo/bin/make vcl.build" --find-required --verbose
 """
@@ -99,11 +96,11 @@ def bisect(lines, marks, min, max, update, command):
                 marks[min] = TEST_ON
                 update(lines, marks)
                 if command():
-                    log(' Found #{}: {}'.format(min+1, lines[min].strip('\n')))
+                    log(' Found @{}: {}'.format(min+1, lines[min].strip('\n')))
                     marks[min] = GOOD
                     return marks
             else:
-                log(' Found #{}: {}'.format(min+1, lines[min].strip('\n')))
+                log(' Found @{}: {}'.format(min+1, lines[min].strip('\n')))
             # Either way, this one is irrelevant.
             marks[min] = BAD
             return marks
@@ -233,7 +230,7 @@ def main():
 
 if __name__ == '__main__':
 
-    if len(sys.argv) in (3, 5):
+    if len(sys.argv) in (3, 4, 5):
         status = main()
         sys.exit(status)