summaryrefslogtreecommitdiff
path: root/external/python3
AgeCommit message (Collapse)Author
2022-03-04external/python3: Avoid "ThreadSanitizer: destroy of a locked mutex"Stephan Bergmann
..when building ExternalProject_python3 with Clang -fsanitize=thread, > WARNING: ThreadSanitizer: destroy of a locked mutex (pid=973799) > #0 in AnnotateRWLockDestroy at /home/sbergman/github.com/llvm/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interface_ann.cpp:184:3 (workdir/UnpackedTarball/python3/python +0x498460) > #1 in recreate_gil at workdir/UnpackedTarball/python3/Python/ceval_gil.h:138:5 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0xe84aa9) > #2 in _PyEval_ReInitThreads at workdir/UnpackedTarball/python3/Python/ceval.c:350:5 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0xe847c7) > #3 in PyOS_AfterFork_Child at workdir/UnpackedTarball/python3/./Modules/posixmodule.c:469:5 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x1163dbd) > #4 in os_fork_impl at workdir/UnpackedTarball/python3/./Modules/posixmodule.c:6253:9 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x11adcad) > #5 in os_fork at workdir/UnpackedTarball/python3/./Modules/clinic/posixmodule.c.h:2750:12 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x117b481) > #6 in cfunction_vectorcall_NOARGS at workdir/UnpackedTarball/python3/Objects/methodobject.c:463:24 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x9db7e1) > #7 in _PyObject_Vectorcall at workdir/UnpackedTarball/python3/./Include/cpython/abstract.h:127:11 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0xf0225e) > #8 in call_function at workdir/UnpackedTarball/python3/Python/ceval.c:4963:13 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0xef3f4a) > #9 in _PyEval_EvalFrameDefault at workdir/UnpackedTarball/python3/Python/ceval.c:3469:23 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0xedc5d8) [...] > #143 in pymain_run_python at workdir/UnpackedTarball/python3/Modules/main.c:610:21 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x1149f6c) > #144 in Py_RunMain at workdir/UnpackedTarball/python3/Modules/main.c:689:5 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x11495e9) > #145 in pymain_main at workdir/UnpackedTarball/python3/Modules/main.c:719:12 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x114a299) > #146 in Py_BytesMain at workdir/UnpackedTarball/python3/Modules/main.c:743:12 (workdir/UnpackedTarball/python3/libpython3.8d.so.1.0 +0x114a30d) > #147 in main at workdir/UnpackedTarball/python3/./Programs/python.c:16:12 (workdir/UnpackedTarball/python3/python +0x4d00f8) Assuming that the GIL is always locked before the fork, better tell TSan about a fake RELEASED before telling it about a fake DESTROY, to keep TSan's model consistent. Change-Id: I6c68d7e415aa0ffc3047e5a5c4c4aca6b0cce8cb Reviewed-on: https://gerrit.libreoffice.org/c/core/+/130985 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2022-02-23external/python3: Explicitly check that all extension modules got builtStephan Bergmann
...to avoid issues like we now experienced on Jenkins box tb76, where e.g. <https://ci.libreoffice.org/job/gerrit_linux_clang_dbgutil/108402/> failed with just an unhelpful > /home/tdf/lode/jenkins/workspace/lo_gerrit/Config/linux_clang_dbgutil_64/external/python3/ExternalPackage_python3.mk:46: *** file /home/tdf/lode/jenkins/workspace/lo_gerrit/Config/linux_clang_dbgutil_64/workdir/UnpackedTarball/python3/LO_lib/_elementtree.cpython-3.8d.so does not exist in the tarball. Stop. > make[1]: *** Waiting for unfinished jobs.... > Makefile:299: recipe for target 'build' failed > make: *** [build] Error 2 after ExternalProject_python3 had been built successfully, outputting just > [build PRJ] python3 even though its workdir/UnpackedTarball/python3/build.log states > *** WARNING: renaming "_elementtree" since importing it failed: pyexpat version is incompatible and > Following modules built successfully but were removed because they could not be imported: > _elementtree (but which got hidden by gbuild) Change-Id: I28904ef41cb823e308bb8e15cbe969872702cb55 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/130355 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2022-01-31externals: always provide platform configure flagsJan-Marek Glogowski
No idea why we just provided the platform flags when cross- compiling. In the curious case, where the host platform is detected as x86_64-pc-mingw32 per default and we actually want to override it with x86_64-pc-cygwin, we don't do a cross compile, but must override the host platform. But there is additional special handling needed for the omitted cross-platform build in the special case of --host=i686-pc-cygwin and --build=x86_64-pc-cygwin, where we deliberatly ignore cross building; Windows is already a slow build, so try to keep this optimization (AMD64 can execute x86 binaries). There is the theoretical case, where the externals config.guess would have detected something else and that "magically" even worked, while the LO detected triplet would fail, but this should be fixed in the external in any way. Change-Id: Ib7a9719e0e406fe90334b7611dc3f01b51692bfa Reviewed-on: https://gerrit.libreoffice.org/c/core/+/129153 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2021-12-24python3: fix build on Win 10Aron Budea
With Windows 11 SDK (10.0.22000.0). Error message is: fatal error RC1116: RC terminating after preprocessor errors https://bugs.python.org/issue45220 Applied fixing patches to 3.8. Change-Id: I0860b05fd963ea81b493a4b9df7f39db86598dd0 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/127395 Tested-by: Mike Kaganski <mike.kaganski@collabora.com> Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-10-08Centralize VS-to-toolset mapping in configureMike Kaganski
This allows to define the mapping once, and avoid modification in multiple places each time a new VS version support is added Change-Id: I93de4c9d78c3f67a0a2e157007e9d13b6f557937 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/123163 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-09-16Add preliminary VS 2022 supportHossein
This patch changes the configure.ac, so that LibreOffice compiles with the latest VS 2022 preview. The option --with-visual-studio=2022 should be added to the autogen.input, in order to use VS 2022 preview. Change-Id: Ia1d9bbeabbbd44ffe82af3624151b69d36c0a45c Reviewed-on: https://gerrit.libreoffice.org/c/core/+/122133 Tested-by: Jenkins Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2021-08-20Make some scripts more portableIlmari Lauhakangas
Change-Id: Ia89059eea51ca396a7c74143625ac9a6706de198 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/120773 Tested-by: Jenkins Reviewed-by: Ilmari Lauhakangas <ilmari.lauhakangas@libreoffice.org>
2021-06-25python3: update to 3.8.10Jan-Marek Glogowski
So we don't build 3.8.8rc1 anymore. I didn't look into 3.9. Change-Id: Ife7d898c913b9b164168b0ef23a055deea55815f Reviewed-on: https://gerrit.libreoffice.org/c/core/+/117757 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2021-05-07external/python3: Mark configure part of darwin.patch.0 as reported upstreamStephan Bergmann
Change-Id: Ibf9fb088eb1d7e11582518aeafa233dfa7c56bf6 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/115223 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2021-05-07external/python3: Clang 13 trunk implements --print-multiarch nowStephan Bergmann
...since <https://github.com/llvm/llvm-project/commit/a921d2d2fb46b898794091e7410426c518a4f0cc> "[Driver] Add -print-multiarch", which causes an issue when building ExternalProject_python3 on macOS: > checking build system type... x86_64-apple-darwin19.6.0 > checking host system type... x86_64-apple-darwin19.6.0 [...] > checking for the platform triplet based on compiler characteristics... darwin configure: error: internal configure error for the platform triplet, please file a bug report > make[1]: *** [/Users/stephan/Software/lo/core/external/python3/ExternalProject_python3.mk:80: /Users/stephan/Software/lo/core/workdir/ExternalProject/python3/build] Error 1 as workdir/UnpackedTarball/python3/configure.ac computes PLATFORM_TRIPLET as "darwin", and instead of computing MULTIARCH as empty (as `$CC --print-multiarch` used to just print > clang: error: unsupported option '--print-multiarch' > clang: error: no input files to stderr), it now computes it as e.g. "x86_64-apple-darwin19.6.0" (or whatever -target is explicitly set to in $CC), so the check that they have equal values if they are bot nonempty fails now when building against Clang 13 trunk. (This is not yet an issue with any Apple Clang version, though.) Until this is eventually fixed upstream at <https://github.com/python/cpython>, just keep pretending that `clang --print-multiarch` would cause no stdout output on macOS when determining MULTIARCH. Change-Id: Ic1b27c6791b327d5709a9d61a6d675c3fa8989bf Reviewed-on: https://gerrit.libreoffice.org/c/core/+/115219 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2021-05-07external/python3: First removeunnecessarystuff, then fixinstallnamesStephan Bergmann
otherwise, we could get issues like > error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /Users/stephan/Software/lo/core/workdir/UnpackedTarball/python3/python-inst/@__________________________________________________OOO/LibreOfficePython.framework/Versions/3.8/lib/python3.8/lib-dynload/_curses_panel.cpython-3.8.so (No such file or directory) > error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /Users/stephan/Software/lo/core/workdir/UnpackedTarball/python3/python-inst/@__________________________________________________OOO/LibreOfficePython.framework/Versions/3.8/lib/python3.8/lib-dynload/_curses.cpython-3.8.so (No such file or directory) > error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /Users/stephan/Software/lo/core/workdir/UnpackedTarball/python3/python-inst/@__________________________________________________OOO/LibreOfficePython.framework/Versions/3.8/lib/python3.8/lib-dynload/_testinternalcapi.cpython-3.8.so (No such file or directory) > error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /Users/stephan/Software/lo/core/workdir/UnpackedTarball/python3/python-inst/@__________________________________________________OOO/LibreOfficePython.framework/Versions/3.8/lib/python3.8/lib-dynload/_dbm.cpython-3.8.so (No such file or directory) when the two jobs run in parallel Change-Id: I6db18d7a6fa0ce177e88f8f714434acf9afe3ea5 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/115218 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2021-05-05WASM: add initial support for Emscripten cross buildJan-Marek Glogowski
- configure with: - --host=wasm64-local-emscripten - had to make a few externals optional, so adding: - --disable-nss - --disable-cmis - --disable-curl Change-Id: I48d1c73d2675ad2e2beaf2c341578199efbd24ee Reviewed-on: https://gerrit.libreoffice.org/c/core/+/111130 Tested-by: Jenkins Reviewed-by: Thorsten Behrens <thorsten.behrens@allotropia.de>
2021-05-05Switch OPENSSL config var from negative to positiveJan-Marek Glogowski
- align with most of the rest of config_host - rename DISABLE_OPENSSL to ENABLE_OPENSSL - make this configurable Change-Id: Ic3b41fcdda38db66134939f12265e0da24833d60 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/114564 Tested-by: Jenkins Reviewed-by: Thorsten Behrens <thorsten.behrens@allotropia.de>
2021-04-21Python .so files names have changed on macOS at least for some reasonTor Lillqvist
There is no longer any "m" in the name suffix. Adapt the names of the ones we want to remove for macOS. The idlelib and tkinter dylibs are no longer there at all. Don't use the -f flag to the rm commands. Thus we will notice the next time something we want to remove isn't actually there or has been renamed. But sadly for some unknown reason we do need to use a * wildcard in the .so names. Fixes a problem caught by App Store review where the Python curses module had not been removed as it should have been. (It uses non-public APIs and can thus not be included.) Change-Id: I51b9728cc9ca0a962908d3994e3a0ff8e4fa7f60 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/114372 Tested-by: Jenkins Reviewed-by: Tor Lillqvist <tml@collabora.com>
2021-03-23Rename LO Windows arm64 ID to aarch64Jan-Marek Glogowski
The Windows platform is called Arm64. But now that the ID for Mac is also going to be renamed from arm64 to aarch64, this get's rid of the arm64 as the UNO identifier and user in gbuild, just like on all other Arm64 platforms. Change-Id: I60a7eafd04b426f17b6e41ad9a09e6405c0d4173 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/112973 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2021-01-18mac: don't put script files into Contents/MacOS or framework-bin directoryChristian Lohmaier
Signing them as executable code would require external attributes, and those in turn break packaging into hfs+ dmg when building on apfs with Big Sur. It is not a new thing - the old Code Signing in Depth technote https://developer.apple.com/library/archive/technotes/tn2206/_index.html already reads: "Store Python, Perl, shell, and other script files and other non-Mach-O executables in your app's Contents/Resources directory. While it's possible to sign such executables and store them in Contents/MacOS, this is not recommended. […] Put another way, a properly-signed app that has all of its files in the correct places will not contain any signatures stored as extended attributes." The patch does exactly that for LO and the shipped python framework and adds symlinks for the moved files. Same applies for the Language pack applescript and the tarball - those are also moved into Contents/Resources Change-Id: Iab21e77b73f941248ca89c6e80703fdf67a1057c Reviewed-on: https://gerrit.libreoffice.org/c/core/+/109537 Tested-by: Jenkins Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
2021-01-02Fix build error in Python3 for Raspberry pi4bJulien Nabet
/home/pi/lo/libreoffice/external/python3/ExternalPackage_python3.mk:48: *** file /home/pi/lo/libreoffice/workdir/UnpackedTarball/python3/LO_lib/_sysconfigdata__linux_armv7l-unknown-linux-gnueabihf.py does not exist in the tarball. Arrêt. Change-Id: If348c9f55883c1afcc37ff38e84361786bef1845 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/108595 Tested-by: Jenkins Reviewed-by: Julien Nabet <serval2412@yahoo.fr>
2020-12-03Drop Python modules we don't want on macOS, too, like on other platformsTor Lillqvist
On macOS, for various reasons, we use a different approach than on other platforms to construct the bundled Python. Instead of explicitly listing what to include (out of what Python contains and builds) (in ExternalPackage_python3.mk), after Python is built, we remove stuff we don't want (in ExternalProject_python3.mk) and then include everything left in the LibreOfficePython.framework (in GeneratedPackage_python3.mk). This fixes a problem in App Store review: For some reason the review said that the setcchar() function from the ncurses library, used by Python's curses module, is non-public. No idea why the (automated) review picked on that function. As far as I see from the ncurses header in the SDK, that function is no less public than the other ncurses functions that the Python module uses. But oh well, we don't actually ship the curses module anyway on other platforms, so just drop it on macOS, too. And while at it, drop the other unwanted ones, too. And any binary shared libraries for them. Change-Id: Idecaf10a6fb1c59e8711095927f5699b8d2ec98e Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107055 Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Tor Lillqvist <tml@collabora.com> Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107141 Tested-by: Tor Lillqvist <tml@collabora.com>
2020-12-02Guard against sysconf(_SC_OPEN_MAX) returning LONG_MAXTor Lillqvist
That seems to happen in a sandboxed process on macOS, at least. This caused an apparent hang when invoking Python, for instance simply through Tools > Macros > Run Macro... . Change-Id: I6bc055b44f298251ed596084538b98442c215fce Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107013 Tested-by: Jenkins Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-12-02external/python3: Silence UBSan errors with --with-pydebugStephan Bergmann
...that happen when building ExternalProject_python3 itself after 12142490cd43f8568ab29e0ddfa75b334d6d39d5 "Enable Python Py_DEBUG setting when built with --enable-dbgutil on Linux": For one, silence > Modules/posixmodule.c:14395:9: runtime error: left shift of 34 by 26 places cannot be represented in type 'int' > #0 in all_ins at workdir/UnpackedTarball/python3/./Modules/posixmodule.c:14395:9 where at least my kernel-headers-5.9.9-200.fc33.x86_64 /usr/include/linux/memfd.h has > #define MFD_HUGE_16GB HUGETLB_FLAG_ENCODE_16GB and /usr/include/asm-generic/hugetlb_encode.h has > #define HUGETLB_FLAG_ENCODE_16GB (34 << HUGETLB_FLAG_ENCODE_SHIFT) For another (and as predicted in 29d47d22c43e6adc1850b7db5880028dcd07d1b3 "Fix passing --disable-optimized into external/python3": "in a Linux UBsan build, making ExternalProject_python3 would have started to cause some 'applying zero offset to null pointer' failures, but which would have been easy to fix"), silence > Objects/listobject.c:551:24: runtime error: applying zero offset to null pointer > #0 in list_concat at workdir/UnpackedTarball/python3/Objects/listobject.c:551:24 Change-Id: I0523cd35e393000c8e67629a0522b2db1d8c16f0 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106984 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2020-12-01Enable Python Py_DEBUG setting when built with --enable-dbgutil on LinuxThomas Viehmann
This has been suggested on IRC for testing fixes/avoiding regressions when working on the GIL locking in PyUNO. Change-Id: Ifda21a867b3c0c7db636a9ec950050012e4742de Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106791 Reviewed-by: Michael Stahl <michael.stahl@cib.de> Tested-by: Jenkins
2020-11-19Add comment on how to run Python's own unit testsTor Lillqvist
Change-Id: Id62a688d2ddf7493d419a00a9fd0dfc6a6f13f0c Reviewed-on: https://gerrit.libreoffice.org/c/core/+/106173 Tested-by: Tor Lillqvist <tml@collabora.com> Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-11-15Make one more thing in Python compile for macOS on arm64Tor Lillqvist
Change-Id: I6716048f0b58eb502b9d1ade8a13b8e33e4aaf2b Reviewed-on: https://gerrit.libreoffice.org/c/core/+/105905 Tested-by: Jenkins Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-11-15Make python3 build on macOS 11, including for arm64Tor Lillqvist
There is no /usr/lib/libz.dylib any longer in macOS 11. No idea whether it works (especially on arm64), but that is another issue. Change-Id: I92ac0c500388730eca0be4766f07b1af2d2808e0 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/105897 Tested-by: Jenkins Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-09-17python3: fix Windows Arm64 buildJan-Marek Glogowski
Change-Id: I2e9f9ca5fcf40a3ff53c036ebc51a75b882d91f3 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/102854 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2020-09-15WIN add and apply default msbuild platform+configJan-Marek Glogowski
Adds three Windows gb_* variables: - gb_MSBUILD_CONFIG_AND_PLATFORM can be passed as msbuild flags - gb_MSBUILD_PLATFORM maps debug / release settings - gb_MSBUILD_CONFIG maps the CPUTYPE to the default msbuild names and converts the users in external projects. Change-Id: Ie9b817721180d78d104db11c44241e4b3e46bba9 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/102701 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2020-07-24Name of python.bin-gdb.py must match name of python.bin executableStephan Bergmann
...for gdb to autoload it Change-Id: I9a65a03fe18623181d5791b4596b4416228c6c8c Reviewed-on: https://gerrit.libreoffice.org/c/core/+/99372 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2020-07-17python3: update to 3.8.4Jan-Marek Glogowski
With all the prerequisites in place, LO can be updated to the current Python release. Interestingly I found that Cygwin always seems to use LC_COLLATE=C, probably because the default collation rules are missing. Then there are the changes introduced in "PEP 587 -- Python Initialization Configuration", which appearingly have modified the DLL search path behaviour on Windows, so the OpenSLL DLLs aren't found anymore in the program directory. As a workaround, the OpenSLL and libffi DLLs are now (also) installed into the Python lib dir on Windows. Change-Id: Ib82f7b77213da9c525f8c79a13d128d9eec9ca64 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/98437 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2020-07-17libffi: build DLL on WindowsJan-Marek Glogowski
The build setup is rather horrible, with some minimal gcc MSVC wrapper. But the DLL is a prerequisite for the Python 3.8 build, which dropped the internal libffi. It's also possible to build it statically, but then you have to patch the Python 3 _ctypes msbuild properties. This also defaults to explicit --build and --host settings, even without a cross build, because the predicted name would otherwise differ (*-unknown-* instead of *-pc-*). Additionally a "make install" also fails... Change-Id: Ifb7dac840e23efffb9a5e342560aef9e11e0db79 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/98436 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2020-07-17openssl: update to 1.1.1gJan-Marek Glogowski
The OpenSSL 1.1.1 release is currently the only supported version and it already has the Windows Arm64 support I was looking for. This change also explicitly enables thread support, which Python depends on since release 3.7, which removed the --without-threads build option. But the explicit OPENSSL_THREADS was just added in 3.8.4, so the old no-threads build fails now and was wrong since probably much longer. Change-Id: I61d94f966bc59407f213f4a81f0a49d0c05f8948 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/98435 Tested-by: Jenkins Reviewed-by: Jan-Marek Glogowski <glogow@fbihome.de>
2020-04-29Fix gb_ExternalProject_get_state_target,python3,removeunnecessarystuffStephan Bergmann
...which had been introduced with 0f2f719ccd5544eb37d1aacb0a50c317ae963e50 "tdf#106324: Remove unnecessary test folder from LibreOfficePython framework", but had apparently forgotten to touch the target file when done, so any make target that would (indirectly) depend on it would keep rebuilding python3 et al. Change-Id: I0ac3611383c82c4e91a1eaa02e4cf5db28d326fd Reviewed-on: https://gerrit.libreoffice.org/c/core/+/93117 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2020-04-16Fix passing --disable-optimized into external/python3Stephan Bergmann
This had originally been covered by dccf47b7f61e088622747539d1487590080da3b8 "Build python3 with debug flags if --enable-debug", which got broken by eeeec33ada5923f1f534334b22c15d6e2c6f1d35 "merge --enable-selective-debuginfo into --enable-symbols" (which removed the definition of gb_Module_CURRENTMODULE_DEBUG_ENABLED without adapting its use here). But looking again, setting OPT for workdir/UnpackedTarball/python3/configure.ac based on our various flags doesn't seem to be such a good idea anyway: It is used to specify a mixture of debuginfo (-g; which is set rather unconditionally, so no need for us to cater for --enable-symbols here), optimization (-fwrapv, -O*), and warning (-Wall) flags. So better let workdir/UnpackedTarball/python3/configure.ac keep deciding for itself what flags to set in OPT, and then just override via CFLAGS those that do not suite us. (Where it appears to be a happy coincidence that the Python build system puts CFLAGS after OPT, so the former can override the latter.) (An alternative approach could have been to pass --with-pydebug based on e.g. --enable-dbgutil, as the former (a) causes OPT to include -O0 rather than -O3, and (b) is documented to change the ABI (see workdir/UnpackedTarball/python3/configure.ac: "Py_DEBUG implies assertions, but also changes the ABI."), so probably best fits --enable-dbgutil. However, at least on Linux, --with-pydebug produces workdir/UnpackedTarball/python3/libpython3.7dm.so rather than workdir/UnpackedTarball/python3/libpython3.so, so would have required further modifications. Also, in a Linux UBsan build, making ExternalProject_python3 would have started to cause some "applying zero offset to null pointer" failures, but which would have been easy to fix.) (I noticed the missing -O0 when I attached gdb to an instdir/program/python.bin process and `py-bt` only showed "(frame information optimized out)" frames, which this change fixes for --disable-optimized builds.) Change-Id: I9583e60692ae7130377422062f3c6df9334d693f Reviewed-on: https://gerrit.libreoffice.org/c/core/+/92362 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2020-03-27revert the gyp-based nss build changesLuboš Luňák
https://lists.freedesktop.org/archives/libreoffice/2020-March/084769.html etc. This reverts commit c76fdcf1cfa1242e66b50ebe80d6eac1baae37a9. This reverts commit 10f52ab4d27263439d59f55f40e88ad2fde0cf71. This reverts commit eac806e8dcd9ee6439ac8695978ff6b62cc6b8d2. This reverts commit d591a682e46ff352f06a61c024ef661dd17f4ea4. This reverts commit 12235d3390a7fc5146bf65f9d6166034b8a048ee. This reverts commit 23245f723fb29262b8543d6447d1b0bb69cb50fb. This reverts commit 91658b402b66b67c785687d5b3a76e3183fe76bf. This reverts commit 5feadfad0cc3be2680213d2e5f6f786b2f4cc74f. This reverts commit fecca49c309fc723c524f12fa671114b316a5562. This reverts commit c6a9454e744289cf2004b42b3c90854b2db8382b. This reverts commit a1a62a70411cb6041b5930ead08280d5e1e7b5f9. This reverts commit 8512f4ca090c85477a6670438aeefe7fdfcf8a98. This reverts commit 532ffb7a297d55b495141ce33692df5d9917b54f. Change-Id: Iaa48d692bea2ca2468cdd5f8ad26ad91c0c31dde Reviewed-on: https://gerrit.libreoffice.org/c/core/+/91199 Tested-by: Jenkins Reviewed-by: Luboš Luňák <l.lunak@collabora.com>
2020-03-23build nss using their new build system (gyp/ninja-based)Luboš Luňák
This requires installed ninja, gyp is included as source. This allows nss be built as a parallel build, unlike the old Makefile build system. Since gyp internally uses python, even recursively, this requires more complicated setup in case our internal python is used. Moreover gyp itself seems to be kind of deprecated itself and hasn't been ported to python3 yet, so that needs patching too. So far only easy Unix-like systems are converted, Windows I'll do later, the more complicated systems I'll leave to whoever has access to them. Change-Id: I358baad7690d2aa6df44bafa9244dc7cc828fc3f Reviewed-on: https://gerrit.libreoffice.org/c/core/+/90115 Tested-by: Jenkins Reviewed-by: Luboš Luňák <l.lunak@collabora.com>
2020-03-04Bump Windows build baseline to Visual Studio 2019 16.4Stephan Bergmann
(where 16.4 is currently the latest version of Visual Studio 2019 available at <https://visualstudio.microsoft.com/downloads/>), see <https://lists.freedesktop.org/archives/libreoffice/2020-February/084575.html> "ESC meeting minutes: 2020-02-27": "Update baseline to VS2019 on master before 7.0 [...] check what’s the current patch level, require that? [...] no objections" The code from 4ea0059bca6dd84f10abcf52f6d6b81c1afec397 "VS detection: Fallback to old registry check if vswhere failed" has been removed in accordance with its comment "The below hack does not work for VS 2019 anyway, so should be removed when upgrading baseline. (Changing the comment "go to Start menu, open 'Visual Studio 2017', [...]" regarding the installation of GNU Make from source is somewhat arbitrary, but lets stick to the tradition of bumping that version number along with any build baseline bump.) Change-Id: Ic4fe8a3d347aa1748377f2d3205e302bff189b79 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/89699 Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com> Tested-by: Jenkins
2020-02-19tdf#130404 python3: add new windows .pyd modules for 3.7Michael Stahl
Unfortunately forgot that in b10be5d48433076f0b7238d818020f708553e114 Change-Id: I59043a576c45f9329a3fad9a5d50e7fefa901934 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/88977 Tested-by: Jenkins Reviewed-by: Michael Stahl <michael.stahl@cib.de>
2020-02-16GBUILD_TRACE, support for finding out where the build time is spentLuboš Luňák
See instructions in solenv/gbuild/Trace.mk . This generates a file than can be viewed e.g. in the Chromium tracing view. Change-Id: I5f90647c58ca729375525b6daed2d4918adc8188 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/88754 Tested-by: Jenkins Reviewed-by: Luboš Luňák <l.lunak@collabora.com>
2020-01-24python3+WIN: don't fail copy of openssl DLLs+PDBsJan-Marek Glogowski
The LO python3 target fails for me on Windows with: C:\lode\dev\core\workdir\UnpackedTarball\python3\PCBuild\openssl.props(24,5): error MSB3030: Datei "C:/lode/dev/core/workdir/UnpackedTarball/openssl/out32dll\libcrypto-1_1.dll" konnte nicht kopiert werden, da die Datei nicht gefunden wurde. [C:\lode\dev\core\workdir\UnpackedTarball\python3\PCBuild\_ssl.vcxproj] Same for "C:/lode/dev/core/workdir/UnpackedTarball/openssl/out32dll\libssl-1_1.pdb" "C:/lode/dev/core/workdir/UnpackedTarball/openssl/out32dll\libcrypto-1_1.pdb" "C:/lode/dev/core/workdir/UnpackedTarball/openssl/out32dll\libssl-1_1.dll" Other files were also renamed in a previous hunk of this patch. For other people these failures are silently ignored, but they show up in their python3 build.log. But my msbuild version 15.9.21+g9802d43bc3 from VS2017 fails hard on these errors. So this just adapt the pdb and dll names to match the previous renames, which passes the copy calls, so the build continues. Change-Id: Iffc848c334caec534f6e99f8bf83a42da570bbb1 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/87358 Reviewed-by: Michael Stahl <michael.stahl@cib.de> Tested-by: Jenkins
2020-01-09python3: bundle libffi for GNU/Linux buildsMichael Stahl
CPython commit f40d4ddff3c800b3c956a5e8820aabe3aa87cddd "Closes #27979: Remove bundled copy of libffi" causes a bit of a problem because it turns out that libffi isn't all that stable; there's libffi.so.5 on CentOS 6, libffi.so.6 on CentOS 7 and libffi.so.7 on lo_daily_update_gandalf tinderbox. So we have to bundle it in LO; it's only used on GNU/Linux currently. CPython commit 32119e10b792ad7ee4e5f951a2d89ddbaf111cc5 "bpo-35947: Update Windows to the current version of libffi (GH-11797)" also removes the libffi for MSVC, so in a future python upgrade we will have to build libffi for MSVC too. The libffi fork for MacOSX is still in CPython git master. (regression from b10be5d48433076f0b7238d818020f708553e114) Change-Id: Ibc20cf8cd3614cf9941b6970662bd930496776b2 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/86493 Tested-by: Jenkins Reviewed-by: Michael Stahl <michael.stahl@cib.de>
2020-01-09python3: fix 32-bit x86 buildMichael Stahl
The reinvented wheel needs another subst. Change-Id: I5bc01b0213f00d383cf971d34f0dc2a9e6817ab9 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/86480 Tested-by: Jenkins Reviewed-by: Michael Stahl <michael.stahl@cib.de>
2020-01-08python3: remove _uuid moduleMichael Stahl
Of course the autotetection in setup.py strikes again, the build will fail if the user doesn't have libuuid-devel installed; we'd need to add a check to LO's configure.ac for libuuid. Let's just not ship it, not sure if anybody needs it. Change-Id: I9079309da7d9c16e666fbab30542365124f97860 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/86433 Tested-by: Jenkins Reviewed-by: Michael Stahl <michael.stahl@cib.de>
2020-01-08python3: upgrade to release 3.7.6Michael Stahl
* external/python3/python-3.3.3-aix.patch.1: most of it doesn't apply and AIX port isn't maintained anyway so remove it for now * external/python3/ubsan.patch.0: apparently one of the files was removed * 0001-3.6-bpo-17239-Disable-external-entities-in-SAX-parse.patch.1: fixed upstream * python3-osx-avoid-new-10.13.patch.1: replace with simply passing ac_cv_func_utimensat=no to configure * external/python3/python-3.5.4-ssl.patch.1: project files to build OpenSSL removed upstream * There have been changes to how python locates OpenSSL; new variables OPENSSL_INCLUDES etc; it turns out that you have to pass one directory to --with-openssl, as the variables cannot be passed * libuuid.so.1 is a new dependency of the _uuid module * libffi.so.6 is a new dependency of the _ctypes module (the bundled copy of libffi for non-Darwin platforms was removed) * python-3.3.0-pythreadstate.patch.1: the PyThreadState functions have been changed such that CppunitTest_services asserts when there is a PyThreadAttach on top of PyThreadDetach on top of PyThreadAttach, i.e., 2 PyThreadState per thread (PyGILState_Check() fails). Instead of patching in additional workarounds, change PyThreadAttach so that it re-uses an existing PyThreadState if one exists for the thread. Change-Id: I24c19d79b43a30709261fd9db66312b2e3872fd9 Reviewed-on: https://gerrit.libreoffice.org/c/core/+/84765 Tested-by: Jenkins Reviewed-by: Michael Stahl <michael.stahl@cib.de>
2019-11-19python3: upgrade to release 3.5.9Michael Stahl
Fixes CVE-2019-9948 CVE-2019-9740 CVE-2019-10160 CVE-2019-16056 and expat CVE-2019-15903. python-3.3.5-pyexpat-symbols.patch.1 fails to apply, and it's a mystery why --with-system-expat is used everywhere but on MacOSX, where 292af048ace2d4b455b2da3a22c784cb05db1d09 disabled it for no obvious reason, so try to remove the special case and get rid of the patch. Change-Id: I5ba4532eb6e7c2fb90daba95d132dcc7c9013d96 Reviewed-on: https://gerrit.libreoffice.org/83117 Tested-by: Jenkins Reviewed-by: Michael Stahl <michael.stahl@cib.de>
2019-04-16Initial VS 2019 SupportTomoyuki Kubota
Change-Id: I8e08efb549ebd52c37183a1185d6de73f2b00601 Reviewed-on: https://gerrit.libreoffice.org/64630 Reviewed-by: himajin100000 <himajin100000@gmail.com> Reviewed-by: Michael Stahl <Michael.Stahl@cib.de> Tested-by: Michael Stahl <Michael.Stahl@cib.de>
2019-01-24Drop unnecessary gb_DEBUG_CFLAGSStephan Bergmann
...which was at maximum set to GCC's -finline-limit=0 -fno-inline (solenv/gbuild/platform/com_GCC_defs.mk). Those options were set for debug builds "since forever", but that looks very much like cargo cult: -fno-inline "is the default when not optimizing" anyway (<https://gcc.gnu.org/onlinedocs/gcc-7.4.0/gcc/Optimize-Options.html>), and it is unclear to me how -finline-limit=0 should have any impact beyond -fno-inline (and maybe was present for ancient compilers that only supported -finline-limit but not -fno-inline?). Change-Id: Id6752d03b1b7ec8763defabc5720d4dd08790874 Reviewed-on: https://gerrit.libreoffice.org/66836 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2019-01-16Minimum Supported Version is VS2017himajin100000
Change-Id: Id771f1fe0d8c6702a52836f6229a944d259fed4c Reviewed-on: https://gerrit.libreoffice.org/66424 Tested-by: Jenkins Reviewed-by: Michael Stahl <Michael.Stahl@cib.de>
2019-01-15python3: add patch bpo-17239: Disable external entities in SAX parserMichael Stahl
Change-Id: I44e969d8d3a8fe6b6426d61a1cbe83154c8518dd Reviewed-on: https://gerrit.libreoffice.org/66329 Tested-by: Jenkins Reviewed-by: Michael Stahl <Michael.Stahl@cib.de>
2018-12-29external/python3: Work around macOS Clang trunk errorStephan Bergmann
..."target does not support '.file' without a number", which was introduced into LLVM with <http://llvm.org/viewvc/llvm-project?view=revision&revision=349976> "[MC] Enable .file support on COFF and diagnose it on unsupported targets", stating: "The 'single parameter' .file directive appears to be an ELF-only featurea [sic] that is intended to insert the main source filename into the string table table [sic]." And <https://sourceware.org/binutils/docs-2.31/as/File.html> states about the default single (file name) argument version: "This statement may go away in future: it is only recognized to be compatible with old as programs." As external/python3 builds just fine on macOS with that .file directive removed, lets just do that for now. Change-Id: Ib28c29d0cacd151437447ccb2f6cfb8925e3e85a Reviewed-on: https://gerrit.libreoffice.org/65704 Tested-by: Jenkins Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2018-08-15tdf#106324: Remove unnecessary test folder from LibreOfficePython frameworkTor Lillqvist
No need to distribute that, apparently. We don't ship it on Windows, either. Change-Id: I76bf77311caceccd07afb0afa2f097b63f58bf54 Reviewed-on: https://gerrit.libreoffice.org/59034 Tested-by: Jenkins Reviewed-by: Tor Lillqvist <tml@collabora.com>
2018-06-05try building python in parallel on windowsNoel Grandin
Change-Id: Ib920ae6d6a3ced06ffe9f1fc4adba67d95f99a17 Reviewed-on: https://gerrit.libreoffice.org/55207 Tested-by: Jenkins <ci@libreoffice.org> Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
generated by cgit (git 2.34.1) at 2025-01-30 11:38:24 +0000