Petr Štetiar [Fri, 22 Dec 2023 10:35:23 +0000 (10:35 +0000)]
buildmaster: fix Twisted dependency hell by using twisted==22.10.0
Force twisted==22.10.0 which fixes following buildbot master startup
issue:
Unhandled error in Deferred:
Traceback (most recent call last):
File "/.../site-packages/buildbot/scripts/create_master.py", line 84, in createDB
master = BuildMaster(config['basedir'])
File "/.../site-packages/buildbot/master.py", line 102, in __init__
self._services_d = self.create_child_services()
File "/.../site-packages/twisted/internet/defer.py", line 2245, in unwindGenerator
return _cancellableInlineCallbacks(gen)
File "/.../site-packages/twisted/internet/defer.py", line 2157, in _cancellableInlineCallbacks
_inlineCallbacks(None, gen, status, _copy_context())
--- <exception caught here> ---
File "/.../site-packages/twisted/internet/defer.py", line 1997, in _inlineCallbacks
result = context.run(gen.send, result)
File "/.../site-packages/buildbot/master.py", line 188, in create_child_services
self.www = wwwservice.WWWService()
File "/.../site-packages/buildbot/www/service.py", line 196, in __init__
self.apps = get_plugins('www', None, load_now=True)
File "/.../site-packages/buildbot/plugins/db.py", line 356, in get_plugins
return _DB.add_namespace(namespace, interface, check_extras, load_now)
File "/.../site-packages/buildbot/plugins/db.py", line 306, in add_namespace
tempo.load()
File "/.../site-packages/buildbot/plugins/db.py", line 242, in load
self._tree.load()
File "/.../site-packages/buildbot/plugins/db.py", line 112, in load
child.load()
File "/.../site-packages/buildbot/plugins/db.py", line 45, in load
self._value = self._loader(self._entry)
File "/.../site-packages/buildbot/plugins/db.py", line 214, in _load_entry
raise PluginDBError('Requirements are not satisfied '
buildbot.errors.PluginDBError: Requirements are not satisfied for buildbot.www:base:
The 'zope-interface>=5' distribution was not found and is required by Twisted
This commit should be reverted once we bump to buildbot >= 3.10 which
has this workaround integrated.
References: https://github.com/buildbot/buildbot/commit/
94e2d59c23472f3fe640437630309bea518c5b9e
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Fri, 22 Dec 2023 09:46:01 +0000 (09:46 +0000)]
ci: improve QA with cram based tests
Currently we've broken master container but we're not aware about it as
current tests are very lame, so lets improve it a bit with some more
reliable, extensible solution.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Fri, 22 Dec 2023 09:28:40 +0000 (09:28 +0000)]
buildmaster: entry.sh: fix errors by checking for dir existence
Get rid of following annyoing errors during testing:
chown: cannot access '/config': No such file or directory
chown: cannot access '/certs': No such file or directory
chmod: cannot access '/config': No such file or directory
chmod: cannot access '/certs': No such file or directory
As those dirs doesn't exist in the container, they're being provided as
volumes during deployment.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Thibaut VARÈNE [Wed, 15 Nov 2023 14:12:17 +0000 (15:12 +0100)]
phase2: remove unused git_ssh plumbing
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 15 Nov 2023 12:45:18 +0000 (13:45 +0100)]
scripts: remove unused expire.sh
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 15 Nov 2023 12:44:42 +0000 (13:44 +0100)]
phase2: remove unused tree_expire option and steps
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 15 Nov 2023 11:21:41 +0000 (12:21 +0100)]
phase2: rsync: use --size-only instead of --checksum for sourceupload
Align with phase1 (
62a01a1)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 15 Nov 2023 10:50:01 +0000 (11:50 +0100)]
phase2: use sha2rsync.pl for 'targetupload'
Align with phase1 (
3246628)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 14 Nov 2023 17:58:43 +0000 (18:58 +0100)]
phase2: compute checksums
This will be necessary to get rid of 'rsync --checksum' and use
sha2rsync.pl instead, as on phase1, thereby easing the load on the rsync
server.
This uses the same construct as "make checksum" in the main repo.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 15 Nov 2023 10:50:01 +0000 (11:50 +0100)]
phase2: reduce verbosity of rsync and use sh helper
Align with phase1 (
e55b76f)
This patch:
- removes '--progress' rsync parameter
- uses the wrapper script 'rsync.sh'
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 15 Nov 2023 09:35:44 +0000 (10:35 +0100)]
phase2: don't enable rsync compression where unnecessary
Align with phase1 (
cf7b9ba)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 14 Nov 2023 11:23:11 +0000 (12:23 +0100)]
phase2: regroup common rsync options and add timeout
Align with phase1 (
3deb6c0)
"-4 -v --timeout=120"
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 14 Nov 2023 10:20:34 +0000 (11:20 +0100)]
phase2: report ccache stats at end of build
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 13 Nov 2023 16:32:34 +0000 (17:32 +0100)]
Revert "phase2: use full git history for reproducibility"
Following openwrt/
11bb5337b8d8b5018e48f0df415efb99e2f49d0d we no longer
need the full git history.
This reverts commit
5b96616d056e26adbd50cc73a5e51b8449110b7d.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Sat, 11 Nov 2023 16:26:41 +0000 (17:26 +0100)]
phase2: use Interpolate instead of WithProperties
Align with phase1 (
d1a92ba)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Sat, 11 Nov 2023 15:28:07 +0000 (16:28 +0100)]
phase2: s/SetProperty/SetPropertyFromCommand/
Align with phase1 (
3141c97)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Sat, 11 Nov 2023 15:12:37 +0000 (16:12 +0100)]
phase2: do not exceed nproc build concurrency
Align with phase1 (
2ad0478)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Sat, 11 Nov 2023 15:11:25 +0000 (16:11 +0100)]
phase2: remove unused 'other_builds' property
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 14 Nov 2023 13:23:50 +0000 (14:23 +0100)]
phase1: treat all branches equally for building
Following discussion here:
https://lists.openwrt.org/pipermail/openwrt-devel/2023-November/041769.html
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 13 Nov 2023 14:23:49 +0000 (15:23 +0100)]
phase1: raise priority of tag builds
Currently the buildmaster would only order tag builds within their own
branch, meaning that if e.g. a higher priority branch has normal
buildrequests (i.e. buildrequests comming from the AnyBranchScheduler),
and a lower priority branch has "tag" buildrequests (i.e. from the
Triggerable scheduler), the former builderequests would be served first,
deferring the build of e.g. release tags.
We want forced builds (release tag) to have maximum priority, regardless
of branch priority. This commit attempts to solve this problem by
leveraging the newly (as of buildbot 3.9.0) introduced "priority"
scheduler parameter, by raising the Triggerable scheduler buildrequests
priority, and then considering this higher priority in Builders' order.
The net result is that Builders are now prioritized if they have
pending higher priority buildrequest, still preserving the branch order.
In other words, tag requests are front run while preserving branch order,
meaning that if two branches have tag buildrequests, the higher priority
branch is still served first.
This requires buildbot 3.9.0
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Christian Marangi [Tue, 14 Nov 2023 13:53:12 +0000 (14:53 +0100)]
ci: split container push steps to separate job and add deploy tag
Split container push related steps to separate jobs and add deploy tag.
This is to better organize the workflow and drop additional checks for
single steps moving them to the single job.
Also we use a feature of github to better track changes deployed to our
buildbot.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Christian Marangi [Tue, 14 Nov 2023 13:49:37 +0000 (14:49 +0100)]
ci: generalize container test step
Generalize container test step by using include feature of matrix
strategy and defining additional values for container command test and
config verification.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Christian Marangi [Tue, 14 Nov 2023 13:47:26 +0000 (14:47 +0100)]
ci: move git short sha length to ENV
Move git short sha length to ENV to make it easier to configure in the
future if needed.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Paul Spooren [Tue, 14 Nov 2023 11:27:08 +0000 (12:27 +0100)]
docker,worker: install zstd
Faster compression with partly better rates. Should be used eventually
for ImageBuilders etc.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Ben Whitten [Thu, 27 Jul 2023 20:25:56 +0000 (21:25 +0100)]
docker,worker: install pyelftools
UBoot version of at least 2023.07.02 that use binman also have a dependency of pyelftools on the build host.
Signed-off-by: Ben Whitten <BWhitten@users.noreply.github.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz> [whitespace fix]
Petr Štetiar [Sat, 22 Jul 2023 08:53:45 +0000 (10:53 +0200)]
phase1: workaround dlprune recursive directory removal
Workarounds following issue:
find: cannot delete ‘dl/ath10k-ct-firmware-2020-11-08’: Directory not empty
References: #13
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Sun, 18 Jun 2023 05:32:22 +0000 (07:32 +0200)]
phase1: prepareFactory: use variables instead of list index
Thus make the code more readable.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Sun, 18 Jun 2023 05:28:01 +0000 (07:28 +0200)]
phase1: populateTargets: log branch name as well
So the progress is more verbose.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Sun, 18 Jun 2023 05:26:17 +0000 (07:26 +0200)]
phase1: do not leak targets between branches
Robert noticed, that after rename of `ipq807x` target in main branch to
`qualcommax/ipq807x` subtarget, that buildbot is trying to build this
new `qualcommax/ipq807x` subtarget on `openwrt-23.05` branch as well.
Thibaut later explained, that this is by design, his initial idea was to
find exhaustive list of all targets and let the `checkarch` step do the
final triaging.
I find this approach confusing, because if the subtarget is not present
in that branch, we shouldn't have builder for it configured as well.
Furthermore wasting roughly 5 minutes of precious buildworker time to
checkout all feeds and then just find out, that we're not going to use
those seems suboptimal as well.
So lets fix it by using builders for targets as present in respective
branches.
Fixes: #14
Reported-by: Robert Marko <robimarko@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Sun, 18 Jun 2023 05:19:32 +0000 (07:19 +0200)]
phase1: reformat with black
Making everything consistent, more readable, CI guarded.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Thu, 1 Jun 2023 05:56:49 +0000 (07:56 +0200)]
phase1: dlprune: fix cannot delete ‘dl/’: Not a directory
dlprune currently fails with following error:
find: cannot delete ‘dl/’: Not a directory
Initial idea was to delete only stuff under download directory and not
the download directory itself, so lets adjust the find command
accordingly.
Fixes: 68b20ed67b5e ("phase1: prune unused files from dl/")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Thibaut VARÈNE [Tue, 23 May 2023 18:12:09 +0000 (20:12 +0200)]
phase1: satisfy getver.sh requirements
scripts/getver.sh requires upstream branch tracking to correctly compute
revision number. Buildbot Git() step does not set branch tracking as it
checks out individual commits and then reset the target branch to it, so
in order for getver.sh to work, this workaround forcefully sets the
upstream branch.
This fixes malformed version code in filenames as observed on
openwrt-23.05:
r23001+3-
38c150612c
instead of:
r23004-
7f0db09513
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 22 May 2023 18:20:12 +0000 (20:20 +0200)]
phase1: cleanup redundant steps
These 3 rm steps are unnecessary since these folders are already cleaned
by the Git() step. Thus reduce clutter in already pretty busy Factory.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Petr Štetiar [Tue, 16 May 2023 18:41:05 +0000 (20:41 +0200)]
phase1: workaround gitverify 1st build failures on fresh workers
Currently every 1st build on ephemeral build worker results in the
failure of the `gitverify` build step as the stock buildbot Git() build
step for some reasons just clones the Git repository into detached HEAD
state.
On the 2nd build using the same build worker Git() checkouts the branch
and thus makes `gitverify` step happy and the builds then continues
normally.
This needs to be fixed properly, either by adjusting the `gitverify`
check or adding suitable `mode` into Git() build step, but this needs
more time and testing.
So for now, lets simply workaround that issue by running Git() step two
times.
References: #5
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Alois Klink [Wed, 23 Nov 2022 17:23:48 +0000 (17:23 +0000)]
docker,worker: install g++-multilib
From [openwrt/docker@
5484951][1] (GitHub PR [#89][2]):
Node fails to cross-compile from a 64-bit build machine to 32-bit host
with the following error:
```
In file included from /usr/include/c++/8/memory:62,
from ../deps/v8/src/libplatform/default-foreground-task-runner.h:8,
from ../deps/v8/src/libplatform/default-foreground-task-runner.cc:5:
/usr/include/c++/8/bits/stl_algobase.h:59:10: fatal error: bits/c++config.h:
No such file or directory
#include <bits/c++config.h>
^~~~~~~~~~~~~~~~~~
compilation terminated.
```
On Debian, `g++-multilib` can be installed to fix this.
[1]: https://gitlab.com/openwrt/docker/-/commit/
54849510d7802028b94757051cca6d004a9ca1d1
[2]: https://github.com/openwrt/docker/pull/89
Fixes: https://github.com/openwrt/packages/issues/18476
Fixes: https://forum.openwrt.org/t/why-arent-the-node-and-node-npm-packages-available-on-arm-cortex-a9-vfpv3-d16-in-22-03-2/142722
Signed-off-by: Alois Klink <alois@aloisklink.com>
Petr Štetiar [Tue, 16 May 2023 10:57:56 +0000 (12:57 +0200)]
ci: add basic config checking with ruff
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Thibaut VARÈNE [Mon, 15 May 2023 20:08:50 +0000 (22:08 +0200)]
phase1: actually make rsync -4 configurable
54e80d5ce introduced an interpolate bug that would evaluate an empty
string as being part of the arguments passed to rsync, resulting in a
failure as rsync interprets that empty string as a source.
This commit fixes this by instead appending '4' to regular shortopts
when necessary.
Fixes: 54e80d5ce
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Thibaut VARÈNE [Tue, 16 May 2023 08:13:11 +0000 (10:13 +0200)]
phase1: perform kmodupload only when needed
Fixes: 909e899
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Mon, 15 May 2023 14:18:16 +0000 (16:18 +0200)]
phase1: fix crash on exception during Git update
Fixes following exception:
2023-05-15 14:08:56+0000 [-] <Build master_apm821xx/nand number:1 results:success> build got exception when running step Git(name='git', repourl='https://git.openwrt.org/openwrt/openwrt.git', mode='full', method='fresh', locks=renderer(<function NetLockDl at 0x7f5d33351ee0>), haltOnFailure=True)
2023-05-15 14:08:56+0000 [-] Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/twisted/internet/defer.py", line 1792, in gotResult
_inlineCallbacks(r, gen, status, context)
File "/usr/local/lib/python3.9/dist-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks
result = context.run(
File "/usr/local/lib/python3.9/dist-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/local/lib/python3.9/dist-packages/buildbot/process/build.py", line 576, in _start_next_step_impl
log.err(e)
--- <exception caught here> ---
File "/usr/local/lib/python3.9/dist-packages/buildbot/process/build.py", line 562, in _start_next_step_impl
results = yield step.startStep(self.conn)
builtins.KeyError: None
as None is valid property value, but cant be used as dict index.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Mon, 15 May 2023 07:25:07 +0000 (09:25 +0200)]
ci: allow pushing of containers during pull requests
Make it possible to push containers in pull requests in private forks,
so the container can be actually runtime tested before merging the
changes.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Mon, 15 May 2023 07:14:48 +0000 (09:14 +0200)]
buildworker,buildmaster: bump Debian to version 11
Debian 10 LTS support ends on 6/2024, so it makes no sense to use it as
a base for 23.05 release, so lets switch to Debian 11 which should've
LTS support till 6/2026.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Mon, 15 May 2023 07:12:16 +0000 (09:12 +0200)]
Bump buildbot version to latest stable release 3.8.0
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Petr Štetiar [Mon, 15 May 2023 06:53:58 +0000 (08:53 +0200)]
buildmaster: fix unpinned buildbot package version
It seems, that current pip3 install pinning doesnt work properly as it
can result in the following package versions installation:
buildbot-3.6.1 buildbot-console-view-3.5.0 buildbot-grid-view-3.5.0
buildbot-waterfall-view-3.5.0 buildbot-worker-3.5.0 buildbot-www-3.5.0
So lets pin the buildbot package itself to BUILDBOT_VERSION as well.
References: https://gitlab.com/openwrt/buildbot/-/jobs/
3324049571#L1158
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Thibaut VARÈNE [Tue, 15 Nov 2022 09:44:56 +0000 (10:44 +0100)]
phase1: make 'rsync -4' worker-configurable
We set '-4' in rsync_defopts, asking rsync to "prefer" ipv4 whenever
possible on _all_ workers. This was historically done because some
workers had flaky ipv6 connectivity, however in the future the reverse
may be true, with worker having worse networking over ipv4 than ipv6.
This change introduces an 'rsync_ipv4' worker configuration option:
when set to a true value, the old rsync behavior (adding '-4') is used,
when unset or set to false, the extra rsync argument is not used.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Fri, 28 Oct 2022 09:05:21 +0000 (11:05 +0200)]
phase1: call host ccache in stats
We do not use the staging_dir ccache on buildbots (config seed contains
CONFIG_CCACHE=n), only the host tool if present.
Also only execute this step if 'ccache_command' is set.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 20:35:09 +0000 (22:35 +0200)]
phase1: tag builders with their intended branch name
This helps sorting them in waterfall and builders views.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 18:31:19 +0000 (20:31 +0200)]
phase1: documentation update
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 12:56:21 +0000 (14:56 +0200)]
phase1: allow restricting builders to tag builds only
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 12:32:30 +0000 (14:32 +0200)]
phase1: refactor worker parsing
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 12:11:13 +0000 (14:11 +0200)]
phase1: disable debug messages
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 11:27:56 +0000 (13:27 +0200)]
phase1: prioritize builders by branch order
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 11:09:17 +0000 (13:09 +0200)]
phase1: switch to separate builders per branch per target
This will simplify reporting and ensure consistent build statistics.
It will also simplify prioritizing builders by branch.
This also introduces a single "trigger" scheduler that will handle all
trigger requests for each branch/target combination, speeding up the
forcing of builds.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 27 Oct 2022 08:20:39 +0000 (10:20 +0200)]
phase1: fix ForceBuild validation logic
We shouldn't use the local GitPoller repo to find the commit hash of a
tag, since this repo may not yet have been updated at the time the new
tag value is fetched from the remote repo.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 26 Oct 2022 20:53:12 +0000 (22:53 +0200)]
phase1: run ForceBuilder on local worker
There is no reason to send this over the network to some remote worker,
incurring latency for a purely housekeeping task that should be done on
the buildmaster host.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 26 Oct 2022 19:35:18 +0000 (21:35 +0200)]
phase1: prune unused files from dl/
Remove files that haven't been accessed in >15 days from dl/
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 26 Oct 2022 19:34:19 +0000 (21:34 +0200)]
phase1: workers are running a single build
Worker lock on dl/ is now unnecessary
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 26 Oct 2022 18:51:20 +0000 (20:51 +0200)]
phase1: Fix the force scheduler process
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 26 Oct 2022 17:30:25 +0000 (19:30 +0200)]
phase1: always front-run the 00_force_build builder
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 26 Oct 2022 12:43:15 +0000 (14:43 +0200)]
phase1: fix ForceScheduler
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 26 Oct 2022 07:46:22 +0000 (09:46 +0200)]
phase1: don't log env on statistics steps
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 25 Oct 2022 20:09:55 +0000 (22:09 +0200)]
phase1: consistent step styling
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 25 Oct 2022 19:58:53 +0000 (21:58 +0200)]
phase1: gitcheckout workaround is no longer needed
And it has adverse effects (slows/breaks Git step).
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 25 Oct 2022 18:41:21 +0000 (20:41 +0200)]
phase1: move shared dl to same folder as shared-workdir
The previous command moved it to $HOME/dl, which could pollute the user's
home folder during test runs. This brings the "shared dl" folder in line
with the shared workdir folder, all within buildbot folder hierarchy.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 25 Oct 2022 17:48:46 +0000 (19:48 +0200)]
phase1: perform rsync steps only if configured
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 25 Oct 2022 17:25:39 +0000 (19:25 +0200)]
phase1: adjust steps descriptions
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 25 Oct 2022 13:41:06 +0000 (15:41 +0200)]
phase1: hide/skip ccache steps if not available
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Tue, 25 Oct 2022 11:08:50 +0000 (13:08 +0200)]
phase1: FlattenList doesn't work on renderables
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 16:17:46 +0000 (18:17 +0200)]
phase1: update config.ini.example
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 16:03:26 +0000 (18:03 +0200)]
phase1: populateConfig(): quiet git clone
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 15:09:43 +0000 (17:09 +0200)]
phase1: make scheduler and changesource branch-aware
The ForcedScheduler for tags still needs work
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 14:56:45 +0000 (16:56 +0200)]
phase1: branch-aware signall.sh
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 14:56:37 +0000 (16:56 +0200)]
scripts/signall.sh: make branch aware
This commit is a NO-OP if signall.sh is called with a single argument,
as is currently done, and will allow fetching branch-specific signing
credentials if a branch name is passed as second argument.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 14:41:00 +0000 (16:41 +0200)]
phase1: abort early if current builder does not apply
Some builders (target/subtarget) may only apply to particular branches,
but are nevertheless defined for all branches. Detect such case early,
stop the build but do not mark it FAILURE.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 14:17:03 +0000 (16:17 +0200)]
phase1: make populateTargets() branch aware
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 14:03:38 +0000 (16:03 +0200)]
phase1: make GetNextBuild() branch aware
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 16:08:23 +0000 (18:08 +0200)]
phase1: use single var for rsync_bin_defopts and rsync_src_defopts
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 12:41:28 +0000 (14:41 +0200)]
phase1: use branch config in factory
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Mon, 24 Oct 2022 10:48:11 +0000 (12:48 +0200)]
phase1: rsync_src_url is always set
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Fri, 21 Oct 2022 16:54:10 +0000 (18:54 +0200)]
phase1: prepare to parse branches
This commit breaks config
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Fri, 21 Oct 2022 16:38:58 +0000 (18:38 +0200)]
phase1: Git() no need to specify branch
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Fri, 21 Oct 2022 16:25:03 +0000 (18:25 +0200)]
phase1: replace 'repo_branch' with 'branch' prop in factory
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Fri, 21 Oct 2022 09:16:42 +0000 (11:16 +0200)]
phase1: refactor populateTargets()
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 17:41:57 +0000 (19:41 +0200)]
phase1: move target enumeration to function
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 17:24:56 +0000 (19:24 +0200)]
phase1: reorder/regroup globals
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 14:06:54 +0000 (16:06 +0200)]
phase1: fix option parsing typo for 'ul_lock'
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 13:51:38 +0000 (15:51 +0200)]
phase1: remove 'cleanup' worker option (always set)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 13:49:43 +0000 (15:49 +0200)]
phase1: remove 'shared_wd' worker option (always set)
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 13:48:04 +0000 (15:48 +0200)]
phase1: remove dead code due to shared_wd=1
shared_wd is always set, the conditional on these bits were thus always
the same resulting in dead code.
Remove 'expire' config option which is no longer needed.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 18:16:10 +0000 (20:16 +0200)]
phase1: remove now redundant 'njobs' (== 'nprocs') property
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 13:43:15 +0000 (15:43 +0200)]
phase1: max_builds and shared_wd are always set to 1
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 13:30:45 +0000 (15:30 +0200)]
phase1: rework GetNumJobs()
Ensure we never end up with "0".
Set a build property step with the computed concurrency level, so that
this information is available in build status.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 11:18:25 +0000 (13:18 +0200)]
phase1: remove GetCCache()
ccache_command is set from "which ccache" which is either empty or
contains "ccache". Use a simple interpolation instead.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 11:11:26 +0000 (13:11 +0200)]
phase1: remove unused IsNoTaggingRequested()
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 10:47:05 +0000 (12:47 +0200)]
phase1: deduplicate 'git' step
method renderers are supported now.
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 10:34:48 +0000 (12:34 +0200)]
phase1: remove dead step CleanTargetMap
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 09:37:21 +0000 (11:37 +0200)]
phase1: remove GetCC()/GetCXX()
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Thu, 20 Oct 2022 09:33:10 +0000 (11:33 +0200)]
phase1: remove GetCwd()
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 19 Oct 2022 18:02:51 +0000 (20:02 +0200)]
phase1: buildfactory: remove deadcode
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 19 Oct 2022 17:57:18 +0000 (19:57 +0200)]
phase1: remove checkBuiltin deadcode
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>
Thibaut VARÈNE [Wed, 19 Oct 2022 17:53:54 +0000 (19:53 +0200)]
phase1: remove unused variables
Signed-off-by: Thibaut VARÈNE <hacks@slashdirt.org>