[pw-ci] [RFC v3 2/2] github_get_logs: handle multiple failed step in a single job
Salvatore Daniele
sdaniele at redhat.com
Tue Aug 3 16:59:01 UTC 2021
On Mon, Aug 2, 2021 at 5:42 PM Salvatore Daniele <sdaniele at redhat.com>
wrote:
>
>
> On Mon, Aug 2, 2021 at 5:16 PM Michael Santana <msantana at redhat.com>
> wrote:
>
>> On Thu, Jul 29, 2021 at 5:36 PM Salvatore Daniele <sdaniele at redhat.com>
>> wrote:
>> >
>> > A github job can fail multiple steps within a single job.
>> > The current implementation fails to pull logs when handed a
>> > job with multiple failed steps.
>> >
>> > This patch generates logs for each step of each job.
>> >
>> > Signed-off-by: Salvatore Daniele <sdaniele at redhat.com>
>> > ---
>> > github_get_logs.sh | 9 +++++----
>> > 1 file changed, 5 insertions(+), 4 deletions(-)
>> >
>> > diff --git a/github_get_logs.sh b/github_get_logs.sh
>> > index 2f6cc9c..1cd4086 100755
>> > --- a/github_get_logs.sh
>> > +++ b/github_get_logs.sh
>> > @@ -99,26 +99,27 @@ print_errored_logs_for_commit () {
>> > echo "-----------------------Summary of failed
>> steps-----------------------"
>> > echo "$jobs_results" | jq -r ".[].name" | while read -r job; do
>> > echo "\"$job\" failed at step \"$(echo "$jobs_results" | jq -r
>> ".[] | \
>> > - select(.name==\"$job\") | .failed_step.name")\""
>> > + select(.name==\"$job\") | .failed_step.name")\"" | sed
>> 'N;s/\n/, /'
>> jq does not have a built-in way to make a comma separated list? if not,
>> it's ok.
>>
>> I didn't look very hard when I saw that sed worked, but I will double
> check this. That would probably be cleaner.
>
> I thought sed could not replace new lines. Is that what the N flag is for?
>>
>
> Yes, it reads the next line into pattern space.
>
To follow up, WDYT of this as output for this loop?
-----------------------Summary of failed steps-----------------------
"e2e (control-plane, true, true, true, ipv4, IPv4, true, false)" failed at
step Run Tests
"e2e (control-plane, true, true, true, ipv4, IPv4, true, false)" failed at
step Generate Test Report
"e2e (control-plane, true, true, true, ipv4, IPv4, true, false)" failed at
step Third step
"other workflow" failed at step other step
----------------------End summary of failed steps--------------------
This could be achieved by the following which looks cleaner IMO.
echo "-----------------------Summary of failed steps-----------------------"
echo "$jobs_results" | jq -r ".[] | .name, .failed_step.name " | while \
read -r job && read -r step; do
echo "\"$job\" failed at step $step"
done
echo "----------------------End summary of failed steps--------------------"
My current fix does not work for more than 2 failed steps, and I could not
find a clean way to replace newlines with commas in jq. Tr only allows
replacing 1 character with 1.
Open to other thoughts. Two sed pipes could work.
>
> > done
>> > echo "----------------------End summary of failed
>> steps--------------------"
>> >
>> > echo ""
>> > echo "-------------------------------BEGIN
>> LOGS----------------------------"
>> > spacing=0
>> > +
>> > # Print out logs for failed jobs
>> > - echo "$jobs_results" | jq -r ".[].name" | while read -r job; do
>> > + echo "$jobs_results" | jq -r '.[] | .name, .failed_step.name,
>> .failed_step.number'| while \
>> > + read -r job && read -r step && read -r log_number; do
>> > +
>> Just nit-picking:
>> why switch from " to ' ?
>> add a space between ' and |
>> remove 4 empty trailing spaces
>
>
> Yes will do!
>
>
>>
>
> One question, do the arguments come out one on each line from jq? is
>> that why you had to call read separately multiple times?
>>
>
> Yes, if I recall correctly when I put them all in 1 read statement it
> would read 1 work into each (so the first 3 words of the job title).
> If I tried "while IFS="\n" read -r job read step" it would read only into
> job each time, leaving the other two variables empty.
> Perhaps there is a cleaner way to get this while loop to behave, but I was
> struggling to find it.
>
>
>> > if [ ! "$spacing" -eq "0" ]
>> > then
>> > echo -ne "\n\n\n\n"
>> > fi
>> >
>> > - step="\"$(echo "$jobs_results" | jq -r ".[] |
>> select(.name==\"$job\") | .failed_step.name")\""
>> > echo
>> "####################################################################################"
>> > echo "#### [Begin job log] \"$job\" at step $step"
>> > echo
>> "####################################################################################"
>> >
>> > - log_number=$(echo "$jobs_results" | jq ".[] |
>> select(.name==\"$job\") | .failed_step.number")
>> > cat "build_logs_series_$series_id/$job/$log_number"_* | tail
>> -n 25 | cut -d' ' -f2- | sed 's/\r$//'
>> >
>> > echo
>> "####################################################################################"
>> > --
>> > 2.31.1
>> >
>> > _______________________________________________
>> > Pwci mailing list
>> > Pwci at lists.esnacc.org
>> > http://mail.esnacc.org/mailman/listinfo/pwci
>> >
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.esnacc.org/pipermail/pwci/attachments/20210803/50d49436/attachment.html>
More information about the Pwci
mailing list