<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 3, 2021 at 2:39 PM Michael Santana <<a href="mailto:msantana@redhat.com">msantana@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, Aug 3, 2021 at 12:59 PM Salvatore Daniele <<a href="mailto:sdaniele@redhat.com" target="_blank">sdaniele@redhat.com</a>> wrote:<br>
><br>
><br>
><br>
> On Mon, Aug 2, 2021 at 5:42 PM Salvatore Daniele <<a href="mailto:sdaniele@redhat.com" target="_blank">sdaniele@redhat.com</a>> wrote:<br>
>><br>
>><br>
>><br>
>> On Mon, Aug 2, 2021 at 5:16 PM Michael Santana <<a href="mailto:msantana@redhat.com" target="_blank">msantana@redhat.com</a>> wrote:<br>
>>><br>
>>> On Thu, Jul 29, 2021 at 5:36 PM Salvatore Daniele <<a href="mailto:sdaniele@redhat.com" target="_blank">sdaniele@redhat.com</a>> wrote:<br>
>>> ><br>
>>> > A github job can fail multiple steps within a single job.<br>
>>> > The current implementation fails to pull logs when handed a<br>
>>> > job with multiple failed steps.<br>
>>> ><br>
>>> > This patch generates logs for each step of each job.<br>
>>> ><br>
>>> > Signed-off-by: Salvatore Daniele <<a href="mailto:sdaniele@redhat.com" target="_blank">sdaniele@redhat.com</a>><br>
>>> > ---<br>
>>> > github_get_logs.sh | 9 +++++----<br>
>>> > 1 file changed, 5 insertions(+), 4 deletions(-)<br>
>>> ><br>
>>> > diff --git a/github_get_logs.sh b/github_get_logs.sh<br>
>>> > index 2f6cc9c..1cd4086 100755<br>
>>> > --- a/github_get_logs.sh<br>
>>> > +++ b/github_get_logs.sh<br>
>>> > @@ -99,26 +99,27 @@ print_errored_logs_for_commit () {<br>
>>> > echo "-----------------------Summary of failed steps-----------------------"<br>
>>> > echo "$jobs_results" | jq -r ".[].name" | while read -r job; do<br>
>>> > echo "\"$job\" failed at step \"$(echo "$jobs_results" | jq -r ".[] | \<br>
>>> > - select(.name==\"$job\") | .<a href="http://failed_step.name" rel="noreferrer" target="_blank">failed_step.name</a>")\""<br>
>>> > + select(.name==\"$job\") | .<a href="http://failed_step.name" rel="noreferrer" target="_blank">failed_step.name</a>")\"" | sed 'N;s/\n/, /'<br>
>>> jq does not have a built-in way to make a comma separated list? if not, it's ok.<br>
>>><br>
>> I didn't look very hard when I saw that sed worked, but I will double check this. That would probably be cleaner.<br>
>><br>
>>> I thought sed could not replace new lines. Is that what the N flag is for?<br>
>><br>
>><br>
>> Yes, it reads the next line into pattern space.<br>
><br>
><br>
> To follow up, WDYT of this as output for this loop?<br>
><br>
> -----------------------Summary of failed steps-----------------------<br>
> "e2e (control-plane, true, true, true, ipv4, IPv4, true, false)" failed at step Run Tests<br>
> "e2e (control-plane, true, true, true, ipv4, IPv4, true, false)" failed at step Generate Test Report<br>
> "e2e (control-plane, true, true, true, ipv4, IPv4, true, false)" failed at step Third step<br>
> "other workflow" failed at step other step<br>
> ----------------------End summary of failed steps--------------------<br>
><br>
> This could be achieved by the following which looks cleaner IMO.<br>
><br>
> echo "-----------------------Summary of failed steps-----------------------"<br>
> echo "$jobs_results" | jq -r ".[] | .name, .<a href="http://failed_step.name" rel="noreferrer" target="_blank">failed_step.name</a> " | while \<br>
> read -r job && read -r step; do<br>
> echo "\"$job\" failed at step $step"<br>
> done<br>
> echo "----------------------End summary of failed steps--------------------"<br>
><br>
> My current fix does not work for more than 2 failed steps, and I could not find a clean way to replace newlines with commas in jq. Tr only allows replacing 1 character with 1.<br>
So wait, what's the issue now? I thought you got it working<br></blockquote><div><br></div><div>I realized my fix works for 1 or 2 failed steps, but will not work for 3 or more. </div><div>I wanted to get your thoughts on whether printing failed steps 1 per line would be acceptable, which would work for any number of failed steps / workflow.</div><div><br></div><div>If you would prefer a comma separated list that is fine w/ me. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
><br>
> Open to other thoughts. Two sed pipes could work.<br>
><br>
>><br>
>><br>
>>> > done<br>
>>> > echo "----------------------End summary of failed steps--------------------"<br>
>>> ><br>
>>> > echo ""<br>
>>> > echo "-------------------------------BEGIN LOGS----------------------------"<br>
>>> > spacing=0<br>
>>> > +<br>
>>> > # Print out logs for failed jobs<br>
>>> > - echo "$jobs_results" | jq -r ".[].name" | while read -r job; do<br>
>>> > + echo "$jobs_results" | jq -r '.[] | .name, .<a href="http://failed_step.name" rel="noreferrer" target="_blank">failed_step.name</a>, .failed_step.number'| while \<br>
>>> > + read -r job && read -r step && read -r log_number; do<br>
>>> > +<br>
>>> Just nit-picking:<br>
>>> why switch from " to ' ?<br>
>>> add a space between ' and |<br>
>>> remove 4 empty trailing spaces<br>
>><br>
>><br>
>> Yes will do!<br>
>><br>
>>><br>
>>><br>
>>><br>
>>> One question, do the arguments come out one on each line from jq? is<br>
>>> that why you had to call read separately multiple times?<br>
>><br>
>><br>
>> Yes, if I recall correctly when I put them all in 1 read statement it would read 1 work into each (so the first 3 words of the job title).<br>
>> If I tried "while IFS="\n" read -r job read step" it would read only into job each time, leaving the other two variables empty.<br>
>> Perhaps there is a cleaner way to get this while loop to behave, but I was struggling to find it.<br>
>><br>
>>><br>
>>> > if [ ! "$spacing" -eq "0" ]<br>
>>> > then<br>
>>> > echo -ne "\n\n\n\n"<br>
>>> > fi<br>
>>> ><br>
>>> > - step="\"$(echo "$jobs_results" | jq -r ".[] | select(.name==\"$job\") | .<a href="http://failed_step.name" rel="noreferrer" target="_blank">failed_step.name</a>")\""<br>
>>> > echo "####################################################################################"<br>
>>> > echo "#### [Begin job log] \"$job\" at step $step"<br>
>>> > echo "####################################################################################"<br>
>>> ><br>
>>> > - log_number=$(echo "$jobs_results" | jq ".[] | select(.name==\"$job\") | .failed_step.number")<br>
>>> > cat "build_logs_series_$series_id/$job/$log_number"_* | tail -n 25 | cut -d' ' -f2- | sed 's/\r$//'<br>
>>> ><br>
>>> > echo "####################################################################################"<br>
>>> > --<br>
>>> > 2.31.1<br>
>>> ><br>
>>> > _______________________________________________<br>
>>> > Pwci mailing list<br>
>>> > <a href="mailto:Pwci@lists.esnacc.org" target="_blank">Pwci@lists.esnacc.org</a><br>
>>> > <a href="http://mail.esnacc.org/mailman/listinfo/pwci" rel="noreferrer" target="_blank">http://mail.esnacc.org/mailman/listinfo/pwci</a><br>
>>> ><br>
>>><br>
<br>
</blockquote></div></div>