[configs/releng] Add sha256sums for TianoCore efi downloads

We should be integrity checking these downloads.

This will also aid in future reproducibility efforts as the build will bomb
out early in case of failure.

Signed-off-by: Daniel Edgecumbe <git@esotericnonsense.com>
---
 configs/releng/build.sh | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

On 9/4/19 11:16 PM, Daniel Edgecumbe wrote:
> We should be integrity checking these downloads.
> 
> This will also aid in future reproducibility efforts as the build will bomb
> out early in case of failure.
> 
> Signed-off-by: Daniel Edgecumbe <git@esotericnonsense.com>
> ---
>  configs/releng/build.sh | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/configs/releng/build.sh b/configs/releng/build.sh
> index 659e8de..857e01d 100755
> --- a/configs/releng/build.sh
> +++ b/configs/releng/build.sh
> @@ -168,9 +168,14 @@ make_efi() {
>          ${script_path}/efiboot/loader/entries/archiso-x86_64-usb.conf > ${work_dir}/iso/loader/entries/archiso-x86_64.conf
>  
>      # EFI Shell 2.0 for UEFI 2.3+
> -    curl -o ${work_dir}/iso/EFI/shellx64_v2.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/ShellBinPkg/UefiShell/X64/Shell.efi
> +    echo "Downloading shellx64_v2.efi..."
> +    curl -sSo ${work_dir}/iso/EFI/shellx64_v2.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/ShellBinPkg/UefiShell/X64/Shell.efi
> +    echo "04c89f19efee2a22660fd4650ff9add88e962d102b1b713e535f4e32a07c5185 ${work_dir}/iso/EFI/shellx64_v2.efi" | sha256sum -c > /dev/null
> +
>      # EFI Shell 1.0 for non UEFI 2.3+
> -    curl -o ${work_dir}/iso/EFI/shellx64_v1.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/EdkShellBinPkg/FullShell/X64/Shell_Full.efi
> +    echo "Downloading shellx64_v1.efi..."
> +    curl -sSo ${work_dir}/iso/EFI/shellx64_v1.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/EdkShellBinPkg/FullShell/X64/Shell_Full.efi
> +    echo "ea5e763a8a5f9733dbf7c33ffa16a19e078c6af635b51d8457bc377a22106a8c ${work_dir}/iso/EFI/shellx64_v1.efi" | sha256sum -c > /dev/null
>  }
>  
>  # Prepare efiboot.img::/EFI for "El Torito" EFI boot mode

+1, this seems a lot more reasonable. Although I wonder if maybe it
would make sense to build it from source ourselves, possibly as a pacman
package.

I'm not so sure about building from source, that may make sense, but I do think
that having history beyond 'here is the hash, it was this file' is useful.

There are some other parts of the releng build that I think should probably either be
packages (and therefore mirrored, and therefore archived or at least have reasonable
ways to determine the history of) or at least not simply be pulled in at build time
like that.

The pacman mirror list for example is pulled in from the generator at build time, this
can probably be replaced by using the pacman-mirrorlist package.

Daniel

On 05/09/2019 05.25, Eli Schwartz via arch-releng wrote:
> On 9/4/19 11:16 PM, Daniel Edgecumbe wrote:
>> We should be integrity checking these downloads.
>>
>> This will also aid in future reproducibility efforts as the build will bomb
>> out early in case of failure.
>>
>> Signed-off-by: Daniel Edgecumbe <git@esotericnonsense.com>
>> ---
>>  configs/releng/build.sh | 9 +++++++--
>>  1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/configs/releng/build.sh b/configs/releng/build.sh
>> index 659e8de..857e01d 100755
>> --- a/configs/releng/build.sh
>> +++ b/configs/releng/build.sh
>> @@ -168,9 +168,14 @@ make_efi() {
>>          ${script_path}/efiboot/loader/entries/archiso-x86_64-usb.conf > ${work_dir}/iso/loader/entries/archiso-x86_64.conf
>>  
>>      # EFI Shell 2.0 for UEFI 2.3+
>> -    curl -o ${work_dir}/iso/EFI/shellx64_v2.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/ShellBinPkg/UefiShell/X64/Shell.efi
>> +    echo "Downloading shellx64_v2.efi..."
>> +    curl -sSo ${work_dir}/iso/EFI/shellx64_v2.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/ShellBinPkg/UefiShell/X64/Shell.efi
>> +    echo "04c89f19efee2a22660fd4650ff9add88e962d102b1b713e535f4e32a07c5185 ${work_dir}/iso/EFI/shellx64_v2.efi" | sha256sum -c > /dev/null
>> +
>>      # EFI Shell 1.0 for non UEFI 2.3+
>> -    curl -o ${work_dir}/iso/EFI/shellx64_v1.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/EdkShellBinPkg/FullShell/X64/Shell_Full.efi
>> +    echo "Downloading shellx64_v1.efi..."
>> +    curl -sSo ${work_dir}/iso/EFI/shellx64_v1.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/EdkShellBinPkg/FullShell/X64/Shell_Full.efi
>> +    echo "ea5e763a8a5f9733dbf7c33ffa16a19e078c6af635b51d8457bc377a22106a8c ${work_dir}/iso/EFI/shellx64_v1.efi" | sha256sum -c > /dev/null
>>  }
>>  
>>  # Prepare efiboot.img::/EFI for "El Torito" EFI boot mode
> 
> +1, this seems a lot more reasonable. Although I wonder if maybe it
> would make sense to build it from source ourselves, possibly as a pacman
> package.
>

On 2019-09-05 04:16:34 (+0100), Daniel Edgecumbe wrote:
> We should be integrity checking these downloads.
> 
> This will also aid in future reproducibility efforts as the build will bomb
> out early in case of failure.
> 
> Signed-off-by: Daniel Edgecumbe <git@esotericnonsense.com>
> ---
>  configs/releng/build.sh | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/configs/releng/build.sh b/configs/releng/build.sh
> index 659e8de..857e01d 100755
> --- a/configs/releng/build.sh
> +++ b/configs/releng/build.sh
> @@ -168,9 +168,14 @@ make_efi() {
>          ${script_path}/efiboot/loader/entries/archiso-x86_64-usb.conf > ${work_dir}/iso/loader/entries/archiso-x86_64.conf
>  
>      # EFI Shell 2.0 for UEFI 2.3+
> -    curl -o ${work_dir}/iso/EFI/shellx64_v2.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/ShellBinPkg/UefiShell/X64/Shell.efi
> +    echo "Downloading shellx64_v2.efi..."
> +    curl -sSo ${work_dir}/iso/EFI/shellx64_v2.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/ShellBinPkg/UefiShell/X64/Shell.efi
> +    echo "04c89f19efee2a22660fd4650ff9add88e962d102b1b713e535f4e32a07c5185 ${work_dir}/iso/EFI/shellx64_v2.efi" | sha256sum -c > /dev/null
> +
>      # EFI Shell 1.0 for non UEFI 2.3+
> -    curl -o ${work_dir}/iso/EFI/shellx64_v1.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/EdkShellBinPkg/FullShell/X64/Shell_Full.efi
> +    echo "Downloading shellx64_v1.efi..."
> +    curl -sSo ${work_dir}/iso/EFI/shellx64_v1.efi https://raw.githubusercontent.com/tianocore/edk2/UDK2018/EdkShellBinPkg/FullShell/X64/Shell_Full.efi
> +    echo "ea5e763a8a5f9733dbf7c33ffa16a19e078c6af635b51d8457bc377a22106a8c ${work_dir}/iso/EFI/shellx64_v1.efi" | sha256sum -c > /dev/null
>  }
>  
>  # Prepare efiboot.img::/EFI for "El Torito" EFI boot mode
> -- 
> 2.23.0

Thanks for the suggested patch! I'm currently building the edk2
Shell.efi and Shell_Full.efi [1] from source to make this fully
reproducible by copying from a package (edk2-shell in [testing] [2]). I
hope that will make this fix obsolete.

However, I'm wondering about the differences between EFI Shell 1.0 and
2.0 in this particular combination.
The build descriptor only distinguishes between minimal [3] and full [4]
version.
I'm unsure whether either of them is version 1.0 or 2.0 (or whether this
matters for our use-case).

The current downloads are stating ShellBinPkg (for 2.0) and
EdkShellBinPkg (for 1.0). The former is contained in the official binary
releases that upstream provides [5], but I don't know how the latter is/
was created and whether it is still relevant for archiso. The only
reference I could find is a long deprecated repository [6] that seems to
contain the sources.

Maybe Christian Hesse or Gerardo Exequiel Pozzi can comment on this
topic, as they have updated the links in the past.

If we can't find sources to build from, I honestly feel more inclined to
drop the EFI Shell 1.0 altogether and only provide Shell.efi and
Shell_Full.efi from the newly created package.

Best,
David

[1] https://github.com/tianocore/edk2/blob/master/ShellPkg/
[2] https://www.archlinux.org/packages/testing/any/edk2-shell/
[3] https://github.com/tianocore/edk2/blob/master/ShellPkg/ShellPkg.dsc#L14
[4] https://github.com/tianocore/edk2/blob/master/ShellPkg/ShellPkg.dsc#L121
[5] https://github.com/tianocore/edk2/releases/download/edk2-stable202002/ShellBinPkg.zip
[6] https://github.com/tianocore/edk-Shell

David Runge <dave@sleepmap.de> on Sun, 2020/04/19 23:22:
> Thanks for the suggested patch! I'm currently building the edk2
> Shell.efi and Shell_Full.efi [1] from source to make this fully
> reproducible by copying from a package (edk2-shell in [testing] [2]). I
> hope that will make this fix obsolete.
> 
> However, I'm wondering about the differences between EFI Shell 1.0 and
> 2.0 in this particular combination.
> The build descriptor only distinguishes between minimal [3] and full [4]
> version.
> I'm unsure whether either of them is version 1.0 or 2.0 (or whether this
> matters for our use-case).
> 
> The current downloads are stating ShellBinPkg (for 2.0) and
> EdkShellBinPkg (for 1.0). The former is contained in the official binary
> releases that upstream provides [5], but I don't know how the latter is/
> was created and whether it is still relevant for archiso. The only
> reference I could find is a long deprecated repository [6] that seems to
> contain the sources.
> 
> Maybe Christian Hesse or Gerardo Exequiel Pozzi can comment on this
> topic, as they have updated the links in the past.
> 
> If we can't find sources to build from, I honestly feel more inclined to
> drop the EFI Shell 1.0 altogether and only provide Shell.efi and
> Shell_Full.efi from the newly created package.

Hey David,

thanks for looking into this! I've wanted to do this myself, but never did so.

First of all... I do not have a deep knowledge about this. My changes to the
URL just made sure the files were available.

Booting the upstream binary files (as linked in archiso) results in the two
attached screenshots. Booting the files from your package it looks like both
are v2... No idea where the difference is. And even less idea how to boild a
shell v1...

Also I do not know where EFI before version 2.3 (that requires shell v1) is
still relevant...