[pacman-dev] makepkg: respect $SOURCE_DATE_EPOCH to activate reproducible builds

If SOURCE_DATE_EPOCH is set, `touch` all source files before running
build() to fix the modification times. This works around build systems
and compilers that embed the file modification times into the file
contents of release artifacts.

Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
---

Guarded by checking for the variable, because this stomps all over
incremental builds.

I prefer looking for SOURCE_DATE_EPOCH in the environment because less
bloat IMHO, but perhaps this would be preferable as a makepkg flag or
makepkg.conf option?

Is there anything else that needs to be done to force reproducible
builds, other than merely setting SOURCE_DATE_EPOCH which should be
harmless as a global thing?

 scripts/makepkg.sh.in | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

On 07/07/17 03:26, Eli Schwartz wrote:
> If SOURCE_DATE_EPOCH is set, `touch` all source files before running
> build() to fix the modification times. This works around build systems
> and compilers that embed the file modification times into the file
> contents of release artifacts.
> 

I think this is a reasonable compromise.

@Levente:  are you happy with this approach?

On 07/28/2017 04:36 AM, Allan McRae wrote:
> On 07/07/17 03:26, Eli Schwartz wrote:
>> If SOURCE_DATE_EPOCH is set, `touch` all source files before running
>> build() to fix the modification times. This works around build systems
>> and compilers that embed the file modification times into the file
>> contents of release artifacts.
>>
> 
> I think this is a reasonable compromise.
> 
> @Levente:  are you happy with this approach?
> 


The conditional REPRODUCIBLE touch block should be outside of run_build
as former would imply that a build() function is mandatory inside every
PKGBUILD to make this work.
Its quite common (f.e. for python) to not have a build() function and I
believe it should generally work without enforcing to have such function.
Therefor I would recommend we move that block between run_build and
run_prepare, f.e. a place where this works is line 1696 in commit bcc9c417.
A tested/verified adjustment doing the above would be:
https://github.com/anthraxx/pacman/commit/520acf93d83774c4b74ec8c19c1ba31fddbdb8da


Technically this approach will work, but personally i believe this is a
bit too much hidden requirement to make it work. So for clarity I would
either recommend we add a section describing this behaviour inside the
manpage (as manually defining SOURCE_DATE_EPOCH will be mandatory to
make it work) or we use makepkg flags plus config (which will also make
it more transparent).
I don't have any strong favorites, Eli implementation will do its job,
the only difference will be that manually setting SOURCE_DATE_EPOCH is
mandatory for the first initial invocation as well.

cheers,
Levente

PS: thanks :P

On 07/28/2017 07:59 AM, Levente Polyak wrote:
> On 07/28/2017 04:36 AM, Allan McRae wrote:
>> On 07/07/17 03:26, Eli Schwartz wrote:
>>> If SOURCE_DATE_EPOCH is set, `touch` all source files before running
>>> build() to fix the modification times. This works around build systems
>>> and compilers that embed the file modification times into the file
>>> contents of release artifacts.
>>>
>>
>> I think this is a reasonable compromise.
>>
>> @Levente:  are you happy with this approach?
>>
> 
> 
> The conditional REPRODUCIBLE touch block should be outside of run_build
> as former would imply that a build() function is mandatory inside every
> PKGBUILD to make this work.
> Its quite common (f.e. for python) to not have a build() function and I
> believe it should generally work without enforcing to have such function.
> Therefor I would recommend we move that block between run_build and
> run_prepare, f.e. a place where this works is line 1696 in commit bcc9c417.
> A tested/verified adjustment doing the above would be:
> https://github.com/anthraxx/pacman/commit/520acf93d83774c4b74ec8c19c1ba31fddbdb8da

I kind of feel like this is a sign that people are doing the wrong
thing, since you shouldn't be modifying $srcdir during package().

Personally, I am very careful about making sure `python setup.py build`
is run during build(), the same way I am careful about making sure
`make` is run during build() even when `make install` would cause `make
all` to run anyway.

I don't feel like we should make a split between GNU Make and other
build systems, or between a build system that executes a C compiler vs.
one that merely runs pod2man and copies files around.

I won't even mention npm except to say, look at aur/rapydscript-ng-git.

> Technically this approach will work, but personally i believe this is a
> bit too much hidden requirement to make it work. So for clarity I would
> either recommend we add a section describing this behaviour inside the
> manpage (as manually defining SOURCE_DATE_EPOCH will be mandatory to
> make it work) or we use makepkg flags plus config (which will also make
> it more transparent).
> I don't have any strong favorites, Eli implementation will do its job,
> the only difference will be that manually setting SOURCE_DATE_EPOCH is
> mandatory for the first initial invocation as well.

Okay. I personally prefer the implementation that doesn't add more flags
or makepkg.conf options, because at a minimum anyone who wants to make
use of this will need to have tooling to read from a previous invocation
regardless or on top of those flags for the second invocation, so we
might as well follow the same approach the whole time.

Perhaps a REPRODUCIBILITY section in makepkg(1) right after ADDITIONAL
FEATURES, then, to describe what efforts makepkg will go to to ensure a
reproducible build?
And then add SOURCE_DATE_EPOCH for environment variables makepkg will
respect.