Discarded GOs are emitted during the second build.

[MaggieYingYi]

This is spin-off from bug #22 (comment) (“The hash values of an object file are different between two builds without any changes”).

The definition of some global objects may be discarded if not used in its compilation unit.
A discardable GO might have linkonce, internal, private or available externally linkage type.
Any pruned function will always be emitted in the repo object file. Therefore, the discarded GOs are emitted into the object file during the second time build if they are pruned.

[MaggieYingYi]

A small example, two files:

–a.c-
static int f0() {
  return 1;
}
int f1() {
  return f0();
}
–b.c-
int f0() {
  return 1;
}

Compile 'a.c' (first time) and dump the compilation of 'a.o':

$ rm clang.db
$ clang -c -O3  --target="x86_64-pc-linux-gnu-repo" a.c -o a.o
$ pstore-dump -compilation=$(repo-ticket-dump a.o) clang.db
---
- file         :
      path : clang.db
      size : 4194304
  compilations :
      - digest      : 68f6d0ccd5e7757b72eff9b7eb656939
        compilation :
            members :
                - digest  : b05135716cc9d7b8f37b9aae62a56656
                  fext    : { addr: 256, size: 38 }
                  name    : f1
                  linkage : external
            path    : \\\\?\\C:\\MyWork\\repo_bug\\issue22\\case1
            triple  : x86_64-pc-linux-gnu-repo
...

We could see: there is only one compilation member (f1) in the compilation. The function (f0) with internal linkage is discarded since it is not referenced.

Compile 'b.c'.

$ clang -c -O3  --target="x86_64-pc-linux-gnu-repo" b.c -o b.o

The function (f0) is stored into the database since it has an external linkage type.

Compile 'a.c' again (second time) and dump the compilation of 'a.o':

$ clang -c -O3  --target="x86_64-pc-linux-gnu-repo" a.c -o a.o
$ pstore-dump -compilation=$(repo-ticket-dump a.o) clang.db
---
- file         :
      path : clang.db
      size : 4194304
  compilations :
      - digest      : 685b0475bb288e9808f9a6e8e81bf9b1
        compilation :
            members :
                - digest  : b05135716cc9d7b8f37b9aae62a56656
                  fext    : { addr: 256, size: 38 }
                  name    : f1
                  linkage : external
                - digest  : ec98eeddc27fd19e4f69a4d89f867a51
                  fext    : { addr: 696, size: 38 }
                  name    : f0
                  linkage : internal
            path    : \\\\?\\C:\\MyWork\\repo_bug\\issue22\\case1
            triple  : x86_64-pc-linux-gnu-repo
...

We could see: there are two compilation members ('f0' and 'f1') in the compilation. Since the function f0 is pruned during the repo pruning pass, it is emitted into the compilation file even it is not referenced.

[MaggieYingYi]

A possible solution is mentioned in #22 (comment).

The functions/variables, which have internal, private or linkonce linkage type, are only emitted into the compilation file if they are referenced.

[MaggieYingYi]

After I fixed the issue by only emitting the referenced local variables/functions into the compilation file, I found another issue: there are fewer compilation members in the ticket file in the second time build compared to the first time build.

The unreferenced local variables/functions have been emitted into the compilation build during the first time build targeted on repo. Please see the issue #57 for the details.

[MaggieYingYi]

A possible solution is mentioned in #22 (comment).

The functions/variables, which have internal, private or linkonce linkage type, are only emitted into the compilation file if they are referenced.

I have implemented this possible solution in the https://github.com/SNSystems/llvm-project-prepo/tree/issue102_ObjectWriter. However, it does not resolve the case mentioned in issue #102 since X is not removed and still in the compilation. Please see issue #102 for the details.

Note: I called _ZSt16__insertion_sortIPSt10unique_ptrIN12_GLOBAL__N_19IntrinsicESt14default_deleteIS2_EEN9__gnu_cxx5__ops15_Iter_comp_iterIZNS1_10SVEEmitter17createRangeChecksERN4llvm11raw_ostreamEE3$3EEEvT_SG_T0 as X to make more readable.

Reason:

I used the print-after-all flag to print IR code after each pass. Since the X is pruned (with the existing database), its linkage type changed from internal to available_externally. The linkage type change prevents the other passes’ optimisation. The X is not removed and still called by other functions. Therefore, it still exists in the compilation.

Since we want to generate the same ticket file when compiled the same source file with or without existing database, the above solution does not work

[paulhuggett]

A possible solution is mentioned in #22 (comment). I have implemented this possible solution in the https://github.com/SNSystems/llvm-project-prepo/tree/issue102_ObjectWriter. However, it does not resolve the case mentioned in issue #102

Perhaps I’m stating the obvious, but a fix for this bug doesn’t necessarily have to also fix another problem. Unless, of course, they’re the same problem. However, I don’t think we’re yet able to reach that conclusion because the underlying cause of #102 in is not yet understood in detail.