Add jemalloc profiling via HTTP API

[jimmygchen]

Issue Addressed

Add jemalloc profiling via HTTP API.

This is included at build time if jemalloc is enabled, however not activated by default. User will need to enable profiling via the curl command below.

Commands

# Enables profiling
curl -X POST http://localhost:4000/lighthouse/malloc/prof_active -H "Content-Type: application/json" -d "true"

# Produce a dump
curl -X POST http://localhost:4000/lighthouse/malloc/prof_dump \
     -H "Content-Type: application/json" \
     -d '"/home/ubuntu/prof.dump"'

Additional notes

• DOCUMENTED: debug symbol needs to be enabled when building the binary
• needs to confirm it works on different targets
  • so far this works on ubuntu and macOS
• needs to confirm profiling is inactive on startup - confirmed the memory growth was due to a sync bug (Fix leak in range sync components_by_range_requests #7767)
• might need a way to purge samples in memory - I don't think this is strictly necessary at this point. We can add it if we need it later.

[michaelsproul]

LGTM, looking at the source this corresponds to mallctl("prof.dump", NULL, NULL, filename, size) which is exactly what we want

Ref:

• https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Heap-Profiling
• https://docs.rs/tikv-jemalloc-ctl/0.6.0/src/tikv_jemalloc_ctl/raw.rs.html#137-147

[michaelsproul]

LGTM! Let's merge before this bitrots (again)!!

[michaelsproul]

Actually, don't we need some way to trigger this? Paul had an HTTP API I think

[mergify]

This pull request has been removed from the queue for the following reason: pull request dequeued.

Pull request #7746 has been dequeued. The pull request rule doesn't match anymore. The following conditions don't match anymore:

• label=ready-for-merge.

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.
If you do update this pull request, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

[jimmygchen]

Sorry, I didn't include everything - I've pushed all changes now and I'm going to test it first as there were quite a bit of conflicts and changes that I had to make.

[jimmygchen]

Got a bit stuck with getting debug symbols in the dump, will come back to this later.

[jimmygchen]

Managed to get this to work, I've added a page to the lighthouse book with the instructions.

[jimmygchen]

@michaelsproul I feel like we probably should try to merge this even if the output wasn't very useful this time - it's likely at some point in the future we're going to come back and look for this branch and try to revive it (happened twice already!)
I was a bit unsure because of the memory growth, but now we've identified the OOM cause, I feel more comfortable with this PR now.

We can probably try both jemalloc profiling and heaptrack again after the OOM fix.

[chong-he]

Looks good now for the documentation side of the change