Android JNI llama cache temperature in class

[kirklandsign]

So far, for LLM, we can use the new config, with cached temperature from ctor. For llava, we use the old workflow.

We will update the API to move temperature from ctor to generate() next.

Test: instrumentation test

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10287

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [Infra] Jobs got frequently cancelled, sometimes mid-checkout

❌ 1 New Failure, 1 Unrelated Failure

As of commit e8b0a67 with merge base 96c10bb ():

NEW FAILURE - The following job has failed:

• Lint / lintrunner / linux-job (gh)
  >>> Lint for util/python_profiler.py:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build documentation / check-urls (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Copilot]

Pull Request Overview

This PR introduces caching for the temperature parameter in the Android JNI layer for llama models. The changes include adding a new member variable to store temperature, initializing that variable in the constructor, and updating the generation configuration to use the cached value.

• Added a new member variable (temperature_) to cache the temperature.
• Assigned the temperature parameter to temperature_ in the constructor.
• Updated the generation configuration to use temperature_.

Comments suppressed due to low confidence (1)

extension/android/jni/jni_layer_llama.cpp:186

• Please verify that removing the temperature parameter from the MTKLlamaRunner constructor is intentional. If the temperature was previously required by the runner, additional changes in MTKLlamaRunner or its usage may be needed.

tokenizer_path->toStdString().c_str());

[facebook-github-bot]

@kirklandsign has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.