finetune.cpp command-line arg

[lexasub]

add to ggml-opt learning rate (adamw alpha) cmdline arg, and an optimizer enum defaulting to adamw,
preparatory to work to support SGD

these are in common args a set of optimizer options active only for the new FINETUNE example (which includes all the previous finetune.cpp PERPLEXITY options as a precaution)

perhaps breaking with precedent, the ggml_opt_optimizer_params struct is included directly as args - if desired, we can instead just add learning rate and optimizer type to a struct independent of ggml-opt.h

as proposed in
#13835
rebase #13873 graehl:finelayer to master

[lexasub]

@graehl @JohannesGaessler rebase original graehl:finelayer to master

[lexasub]

@graehl need fix for webgpu))
some insites from github copilot (This seems like complete garbage)

1. "Unrecognized schema" Error in json-schema-to-grammar.mjs
Cause:
The error is raised at line 716 in tools/server/public_legacy/json-schema-to-grammar.mjs when the schema has an unexpected type, e.g. { "type": "kaboom" } or { "type": 123 }.

Solution:

Update your tests to only use supported schema types.
If you intend to support more types, extend your schema converter:
js
// At or before line 716 in json-schema-to-grammar.mjs
switch (schema.type) {
  case "string":
  case "number":
  case "object":
    // handle known types
    break;
  default:
    throw new Error(`Unrecognized schema: ${JSON.stringify(schema)}`);
}
Document valid schema types or add test cases for expected failures.
2. "Failed to infer a tool call example (possible template bug)" in chat template tests
Cause:
The template logic in /build/bin/test-chat-template can't infer example tool calls, suggesting a bug in the template definition or missing/invalid test parameters.

Solution:

Check your chat template logic for missing example cases.
Ensure all expected tool call examples are defined.
Add fallback logic or improve error messaging:
C++
// Pseudocode: If no example can be inferred, return a generic message or handle gracefully.
if (!can_infer_example()) {
    std::cerr << "Tool call example could not be inferred. Check template and test data." << std::endl;
}
3. GGUF File Format Errors in test-gguf
Cause:
Multiple errors (invalid magic, version mismatch, tensor issues) indicate the test is using bad or outdated GGUF files, or the GGUF reader code needs updates.

Solution:

Update or regenerate GGUF files used in the tests to match supported versions and formats.
In your GGUF handling code, add more robust validation and clearer error messages.
Example: Ensure version checks and magic headers are correct:
C++
if (magic != EXPECTED_MAGIC) {
    throw std::runtime_error("Invalid GGUF magic header.");
}
if (version > SUPPORTED_VERSION) {
    throw std::runtime_error("GGUF version not supported. Please update your software.");
}
Review tensor constraints (name length, dimensions, alignment) in your test files and code.
4. "test-opt Subprocess Aborted"
Cause:
Test #29 (test-opt) crashed, likely due to a segmentation fault or unhandled exception.

Solution:

Check /build/Testing/Temporary/LastTest.log for details.
Review code in the test-opt binary for unsafe operations, uninitialized pointers, or memory issues.
Add error handling and assertions to catch bad states before crashing:
C++
assert(ptr != nullptr); // Example safety check
5. Argument Parsing Errors
Cause:
Errors like error while handling argument "-m": expected value for argument indicate missing or invalid command-line arguments in your test invocation scripts.

Solution:

Ensure all required arguments are passed in your CI scripts and test definitions.
Add checks for missing or invalid arguments, and provide usage help.
Example fix for test script:

sh
# Make sure to pass a valid model path to -m
./test-binary -m /path/to/model
Summary of Immediate Actions:

Fix schema types in tests or extend schema handling logic.
Check template and tool call example logic for completeness.
Update GGUF test files and handling code.
Investigate and fix the crash in test-opt.
Ensure all test scripts pass valid arguments.```