Georgi Gerganov
a17e94d229
* tests : consolidate the context state tests into test-llama-context test-save-load-state, test-recurrent-state-rollback and test-state-restore-fragmented check the same object from three places: the KV cache and the recurrent state of a llama_context, saved, loaded, copied, rolled back. Each had its own main, its own loop over the models, its own copy of the token and the comparison helpers, and a run of all three loaded the same models several times. They are one binary now, and the case is the unit of the test: tests/llama-context/test-case.h the interface a case implements tests/llama-context/cases-kv.cpp the cases of the KV cache tests/llama-context/cases-rs.cpp the cases of the recurrent state tests/llama-context/test-util.cpp tokens, generation, comparison tests/llama-context/test-runner.cpp the models of a case, on threads tests/llama-context/test-log.cpp the log file of a model A run is a table: a row per model, a column per case, a cell holds the verdict of that pair. The models load on several threads (-j, --threads, min(cores, 4) by default), the table is printed when it is complete and the log of a model that failed is printed after it, in the order of the rows. A case returns a verdict and the reason for it, and the runner is the only one who reports - the test does not use tests/testing.h, a run over 130 models is a matrix of verdicts and not a tree of subtests. Staging no longer travels through the environment: llama_model_set_attn_rot replaces the LLAMA_ATTN_ROT_DISABLE variable, which a model loaded in one thread used to change for the models the other threads were loading, and llama_memory_get_buffers is added for the cases that copy the buffers of a memory. common_log_set_stdo keeps the console quiet while the table is being built. ggml_aligned_malloc allocates with posix_memalign when the thread sanitizer is on. On macOS it allocates with the Mach VM, which the sanitizer does not intercept, so it never sees a page range change hands and reports the first write into recycled memory as a race - 30 of them in this test, and every one of them false. Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next * cont : run single-threaded by default * cont : try to build the tests on all platforms * cont : trim down some body fat * cont : fix * cont : fix the fix
34 lines
1.5 KiB
C++
34 lines
1.5 KiB
C++
#pragma once
|
|
|
|
#include "log.h"
|
|
|
|
#include <string>
|
|
|
|
//
|
|
// the log of a model goes to a file of its own, so that a model which failed can be read on its
|
|
// own. a worker thread runs one model and the log callback fires on the thread that produced the
|
|
// message, so a capture belongs to the thread that opened it
|
|
//
|
|
|
|
// open the log of this thread, an empty path closes it - a capture holds what the verbosity of
|
|
// the process allows, never less than info, the details of a failure are looked for in the file
|
|
void test_log_capture(const std::string & path);
|
|
|
|
// llama, ggml and the backends log through this: into the file of this thread when it captures,
|
|
// into the default callback otherwise
|
|
void test_log_callback(enum ggml_log_level level, const char * text, void * user_data);
|
|
|
|
// the lines of the test itself go to the file of this thread, an error goes to stderr as well when
|
|
// there is no capture and the run is followed on the console
|
|
void test_log_add(enum ggml_log_level level, const char * fmt, ...) LOG_ATTRIBUTE_FORMAT(2, 3);
|
|
|
|
// print a captured log, false when it was printed in full and the file can be removed
|
|
bool test_log_dump(const std::string & path);
|
|
|
|
std::string test_log_path(const std::string & logs_dir, const std::string & model_name);
|
|
|
|
// a trace is a detail of a case, it is there to read in the file of the model
|
|
#define TLOG_ERR(...) test_log_add(GGML_LOG_LEVEL_ERROR, __VA_ARGS__)
|
|
#define TLOG_INF(...) test_log_add(GGML_LOG_LEVEL_INFO, __VA_ARGS__)
|
|
#define TLOG_TRC(...) test_log_add(GGML_LOG_LEVEL_DEBUG, __VA_ARGS__)
|