Model Card for Model ID

Model Details

Model Description

Evaluation

hf-causal-experimental (pretrained=T3Q-LLM/T3Q-LMM3-MK-v1.0,use_accelerate=true,trust_remote_code=true), limit: None, provide_description: False, num_fewshot: 0, batch_size: 8

Task	Version	Metric	Value		Stderr
kobest_boolq	0	acc	0.5613	±	0.0132
		macro_f1	0.4616	±	0.0128
kobest_copa	0	acc	0.6710	±	0.0149
		macro_f1	0.6705	±	0.0149
kobest_hellaswag	0	acc	0.4560	±	0.0223
		acc_norm	0.5940	±	0.0220
		macro_f1	0.4543	±	0.0223
kobest_sentineg	0	acc	0.6423	±	0.0241
		macro_f1	0.6108	±	0.0252