Edit model card

Model Card for Alpacazord-Viking-7B

This is a merge of mpasila/Alpacazord-Viking-LoRA-7B.

LoRA trained with text-generation-webui in 4-bit using LumiOpen/Viking-7B as the base model for 1 epoch. Dataset used with the LoRA is mpasila/Alpacazord-V1.

It uses Alpaca format like so:

{
    "instruction,output": "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n%instruction%\n\n### Response:\n%output%",
    "instruction,input,output": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n%instruction%\n\n### Input:\n%input%\n\n### Response:\n%output%"
}

Merged using this Colab notebook. It might not be the best way to merge a quantized LoRA on to a float16 model but I just wanted to quickly do something. You can try merging it better if you want.

Evaluation

Model Size Type FIN-bench (score)
mpasila/Alpacazord-Viking-7B 7B Instruct 0.4123
mpasila/Finnish-Alpaca-Small-7B 7B Instruct 0.3586
mpasila/Finnish-Alpaca-Tiny-V2-7B 7B Instruct 0.4654
mpasila/Finnish-Viking-Alpaca-V1-7B 7B Instruct 0.3943
mpasila/NordicAlpaca-Finnish-V1-7B 7B Instruct 0.3891
Finnish-NLP/llama-7b-finnish-instruct-v0.1 7B Instruct 0.4365
Finnish-NLP/llama-7b-finnish-instruct-v0.2 7B Instruct 0.3993
Finnish-NLP/llama-7b-finnish 7B Base 0.2350
LumiOpen/Viking-7B (1000B) 7B Base 0.3721
HPLT/gpt-7b-nordic-prerelease 7B Base 0.3169

Source

FIN-bench scores:

Task Version Metric Value Stderr
bigbench_analogies 0 multiple_choice_grade 0.5000 ± 0.0440
bigbench_arithmetic_1_digit_addition 0 multiple_choice_grade 0.6600 ± 0.0476
bigbench_arithmetic_1_digit_division 0 multiple_choice_grade 0.6957 ± 0.0981
bigbench_arithmetic_1_digit_multiplication 0 multiple_choice_grade 0.3800 ± 0.0488
bigbench_arithmetic_1_digit_subtraction 0 multiple_choice_grade 0.4400 ± 0.0499
bigbench_arithmetic_2_digit_addition 0 multiple_choice_grade 0.5000 ± 0.0503
bigbench_arithmetic_2_digit_division 0 multiple_choice_grade 0.5400 ± 0.0501
bigbench_arithmetic_2_digit_multiplication 0 multiple_choice_grade 0.2100 ± 0.0409
bigbench_arithmetic_2_digit_subtraction 0 multiple_choice_grade 0.3100 ± 0.0465
bigbench_arithmetic_3_digit_addition 0 multiple_choice_grade 0.5200 ± 0.0502
bigbench_arithmetic_3_digit_division 0 multiple_choice_grade 0.2800 ± 0.0451
bigbench_arithmetic_3_digit_multiplication 0 multiple_choice_grade 0.1700 ± 0.0378
bigbench_arithmetic_3_digit_subtraction 0 multiple_choice_grade 0.5000 ± 0.0503
bigbench_arithmetic_4_digit_addition 0 multiple_choice_grade 0.3600 ± 0.0482
bigbench_arithmetic_4_digit_division 0 multiple_choice_grade 0.3000 ± 0.0461
bigbench_arithmetic_4_digit_multiplication 0 multiple_choice_grade 0.0400 ± 0.0197
bigbench_arithmetic_4_digit_subtraction 0 multiple_choice_grade 0.4900 ± 0.0502
bigbench_arithmetic_5_digit_addition 0 multiple_choice_grade 0.3700 ± 0.0485
bigbench_arithmetic_5_digit_division 0 multiple_choice_grade 0.1400 ± 0.0349
bigbench_arithmetic_5_digit_multiplication 0 multiple_choice_grade 0.0000 ± 0.0000
bigbench_arithmetic_5_digit_subtraction 0 multiple_choice_grade 0.4500 ± 0.0500
bigbench_cause_and_effect_one_sentence 0 multiple_choice_grade 0.5882 ± 0.0696
bigbench_cause_and_effect_one_sentence_no_prompt 0 multiple_choice_grade 0.7451 ± 0.0616
bigbench_cause_and_effect_two_sentences 0 multiple_choice_grade 0.4706 ± 0.0706
bigbench_emotions 0 multiple_choice_grade 0.2938 ± 0.0361
bigbench_empirical_judgments 0 multiple_choice_grade 0.3333 ± 0.0476
bigbench_general_knowledge 0 multiple_choice_grade 0.3429 ± 0.0571
bigbench_hhh_alignment_harmless 0 multiple_choice_grade 0.3621 ± 0.0637
bigbench_hhh_alignment_helpful 0 multiple_choice_grade 0.3559 ± 0.0629
bigbench_hhh_alignment_honest 0 multiple_choice_grade 0.4068 ± 0.0645
bigbench_hhh_alignment_other 0 multiple_choice_grade 0.5581 ± 0.0766
bigbench_intent_recognition 0 multiple_choice_grade 0.2587 ± 0.0167
bigbench_misconceptions 0 multiple_choice_grade 0.5299 ± 0.0433
bigbench_paraphrase 0 multiple_choice_grade 0.5050 ± 0.0354
bigbench_sentence_ambiguity 0 multiple_choice_grade 0.5000 ± 0.0651
bigbench_similarities_abstraction 0 multiple_choice_grade 0.7368 ± 0.0508

Framework versions

  • PEFT 0.8.2
Downloads last month
12
Safetensors
Model size
7.55B params
Tensor type
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for mpasila/Alpacazord-Viking-7B

Base model

LumiOpen/Viking-7B
Finetuned
(19)
this model
Merges
1 model
Quantizations
1 model

Dataset used to train mpasila/Alpacazord-Viking-7B

Collection including mpasila/Alpacazord-Viking-7B