MixtureofMerges-MoE-2x7b-v0.01b-DELLA

MixtureofMerges-MoE-2x7b-v0.01b-DELLA is a merge of the following models using mergekit:

🧩 Configuration

models:
  - model: yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
    parameters:
      weight: 1
  - model: jsfs11/MixtureofMerges-MoE-2x7b-v6
    parameters:
      weight: 1.0
merge_method: della
base_model: yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
parameters:
  density: 0.6 
  epsilon: 0.2  
  lambda: 1.0  
dtype: bfloat16
Downloads last month
6
Safetensors
Model size
12.9B params
Tensor type
BF16
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for jsfs11/MixtureofMerges-MoE-2x7b-v0.01b-DELLA

Quantizations
3 models