--- base_model: - Nexesenex/Llama_3.x_70b_SmarTracks_0.21 - Nexesenex/Llama_3.x_70b_SmarTracks_0.31_R1 - Nexesenex/Llama_3.x_70b_SmarTracks_0.31_NMT library_name: transformers tags: - mergekit - merge --- # merge This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [Nexesenex/Llama_3.x_70b_SmarTracks_0.21](https://huggingface.co/Nexesenex/Llama_3.x_70b_SmarTracks_0.21) as a base. ### Models Merged The following models were included in the merge: * [Nexesenex/Llama_3.x_70b_SmarTracks_0.31_R1](https://huggingface.co/Nexesenex/Llama_3.x_70b_SmarTracks_0.31_R1) * [Nexesenex/Llama_3.x_70b_SmarTracks_0.31_NMT](https://huggingface.co/Nexesenex/Llama_3.x_70b_SmarTracks_0.31_NMT) ### Configuration The following YAML configuration was used to produce this model: ```yaml merge_method: model_stock models: - model: Nexesenex/Llama_3.x_70b_SmarTracks_0.31_NMT parameters: weight: 1.0 - model: Nexesenex/Llama_3.x_70b_SmarTracks_0.31_R1 parameters: weight: 1.0 base_model: Nexesenex/Llama_3.x_70b_SmarTracks_0.21 dtype: bfloat16 out_dtype: bfloat16 parameters: int8_mask: true normalize: true rescale: false chat_template: auto tokenizer: source: union ```