--- widget: - text: "My name is Sylvain and I live in Paris" example_title: "Parisian" - text: "My name is Sarah and I live in London" example_title: "Londoner" datasets: - zeroMN/nlp_corpus_zh - zeroMN/hanlp_date-zh - nyu-mll/glue - aps/super_glue - facebook/anli - tasksource/babi_nli - zeroMN/AVEdate - sick - snli - scitail - hans - alisawuffles/WANLI - tasksource/recast - sileod/probability_words_nli - joey234/nan-nli - pietrolesci/nli_fever - pietrolesci/breaking_nli - pietrolesci/conj_nli - pietrolesci/fracas - pietrolesci/dialogue_nli - pietrolesci/mpe - pietrolesci/dnc - pietrolesci/recast_white - pietrolesci/joci - pietrolesci/robust_nli - pietrolesci/robust_nli_is_sd - pietrolesci/robust_nli_li_ts - pietrolesci/gen_debiased_nli - pietrolesci/add_one_rte - tasksource/imppres - hlgd - paws - medical_questions_pairs - Anthropic/model-written-evals - truthful_qa - nightingal3/fig-qa - tasksource/bigbench - blimp - cos_e - cosmos_qa - dream - openbookqa - qasc - quartz - quail - head_qa - sciq - social_i_qa - wiki_hop - wiqa - piqa - hellaswag - pkavumba/balanced-copa - 12ml/e-CARE - art - winogrande - codah - ai2_arc - definite_pronoun_resolution - swag - math_qa - metaeval/utilitarianism - mteb/amazon_counterfactual - SetFit/insincere-questions - SetFit/toxic_conversations - turingbench/TuringBench - trec - tals/vitaminc - hope_edi - strombergnlp/rumoureval_2019 - ethos - tweet_eval - discovery - pragmeval - silicone - lex_glue - papluca/language-identification - imdb - rotten_tomatoes - ag_news - yelp_review_full - financial_phrasebank - poem_sentiment - dbpedia_14 - amazon_polarity - app_reviews - hate_speech18 - sms_spam - humicroedit - snips_built_in_intents - hate_speech_offensive - yahoo_answers_topics - pacovaldez/stackoverflow-questions - zapsdcn/hyperpartisan_news - zapsdcn/sciie - zapsdcn/citation_intent - go_emotions - allenai/scicite - liar - relbert/lexical_relation_classification - tasksource/linguisticprobing - tasksource/crowdflower - metaeval/ethics - emo - google_wellformed_query - tweets_hate_speech_detection - has_part - blog_authorship_corpus - launch/open_question_type - health_fact - commonsense_qa - mc_taco - ade_corpus_v2 - prajjwal1/discosense - circa - PiC/phrase_similarity - copenlu/scientific-exaggeration-detection - quarel - mwong/fever-evidence-related - numer_sense - dynabench/dynasent - raquiba/Sarcasm_News_Headline - sem_eval_2010_task_8 - demo-org/auditor_review - medmcqa - RuyuanWan/Dynasent_Disagreement - RuyuanWan/Politeness_Disagreement - RuyuanWan/SBIC_Disagreement - RuyuanWan/SChem_Disagreement - RuyuanWan/Dilemmas_Disagreement - lucasmccabe/logiqa - wiki_qa - tasksource/cycic_classification - tasksource/cycic_multiplechoice - tasksource/sts-companion - tasksource/commonsense_qa_2.0 - tasksource/lingnli - tasksource/monotonicity-entailment - tasksource/arct - tasksource/scinli - tasksource/naturallogic - onestop_qa - demelin/moral_stories - corypaik/prost - aps/dynahate - metaeval/syntactic-augmentation-nli - tasksource/autotnli - lasha-nlp/CONDAQA - openai/webgpt_comparisons - Dahoas/synthetic-instruct-gptj-pairwise - metaeval/scruples - metaeval/wouldyourather - metaeval/defeasible-nli - tasksource/help-nli - metaeval/nli-veridicality-transitivity - tasksource/lonli - tasksource/dadc-limit-nli - ColumbiaNLP/FLUTE - tasksource/strategy-qa - openai/summarize_from_feedback - tasksource/folio - yale-nlp/FOLIO - tasksource/tomi-nli - tasksource/avicenna - stanfordnlp/SHP - GBaker/MedQA-USMLE-4-options-hf - sileod/wikimedqa - declare-lab/cicero - amydeng2000/CREAK - tasksource/mutual - inverse-scaling/NeQA - inverse-scaling/quote-repetition - inverse-scaling/redefine-math - tasksource/puzzte - tasksource/implicatures - race - tasksource/race-c - tasksource/spartqa-yn - tasksource/spartqa-mchoice - tasksource/temporal-nli - riddle_sense - tasksource/clcd-english - maximedb/twentyquestions - metaeval/reclor - tasksource/counterfactually-augmented-imdb - tasksource/counterfactually-augmented-snli - metaeval/cnli - tasksource/boolq-natural-perturbations - metaeval/acceptability-prediction - metaeval/equate - tasksource/ScienceQA_text_only - Jiangjie/ekar_english - tasksource/implicit-hate-stg1 - metaeval/chaos-mnli-ambiguity - IlyaGusev/headline_cause - tasksource/logiqa-2.0-nli - tasksource/oasst2_dense_flat - sileod/mindgames - metaeval/ambient - metaeval/path-naturalness-prediction - civil_comments - AndyChiang/cloth - AndyChiang/dgen - tasksource/I2D2 - webis/args_me - webis/Touche23-ValueEval - tasksource/starcon - PolyAI/banking77 - tasksource/ConTRoL-nli - tasksource/tracie - tasksource/sherliic - tasksource/sen-making - tasksource/winowhy - tasksource/robustLR - CLUTRR/v1 - tasksource/logical-fallacy - tasksource/parade - tasksource/cladder - tasksource/subjectivity - tasksource/MOH - tasksource/VUAC - tasksource/TroFi - sharc_modified - tasksource/conceptrules_v2 - metaeval/disrpt - tasksource/zero-shot-label-nli - tasksource/com2sense - tasksource/scone - tasksource/winodict - tasksource/fool-me-twice - tasksource/monli - tasksource/corr2cause - lighteval/lsat_qa - tasksource/apt - zeroshot/twitter-financial-news-sentiment - tasksource/icl-symbol-tuning-instruct - tasksource/SpaceNLI - sihaochen/propsegment - HannahRoseKirk/HatemojiBuild - tasksource/regset - tasksource/esci - lmsys/chatbot_arena_conversations - neurae/dnd_style_intents - hitachi-nlp/FLD.v2 - tasksource/SDOH-NLI - allenai/scifact_entailment - tasksource/feasibilityQA - tasksource/simple_pair - tasksource/AdjectiveScaleProbe-nli - tasksource/resnli - tasksource/SpaRTUN - tasksource/ReSQ - tasksource/semantic_fragments_nli - MoritzLaurer/dataset_train_nli - tasksource/stepgame - tasksource/nlgraph - tasksource/oasst2_pairwise_rlhf_reward - tasksource/hh-rlhf - tasksource/ruletaker - qbao775/PARARULE-Plus - tasksource/proofwriter - tasksource/logical-entailment - tasksource/nope - tasksource/LogicNLI - kiddothe2b/contract-nli - AshtonIsNotHere/nli4ct_semeval2024 - tasksource/lsat-ar - tasksource/lsat-rc - AshtonIsNotHere/biosift-nli - tasksource/brainteasers - Anthropic/persuasion - erbacher/AmbigNQ-clarifying-question - tasksource/SIGA-nli - unigram/FOL-nli - tasksource/goal-step-wikihow - GGLab/PARADISE - tasksource/doc-nli - tasksource/mctest-nli - tasksource/patent-phrase-similarity - tasksource/natural-language-satisfiability - tasksource/idioms-nli - tasksource/lifecycle-entailment - nvidia/HelpSteer - nvidia/HelpSteer2 - sadat2307/MSciNLI - pushpdeep/UltraFeedback-paired - tasksource/AES2-essay-scoring - tasksource/english-grading - tasksource/wice - Dzeniks/hover - sileod/missing-item-prediction - tasksource/tasksource_dpo_pairs language: - en - zh license: apache-2.0 library_name: transformers tags: - multimodal - vqa - text - audio metrics: - accuracy - bleu - wer model-index: - name: Evolutionary Multi-Modal Model results: - task: type: vqa name: Visual Question Answering dataset: type: synthetic-dataset name: Synthetic Multimodal Dataset split: test metrics: - type: accuracy value: 85 pipeline_tag: audio-text-to-text --- ### Model Sources You need to use separate code, audio, text, and natural language together with the model. Because the model will use separate word segmenters and vocabularies to achieve the best results when dealing with special cases. -- - **Repository:** [https://zeromn-zeromn-shmt.hf.space] - **kaggle:** [https://www.kaggle.com/models/zeroeva/evolutionary-multi-modal) (https://www.kaggle.com/models/zeroeva/evolutionary-multi-modal) - **Demo:** [https://zeromn-zeromn-shmt.hf.space] ## Multi-Modal Model # Model Card for Evolutionary -- - ### Model breast_cancer_wisconsin_original test ```python from ucimlrepo import fetch_ucirepo fetch dataset breast_cancer_wisconsin_original = fetch_ucirepo(id=15) data (as pandas dataframes) X = breast_cancer_wisconsin_original.data.features y = breast_cancer_wisconsin_original.data.targets metadata print(breast_cancer_wisconsin_original.metadata) variable information print(breast_cancer_wisconsin_original.variables) ``` ########################################################## - # 0 0.93 0.99 0.96 79 # 1 0.98 0.90 0.94 58 -- #accuracy 0.95 137 -- -- This model, named `Evolutionary Multi-Modal Model`, is a multimodal transformer designed to handle a variety of tasks including vision and audio processing. It is built on top of the `adapter-transformers` and `transformers` libraries and is intended to be a versatile base model for both direct use and fine-tuning. - -- **Developed by:** Independent researcher **Funded by :** Self-funded **Shared by :** Independent researcher **Model type:** Multimodal **Language(s) (NLP):** English zh **License:** Apache-2.0 **Finetuned from model :** None - ## Uses：https://huggingface.co/zeroMN/SHMT ### Direct Use ```python git lfs install git clone https://huggingface.co/zeroMN/SHMT.git ``` ### Downstream Use The model can be fine-tuned for specific tasks such as visual question answering (VQA), image captioning, and audio recognition. ### Out-of-Scope Use The Evolved Multimodal Model is not suitable for tasks that require high expertise or domain-specific expertise beyond its current capabilities. The number of speech frames still needs to be fine-tuned by yourself. ## Bias, Risks, and Limitations ### Recommendations Users (both direct and downstream) should be made aware of the following risks, biases, and limitations: - **Bias:** The model may exhibit biases present in the training data, particularly if the data is not representative of all populations. - **Risks:** The model should not be used in critical applications where high accuracy and reliability are required without thorough testing and validation. - **Limitations:** The model may not perform well on tasks that require fine-grained recognition or highly specialized audio processing. ## How to Get Started with the Model ```python # Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="zeroMN/SHMT") ``` ```python # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("zeroMN/SHMT") ```