Enhancing 3D Asset Retrieval with Semantic Search

Semantic search goes beyond exact keyword matching and instead focuses on the comprehension of search queries' purpose and the surrounding context to deliver relevant search results. In this work, we focus on integrating hybrid search, which …


Enhancing AI Tutoring in Robotics Education: Evaluating the Effect of Retrieval-Augmented Generation and Fine-Tuning on Large Language Models

S Kahl, F Löffler, M Maciol, F Ridder, M Schmitz… - 2024

… The majority of data is gained from CommonCrawl with about 850 billion tokens. CommonCrawl is a free and open of data gathered by web crawls.In addition, the corpus is based on data from C4 (about 190 billion tokens), Github (about 100 billion …


ENHANCING AUTOMATIC INVOICE CODING PERFORMANCE WITH UNSTRUCTURED DATA

… The published word vectors are 300 dimensional and trained on the internet data from Common Crawl3. The pretrained word vectors can be further fine-tuned using the provided code and datasets more similar to the final usage scenario to increase …


Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation

T Wang, N Zhou, Z Chen - arXiv preprint arXiv:2407.05437, 2024

Large language models (LLMs) and prompt engineering hold significant potential for advancing computer programming education through personalized instruction. This paper explores this potential by investigating three critical research questions: the …


Enhancing Cyber Security Enhancement Through Generative AI

… GPT-1: Released in 2018, GPT-1 was trained on the Common Crawl and BookCorpus datasets. It demonstrated proficient language comprehension but limited conversational coherence and retention of context. GPT-2: Trained on …


Enhancing Cybersecurity: A Review and Comparative Analysis of Convolutional Neural Network Approaches for Detecting URL-Based Phishing Attacks

M Nanda, M Saraswat, PK Sharma - e-Prime-Advances in Electrical Engineering …, 2024

Phishing attempts to mimic the official websites of businesses, including banks, e-commerce, government offices, and financial institutions. Phishing websites aim to collect and retrieve sensitive data from users, including passwords, credit card numbers, email …


Enhancing Ecological Knowledge Discovery Using Large Language Models

Earth is home to an astonishing diversity of life forms. Among these, vascular plants stand as one of the most vital and ubiquitous groups, accounting for approximately 80% of global biomass [3]. The census of vascular plants, which presently exceeds …


Enhancing English Translation Quality Assessment through Knowledge Transfer in Artificial Intelligence Context

Abstract Machine translation technology, which employs computers to autonomously convert text between source and target languages, represents a pivotal realm within artificial intelligence and natural language processing research. This paper …


Enhancing knowledge graphs with microdata and LLMs: the case of Schema. org and Wikidata in touristic information

L Gonzalez-Garcia, G González-Carreño… - The Electronic Library, 2024

… We have used the Common Crawl (CC) corpus of crawled Web pages to extract metadata published in the wild, and then used a mapping of Schema.org annotations to Wikidata to get estimates of the added value (in terms of additional …


Enhancing Large Language Models for Structured Data Extraction

This thesis examines the application of large language models for structured data extraction using microdata-annotated webpages from the Web Data Commons (WDC). It compares the performance of models trained on WDC data with those trained on …


Enhancing Misogyny Detection in Bilingual Texts Using FastText and Explainable AI

E Hashmi, MM Yamin, S Imran, SY Yayilgan, M Ullah - 2024 International Conference …, 2024

… It features an extensive dictionary with 2 million words drawn from the Common Crawl database. Every word is depicted in a 300-dimensional vector space, amassing an enormous total of 600 billion word vectors. 1) Unsupervised FastText …


Enhancing Multilingual Hate Speech Detection: From Language-Specific Insights to Cross-Linguistic Integration

E Hashmi, SY Yayilgan, IA Hameed, MM Yamin… - IEEE Access, 2024

… It features a vast vocabulary of 2 million words from the Common Crawl dataset, mapping each to a 300-dimensional vector space. What … This model was built using Common Crawl and Wikipedia using FastText’s unsupervised learning …


Enhancing Parameter Efficiency in Model Inference Using an Ultralight Inter-Transformer Linear Structure

… It was constructed from the Chuweb21 generated based on the April 2021 block of the Common Crawl dataset. The WWW-4 dataset comprises 50 queries. Robust04 dataset is also an English web corpus generated from TREC disks 4 and 5, and …


Enhancing Phishing Detection, Leveraging Deep Learning Techniques

A Ullah, RA Shah, SA Nawaz, N Ahmad, MH Malik - Journal of Computing & …, 2024

… To achieve this, we utilized the Common Crawl Corpus, which aggregates extensive data collected over seven years. This corpus comprises raw web page data, metadata, and textual content. Legitimate domains with substantial backlink …


Enhancing Product Design through AI-Driven Sentiment Analysis of Amazon Reviews Using BERT

MK Shaik Vadla, MA Suresh, VK Viswanathan - Algorithms, 2024

Understanding customer emotions and preferences is paramount for success in the dynamic product design landscape. This paper presents a study to develop a prediction pipeline to detect the aspect and perform sentiment analysis on review …


Enhancing Sentiment Analysis Accuracy in Borobudur Temple Visitor Reviews through Semi-Supervised Learning and SMOTE Upsampling

C Agustina, P Purwanto, F Farikhin - Journal of Advances in Information Technology, 2024

The level of visitor satisfaction with tourist destinations can be known from reviews on social media. One method used is to carry out sentiment analysis on comments given by visitors on social media or related websites. This study was envisioned as a …


Enhancing Sentiment Analysis on Social Media Data with Advanced Deep Learning Techniques.

HH Nguyen - International Journal of Advanced Computer Science & …, 2024

This paper introduces a comprehensive methodology for conducting sentiment analysis on social media using advanced deep learning techniques to address the unique challenges of this domain. As digital platforms play an increasingly pivotal …


Enhancing Text Summarization in a Limited Data and Long Sample Setting

This thesis addresses the challenges of automatic text summarization in a limited amount of data and a significantly large sample size setting. The study introduces a novel dataset of Ukrainian legislative documents professionally translated into …


Enhancing Text Summarization in a Limited Data and Long Sample

In our rapidly digitizing world, the volume of online information is growing at an unprecedented rate. Legal systems worldwide do not lag behind and generate massive amounts of documents daily. Lawyers are tasked with navigating an ever-growing …


Enhancing Vision-Language Pre-training with Rich Supervisions

Y Gao, K Shi, P Zhu, E Belval, O Nuriel, S Appalaraju… - arXiv preprint arXiv …, 2024

… This is expected as our data collected from Common Crawl has different distribution against the original Pix2struct data due to the data size (2M vs. 80M) and potentially different website filtering strategies. Specifically, we filter out website …


Enriching Interactive Explanations with Fuzzy Temporal Constraint Networks

M Canabal-Juanatey, JM Alonso-Moral, A Catala… - International Journal of …, 2024

… such as BERT [2] (Bidirectional Encoder Representations from Transformers), GPT [26] (Generative Pre-trained Transformer) and their successors (ALBERT [4], RoBERTa [3], BART [7], GPT-3 [5] or GPT-4 [27]), which were trained with huge …


Enriching satellite image annotations of forests with keyphrases from a specialized corpus

N Neptune, J Mothe - Multimedia Tools and Applications, 2024

… We test fastText embeddings trained on Common Crawl Footnote 4 and Wikipedia Footnote 5 with dimension 300 and a context size of 5 … These fastText models were pre-trained on Common Crawl and Wikipedia. None of the models was …


Entrant: A large financial dataset for table understanding

E Zavitsanos, D Mavroeidis, E Spyropoulou… - Scientific Data, 2024

Tabular data is a way to structure, organize, and present information conveniently and effectively. Real-world tables present data in two dimensions by arranging cells in matrices that summarize information and facilitate side-by-side comparisons …


EpilepsyLLM: Domain-Specific Large Language Model Fine-tuned with Epilepsy Medical Knowledge

X Zhao, Q Zhao, T Tanaka - arXiv preprint arXiv:2401.05908, 2024

With large training datasets and massive amounts of computing sources, large language models (LLMs) achieve remarkable performance in comprehensive and generative ability. Based on those powerful LLMs, the model fine-tuned with domain-specific …


Error Patterns in Vision Transformers: A Focused Study on InternVL's Real-World Performance

This study focuses on the errors in Vision Transformers, particularly examining the realworld performance of InternVL, an opensource multimodal model designed to rival commercial systems. The primary tasks explored include image captioning and …


Establishing Global AI Accountability: Training Data Transparency, Copyright, and Misinformation

As artificial intelligence (AI) technologies continue advancing at a rapid pace, the systems' growing capabilities as well as their expanding integration into vital social functions are raising complex questions around trust and accountability. AI models …


Establishing the importance of co-creation and self-efficacy in creative collaboration with artificial intelligence

J McGuire, D De Cremer, T Van de Cruys - Scientific Reports, 2024

The emergence of generative AI technologies has led to an increasing number of people collaborating with AI to produce creative works. Across two experimental studies, in which we carefully designed and programmed state-of-the-art human–AI …


Ethical aspects of ChatGPT: An approach to discuss and evaluate key requirements from different ethical perspectives

M Steen, J Greeff, M Boer, C Veenman - AI and Ethics, 2024

… Many LLMs have been trained with data from Common Crawl, Footnote 9 which contains inaccuracies, errors, and bias. Another concern is whether people can access ChatGPT and, for example, change parameters or delete data, so that the …


Ethical Considerations for Generative AI in Social Science Research

BA Brown, KL Heitner - Generative AI and Implications for Ethics, Security, and …, 2024

Social science research embodies the inquiry into people as individuals and their interpersonal interactions with each other in communities and varied societies, with due consideration for their natural, technological, and constructed environments …


Ethics of AI in the Teaching of English

New technologies have always impacted literacy, and artificial intelligence (AI) is no exception (Tyner, 1998). English teachers find themselves having to consider how AI may impact how they teach. NCTE’s Beliefs for Integrating Technology Into the …


EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluation

AL Tonja, IA Azime, TD Belay, MG Yigezu… - arXiv preprint arXiv …, 2024

Large language models (LLMs) have gained popularity recently due to their outstanding performance in various downstream Natural Language Processing (NLP) tasks. However, low-resource languages are still lagging behind current state-of-the-art …


EvalQuiz: self-assessment generated through language transformer models

This thesis explores the constraints of self-assessment creation in higher education, focusing on the lack of tools, standardization, and time. We conduct didactic field expert interviews to investigate how teaching evolves towards learning management …


Evaluating and improving lexical language understanding in neural machine translation

Lexical understanding is an inalienable component of the translation process. In order to correctly map the meaning of a linguistic unit to the appropriate target language expression, the meaning of its constituent words has first to be identified …


Evaluating and Improving the Reasoning Abilities of Language Models

This thesis focuses on evaluating and improving the reasoning abilities of Smaller Language Models (SLMs) and Large Language Models (LLMs). It explores SLMs’ performance on complex tasks and their limitations with simpler ones. This thesis …


Evaluating and Mitigating Limitations of Large Language Models in Clinical Decision Making

P Hager, F Jungmann, K Bhagat, I Hubrecht, M Knauer… - medRxiv, 2024

Clinical decision making is one of the most impactful parts of a physician's responsibilities and stands to benefit greatly from AI solutions and large language models (LLMs) in particular. However, while LLMs have achieved excellent …


Evaluating Differential Privacy Approaches for Query Obfuscation in Information Retrieval

Protecting the privacy of a user while they interact with an Information Retrieval (IR) system is crucial. This becomes more challenging when the IR system is not cooperative in satisfying the user’s privacy needs. Recent advancements in Natural …


Evaluating embedded semantics for accessibility description of web crawl data

R Navarrete, D Martinez-Mosquera, L Recalde… - AHFE (2023) International …, 2023

… The dataset for analysis is provided by Web Data Commons (WDC), an organization that releases extracted data from Common Crawl (CC), the largest web corpus available to the public. The dataset was released in October 2021 …


Evaluating Embedding Models for Clustering Italian Political News: A Comparative Study of Text-Embedding-3-Large and UmBERTo

… This study compares two embedding models: OpenAI's text-embedding-3-large model and umberto-commoncrawl-cased-v1 (hereafter referred to as UmBERTo) (Parisi et al., 2020). UmBERTo, a BERT-based model (more specifically RoBERTa-based) …


Evaluating English-language morphological awareness assessments

CL Hudson Kam, E Sadlier-Brown, S Clark, C Jang… - First Language, 2024

CLH Kam, E Sadlier-Brown, S Clark, C Jang, CD Epp… - First Language, 2024

… These corpora were selected because they contain language that children are exposed to, and so, provide a more realistic picture of children’s potential knowledge than an adult-language based corpus would (eg the Enron email dataset …


Evaluating Generative Language Models in Japanese Using Question-Answering Templates: A Synthetic Data Generation Approach

In the last five years, transformer-based large language models have not only made tremendous advancements in fluent and accurate text generation, but also show unparalleled performance as zero-or few-shot shot learners of numerous …


Evaluating GPT and BERT models for Protein-Protein interaction identification in biomedical text

H Rehana, NB Çam, M Basmaci, J Zheng, C Jemiyo… - Bioinformatics Advances, 2024

Detecting protein-protein interactions is crucial for understanding genetic mechanisms, disease pathogenesis, and drug design. As biomedical literature continues to grow rapidly, there is an increasing need for automated and accurate …


Evaluating Large Language Model Performance on Haskell

I introduce HaskellEval, a Haskell evaluation benchmark for Large Language Models. HaskellEval’s curation leverages a novel synthetic generation framework, streamlining the process of dataset curation by minimizing manual intervention. The …


Evaluating Large Language Models for Generalization and Robustness via Data Compression

Y Li, Y Guo, F Guerin, C Lin - arXiv preprint arXiv:2402.00861, 2024

Existing methods for evaluating large language models face challenges such as data contamination, sensitivity to prompts, and the high cost of benchmark creation. To address this, we propose a lossless data compression based evaluation …


Evaluating Large Language Models on Academic Literature Understanding and Review: An Empirical Study among Early-stage Scholars

J Wang, H Hu, Z Wang, Y Song, Y Sheng, D He - 2024

The rapid advancement of large language models (LLMs) such as ChatGPT makes LLM-based academic tools possible. However, little research has empirically evaluated how scholars perform different types of academic tasks with LLMs …


Evaluating LLMs on large contexts: a RAG approach on text comprehension

While the latest Large Language Models (LLMs) continue to expand in size and context window capacity, their knowledge base remains constrained by their training corpus. Retrieval Augmented Generation (RAG) offers a solution to this limitation by …


Evaluating Multilingual Abstractive Dialogue Summarization in Indian Languages using mT5-small & IndicBART

M Sharma, G Goyal, A Gupta, R Rani, A Sharma, A Dev - 2024 IEEE 9th International …, 2024

… The mC4 dataset consists of natural text in 101 languages including Indian languages like Hindi, Mararti, Punjabi and others, that was collected from the public Common Crawl web scrape. Introduced as a multilingual, sequence-to-sequence pre-trained …


Evaluating Natural Monopoly Conditions in the AI Foundation Model Market

The authors of this report examined the economic and production attributes of pre-trained artificial intelligence (AI) foundation models to answer the following questions: Does the market for foundation models have the characteristics of a natural monopoly, and …


Evaluating Prediction-Based Theories of Bilingual Comprehension of Spanish/English Codeswitches.

This dissertation investigates how bilinguals use their two grammars to comprehend written intra-sentential codeswitches. I focus on adjective/noun constructions in Spanish and English where I manipulate the congruence of grammatical word order …


Evaluating Shortest Edit Script Methods for Contextual Lemmatization

O Toporkov, R Agerri - arXiv preprint arXiv:2403.16968, 2024

Modern contextual lemmatizers often rely on automatically induced Shortest Edit Scripts (SES), namely, the number of edit operations to transform a word form into its lemma. In fact, different methods of computing SES have been proposed as an …


Evaluating the Capabilities of GPT-4 in Full-Stack Web Development: A Practical Approach

With the rapid advancements in Artificial Intelligence, leveraging machine learning models for various domains has become increasingly prevalent. This thesis explores the capabilities of OpenAI’s Generative Pre-trained Transformer 4 (GPT-4) within full-stack …


Evaluating the Experience of LGBTQ+ People Using Large Language Model Based Chatbots for Mental Health Support

Z Ma, Y Mei, Y Long, Z Su, KZ Gajos - arXiv preprint arXiv:2402.09260, 2024

LGBTQ+ individuals are increasingly turning to chatbots powered by large language models (LLMs) to meet their mental health needs. However, little research has explored whether these chatbots can adequately and safely provide tailored support …


Evaluating the Factuality of Zero-shot Summarizers Across Varied Domains

S Ramprasad, K Krishna, ZC Lipton, BC Wallace - arXiv preprint arXiv:2402.03509, 2024

Recent work has shown that large language models (LLMs) are capable of generating summaries zero-shot (ie, without explicit supervision) that, under human assessment, are often comparable or even preferred to manually composed …


Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course

S Kahl, F Löffler, M Maciol, F Ridder, M Schmitz… - arXiv preprint arXiv …, 2024

This study evaluates the performance of Large Language Models (LLMs) as an Artificial Intelligence-based tutor for a university course. In particular, different advanced techniques are utilized, such as prompt engineering, Retrieval-Augmented-Generation …


Evaluating the impact of design decisions on passive DNS-based domain rankings

V Le Pochat, S Fernandez, T Van Goethem…

… They recommend that researchers select a random sample of websites among Common Crawl hosts. Ruth et al. [6] compared top lists to Cloudflare traffic data, concluding that the Chrome User Experience Report [11] most accurately represents …


Evaluating the Quality of AI-Generated Items for a Certification Exam

AD Mead, C Zhou - Journal of Applied Testing Technology, 2024

OpenAI’s GPT-3 model can write multiple-choice exam items. This paper reviewed the literature on automatic item generation and then described the recent history of OpenAI GPT models and their operation, and then described a methodology for …


Evaluating Transformers and Linguistic Features integration for Author Profiling tasks in Spanish

JA García-Díaz, G Beydoun, R Valencia-García - Data & Knowledge Engineering, 2024

… This model is a training from scratch of RoBERTa, trained on the Spanish texts from mC4, compiled from the public Common Crawl web … This is a multilingual version of RoBERTa, pre-trained with about 2.5 TB of data from the 100 different …


Evaluation and Adaptation of Neural Language Models for Under-Resourced Languages

Abstract Language models are now commonly used by researchers, industry, and anyone interested. However, language models of all sizes and types are primarily developed for the English language while efforts on other languages lag behind …


Evaluation of AI-generated Responses by Different Artificial Intelligence Chatbots to the Clinical Decision-Making Case-Based Questions in Oral and Maxillofacial …

A Azadi, F Gorjinejad, H Mohammad-Rahimi, R Tabrizi… - Oral Surgery, Oral Medicine …, 2024

Objectives This study aims to evaluate the correctness of the generated answers by Google Bard, GPT-3.5, GPT-4, Claude-Instant, and Bing chatbots to decision-making clinical questions in the oral and maxillofacial surgery (OMFS) area. Study Design A …


Evaluation of Geographical Distortions in Language Models: A Crucial Step Towards Equitable Representations

R Decoupes, R Interdonato, M Roche, M Teisseire… - arXiv preprint arXiv …, 2024

Language models now constitute essential tools for improving efficiency for many professional tasks such as writing, coding, or learning. For this reason, it is imperative to identify inherent biases. In the field of Natural Language Processing …


Evaluation of word embedding models used for diachronic semantic change analysis

Y Maslennikova, V Bochkarev - Journal of Physics: Conference Series, 2024

… As a reference word2vec model, we used pre-trained word vectors that were trained on English versions of Common Crawl dataset using fastText library [14]. fastText is a library for efficient learning of word representations and sentence …


Evidence for a digital divide? Measuring DNS dependencies in the context of the indigenous population of Australia

R Holz, N Nazemi, O Tavallaie, AY Zomaya

We recently presented a work-in-progress paper at the Workshop on Transparency, Accountability and User Control for a Responsible Internet (TAURIN 2023). Our paper investigates the relationship between the digital divide, Internet transparency …


EVLM: An Efficient Vision-Language Model for Visual Understanding

K Chen, D Shen, H Zhong, H Zhong, K Xia, D Xu… - arXiv preprint arXiv …, 2024

In the field of multi-modal language models, the majority of methods are built on an architecture similar to LLaVA. These models use a single-layer ViT feature as a visual prompt, directly feeding it into the language models alongside textual tokens …


Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

Z Chen, L Xu, H Zheng, L Chen, A Tolba, L Zhao, K Yu… - Computers, Materials and …, 2024

Since the 1950s, when the Turing Test was introduced, there has been notable progress in machine language intelligence. Language modeling, crucial for AI development, has evolved from statistical to neural models over the last two decades …


EXAMINING ACCURACY HETEROGENEITIES IN CLASSIFICATION OF MULTILINGUAL AI-GENERATED TEXT

Tools for detection of AI-generated texts are used globally, however, the nature of the apparent accuracy disparities between languages must be further observed. This paper aims to examine the nature of these differences through testing OpenAI’s …


Experimental Evaluation of Possible Feature Combinations for the Detection of Fraudulent Online Shops

A Janavičiūtė, A Liutkevičius, G Dabužinskas… - Applied Sciences, 2024

Online shopping has become a common and popular form of shopping, so online attackers try to extract money from customers by creating online shops whose purpose is to compel the buyer to disclose credit card details or to pay money for …


Expert Phishing Detection System

A Alsarhan, I Al-Aiash, D Al-Fraihat, M Aljaidi… - 2024 IEEE International …, 2024

… and Common Crawl datasets. The highest accuracy reported in this work was 83.00%, achieved with the RF classifier. In the study conducted by [16], the authors employed Machine Learning (ML) and Heuristic methods on three datasets—PhishTank …


Expert-Like Systems Engineering Artifacts? Insights from an Empirical

M Husain, P Wach, TG Topcu - The Proceedings of the 2024 Conference on Systems …

The old saga of complex system development continues as the vast majority of government and industry programs continue to result in cost and schedule overruns [1–7]. While numerous researchers and government reports attribute this trend to the ever-increasing …


Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda

J Schneider - arXiv preprint arXiv:2404.09554, 2024

Generative AI (GenAI) marked a shift from AI being able to recognize to AI being able to generate solutions for a wide variety of tasks. As the generated solutions and applications become increasingly more complex and multi-faceted, novel needs …


Exploring AI-driven approaches for unstructured document analysis and future horizons

SV Mahadevkar, S Patil, K Kotecha, LW Soong… - Journal of Big Data, 2024

In the current industrial landscape, a significant number of sectors are grappling with the challenges posed by unstructured data, which incurs financial losses amounting to millions annually. If harnessed effectively, this data has the potential to …


Exploring Emergent Phenomena in AI: A Pantheistic Approach to the Underlying Source of Information

Emergent phenomena in artificial intelligence (AI) systems, where creative, novel, and unexpected outputs arise, have long intrigued researchers. While these behaviors are typically seen as products of complex computation, this paper …


Exploring Fine-tuned Generative Models for Keyphrase Selection: A Case Study for Russian

A Glazkova, D Morozov - arXiv preprint arXiv:2409.10640, 2024

Keyphrase selection plays a pivotal role within the domain of scholarly texts, facilitating efficient information retrieval, summarization, and indexing. In this work, we explored how to apply fine-tuned generative transformer-based models to the …


Exploring Language Models and Question Answering in Biomedical and Arabic Domains

Despite the success of the Transformer model and its variations (eg, BERT, ALBERT, ELECTRA, T5) in addressing NLP tasks, similar success is not achieved when these models are applied to specific domains (eg, biomedical) and limited-resources …


Exploring large language models for microstructure evolution in materials

P Satpute, S Tiwari, M Gupta, S Ghosh - Materials Today Communications, 2024

… These models are based on massive machine learning (ML) architectures (specifically transformer neural networks [3], [4]) pre-trained on a massive corpus of text data from diverse internet sources like Common Crawl [5] and Wikipedia [6], each …


Exploring LLMs' Capabilities for Error Detection in Dutch L1 and L2 Writing Products

J Kruijsbergen, S Van Geertruyen, V Hoste… - Computational Linguistics in …, 2024

This research examines the capabilities of Large Language Models for writing error detection, which can be seen as a first step towards automated writing support. Our work focuses on Dutch writing error detection, targeting two envisaged end-users …


Exploring Low-Resource Machine Translation: Case Study of Lao-Vietnamese Translation

QD Tran - 2024 International Conference on Multimedia Analysis …, 2024

Despite ongoing research efforts in recent years, machine translation in low-resource settings remains challenging. This paper explores recent methods to enhance low-resource machine translation (LoResMT), especially for both-side low-resource language …


Exploring Machine Learning for Malware Detection With Feature Selection, Explainable AI, and Generative Adversarial Networks

This research focuses on the detection of malware in sample datasets using machine learning algorithms. As malware becomes more advanced against detection in the cybersecurity sector, researchers have looked for other methods to …


Exploring the competence of ChatGPT for customer and patient service management

A Haleem, M Javaid, RP Singh - Intelligent Pharmacy, 2024

The modern language generation model ChatGPT, created by Open Artificial Intelligence (AI), is recognised for its capacity to comprehend context and produce pertinent content. This model is built on the transformer architecture, which enables …


Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models

AMK Mamaghan, S Papa, KH Johansson, S Bauer… - arXiv preprint arXiv …, 2024

Object-centric (OC) representations, which represent the state of a visual scene by modeling it as a composition of objects, have the potential to be used in various downstream tasks to achieve systematic compositional generalization and facilitate …


Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning

Y Wang, W Chen, X Han, X Lin, H Zhao, Y Liu, B Zhai… - arXiv preprint arXiv …, 2024

Strong Artificial Intelligence (Strong AI) or Artificial General Intelligence (AGI) with abstract reasoning ability is the goal of next-generation AI. Recent advancements in Large Language Models (LLMs), along with the emerging field of Multimodal Large …


Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varieties

E Artemova, V Blaschke, B Plank - arXiv preprint arXiv:2402.02078, 2024

Mainstream cross-lingual task-oriented dialogue (ToD) systems leverage the transfer learning paradigm by training a joint model for intent recognition and slot-filling in English and applying it, zero-shot, to other languages. We address a gap in prior …


Exploring the Role of ChatGPT in Cardiology: A Systematic Review of the Current Literature

A Sharma, T Medapalli, M Alexandrou, E Brilakis… - Cureus, 2024

Chat Generative Pre-Trained Transformer (ChatGPT) is a chatbot based on a large language model that has gained public interest since its release in November 2022. This systematic review examines the current literature on the potential applications …


Exploring the role of ChatGPT in higher education: opportunities, challenges and ethical considerations

A Zeb, R Ullah, R Karim - The International Journal of Information and Learning …, 2024

Purpose This paper aims to examine the opportunities and challenges of using ChatGPT in higher education. Furthermore, it is also discuss the potential risks and plunders of these tools. Design/methodology/approach The paper discuss the use of …


Exploring transformer models in the sentiment analysis task for the under-resource Bengali language

MN Hoque, U Salma, MJ Uddin, MM Ahamad, S Aktar - Natural Language Processing …, 2024

In the sentiment analysis (SA) task, we can obtain a positive or negative-typed comment or feedback from an online user or a customer about any object, such as a movie, drama, food, and others. This user’s sentiment may positively impact various …


Exploring very low-resource translation with llms: The university of edinburgh's submission to americasnlp 2024 translation task

V Iyer, B Malik, W Zhu, P Stepachev, P Chen… - Proceedings of the 4th …, 2024

This paper describes the University of Edinburgh’s submission to the AmericasNLP 2024 shared task on the translation of Spanish into 11 indigenous American languages. We explore the ability of multilingual Large Language Models (LLMs) to …


EXPRESS: AI-Human Hybrids for Marketing Research: Leveraging LLMs as Collaborators

N Arora, I Chakraborty, Y Nishimura - Journal of Marketing, 2024

The authors’ central premise is that a human-LLM hybrid approach leads to efficiency and effectiveness gains in the marketing research process. In qualitative research, they show that LLMs can assist in both data generation and analysis; …


EXPRESS: Malay Lexicon Project 3: The Impact of Orthographic-Semantic Consistency on Lexical Decision Latencies

M Maziyah Mohamed, D Jared - Quarterly Journal of Experimental Psychology, 2024

Theories of word processing propose that readers are sensitive to statistical co-occurrences between spelling and meaning. Orthographic-Semantic Consistency (OSC) measures provide a continuous estimate of the statistical regularities between …


Extending the Comparative Argumentative Machine: Multilingualism and Stance Detection

I Nikishina, A Bondarenko, S Zaczek, OL Haag…

The comparative argumentative machine CAM can retrieve arguments that answer comparative questions—questions that ask which of several to-be-compared options should be favored in some scenario. In this paper, we describe how we equipped …


Extending Translate-Train for ColBERT-X to African Language CLIR

E Yang, DJ Lawrie, P McNamee, J Mayfield - arXiv preprint arXiv:2404.08134, 2024

… We used Common Crawl documents in Afriberta Corpus [6] for the four African languages and collected additional English Common Crawl documents to match the genre. We fine-tune the model for 200,000 update steps using a learning rate of 1×10-5 …


External Attack-Surface of Modern Organizations

N Gelernter, H Schulmann, M Waidner - Proceedings of the 19th ACM Asia …, 2024

Navigating the maze of contemporary organizational attack surfaces is paramount in fortifying our defenses against the relentless tide of cyber incidents. However, existing network reconnaissance and security measurements, which enumerate IP …


Extracting emotion from resource poor language through transfer learning

A Majeed, U Imtiaz, MA Nseem, M Aleem, W Shahzad… - Multimedia Tools and …, 2024

… Common Crawl materials in over 100 languages are used to train the algorithm. XLM-Roberta has seen a significant increase in training data since its first release. Several cross-language benchmarks show that XLM-R performs the best. …


Extracting intersectional stereotypes from embeddings: Developing and validating the Flexible Intersectional Stereotype Extraction procedure

TES Charlesworth, K Ghate, A Caliskan, MR Banaji - PNAS Nexus, 2024

… text corpora, ranging from static GloVe embeddings trained on 840 billion words from the Common Crawl to contextualized BERT embeddings trained on a combination of Wikipedia and Common Crawl text. Additionally, in supplemental …


Factuality of Large Language Models in the Year 2024

Y Wang, M Wang, MA Manzoor, G Georgiev, RJ Das… - arXiv preprint arXiv …, 2024

Large language models (LLMs), especially when instruction-tuned for chat, have become part of our daily lives, freeing people from the process of searching, extracting, and integrating information from multiple sources by offering a …


FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplication

E Slyman, S Lee, S Cohen, K Kafle - arXiv preprint arXiv:2404.16123, 2024

Recent dataset deduplication techniques have demonstrated that content-aware dataset pruning can dramatically reduce the cost of training Vision-Language Pretrained (VLP) models without significant performance losses compared to …


Fairness and Bias in Multimodal AI: A Survey

T Adewumi, L Alkhaled, N Gurung, G van Boven… - arXiv preprint arXiv …, 2024

The importance of addressing fairness and bias in artificial intelligence (AI) systems cannot be over-emphasized. Mainstream media has been awashed with news of incidents around stereotypes and bias in many of these systems in recent years. In …


Fake news detection in low-resource languages: A novel hybrid summarization approach

J Alghamdi, Y Lin, S Luo - Knowledge-Based Systems, 2024

… Pre-training encompasses a vast dataset exceeding 2TB of CommonCrawl data. The overarching concept involves mapping any given language to a language-agnostic vector space, ensuring that inputs in various languages converge to corresponding …


Falcon2-11B Technical Report

Q Malartic, NR Chowdhury, R Cojocaru, M Farooq… - arXiv preprint arXiv …, 2024

We introduce Falcon2-11B, a foundation model trained on over five trillion tokens, and its multimodal counterpart, Falcon2-11B-vlm, which is a vision-to-text model. We report our findings during the training of the Falcon2-11B which follows a multi-stage …


Feasibility of Big Data Analytics in Disaster Psychiatry: Impact of Seoul Itaewon Tragedy on Sentiment Distribution on Twitter

J Kim, M Ki, J Yang, C Na, J Kim, C Han - Social Science & Medicine, 2024

Numerous studies have highlighted the significant impact of disasters on mental health, often leading to psychiatric disorders among affected individuals. Timely identification of disaster-related mental health problems is crucial to prevent long-term …


Feature-augmented model for multilingual discourse relation classification

E Metheniti, C Braud, P Muller - Proceedings of the 5th Workshop on Computational …, 2024

Discourse relation classification within a multilingual, cross-framework setting is a challenging task, and the best-performing systems so far have relied on monolingual and mono-framework approaches. In this paper, we introduce transformer-based …


Federated learning for privacy-preserving depression detection with multilingual language models in social media posts

SS Khalil, NS Tawfik, M Spruit - Patterns, 2024

The incidences of mental health illnesses, such as suicidal ideation and depression, are increasing, which highlights the urgent need for early detection methods. There is a growing interest in using natural language processing (NLP) models to analyze …


Few shot clinical entity recognition in three languages: Masked language models outperform LLM prompting

M Naguib, X Tannier, A Névéol - arXiv preprint arXiv:2402.12801, 2024

Large Language Models are becoming the go-to solution for many natural language processing tasks, including in specialized domains where their few-shot capacities are expected to yield high performance in low-resource settings. Herein, we aim to …


Final Thoughts: Digital Humanities Looking at Generative AI

M Aguiar, S Araújo - Digital Humanities Looking at the World: Exploring …, 2024

In this chapter, we examine generative artificial intelligence in the context of Digital Humanities. We commence by providing a concise overview of this technology and its prevalent models. Following that, we provide a brief survey of the existing …


Final Words

M McTear, M Ashurkina - Transforming Conversational AI: Exploring the Power …, 2024

Conversational AI is a dynamic and fast moving field, and a lot has happened in the six months or so since we began writing this book. We have tried to ensure that what we have covered in the preceding chapters provides a sufficiently general and …


Fine grain emotion analysis in Spanish using linguistic features and transformers

A Salmerón-Ríos, JA García-Díaz, R Pan… - PeerJ Computer Science, 2024

Mental health issues are a global concern, with a particular focus on the rise of depression. Depression affects millions of people worldwide and is a leading cause of suicide, particularly among young people. Recent surveys indicate an increase in …


Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendations

K VM, H Warrier, Y Gupta - arXiv preprint arXiv:2404.10779, 2024

… The datasets are massive text corpus like Common crawl, The Pile and code repositories from Github. Further the model can be fine tuned for specific tasks using specialised datasets. One such series of datasets are called Instruction datasets like …


Fine Tuning LLMs for Low Resource Languages

S Joshi, MS Khan, A Dafe, K Singh, V Zope, T Jhamtani - 2024 5th International …, 2024

Large Language Models (LLMs) hold immense potential, but their data hunger can limit its performance in processing languages with limited resources. This research study explores the techniques for fine-tuning LLMs specifically for low-resource …


Fine-Tuned Large Language Models for Symptom Recognition from Spanish Clinical Text

MA Shaaban, A Akkasi, A Khan, M Komeili, M Yaqub - arXiv preprint arXiv …, 2024

… XLM-RL (561M parameters) and XLM-RB (279M parameters) are multilingual masked language models, trained on a filtered CommonCrawl dataset of over two terabytes. We leverage BBS and BBES both pretrained on Spanish clinical data …


Fine-Tuned Understanding: Enhancing Social Bot Detection with Transformer-based Classification

A Sallah, S Agoujil, MA Wani, M Hammad… - IEEE Access, 2024

… fined common crawl data, XLM-RoBERTa is purposefully crafted as a multilingual version of RoBERTa. The acronym "XLM" denotes its capabilities in crosslingual language modeling; 3… It is trained on 2.5 TB of freshly curated, clean …


FINE-TUNING BERT, DISTILBERT, XLM-ROBERTA AND UKR-ROBERTA MODELS FOR SENTIMENT ANALYSIS OF UKRAINIAN LANGUAGE REVIEWS

… It is pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages. The architecture of the XLMRoBERTa base model consists of 12 layers, 12 attention heads with 768 dimensions, and a feed-forward network [19]. This model is trained …


Fine-tuning BERT-based Models for Negative Content Identification on Indonesian Tweets

AF Hidayatullah, K Kalinaki, MM Aslam, RY Zakari…

Social media platforms like Twitter have become substantial sources of user-generated content, enabling people to easily express their emotions and opinions. However, this freedom has increased the spread of harmful content, such as abusive language …


Fine-tuning Large Language Models for Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection

F Xiong, T Markchom, Z Zheng, S Jung, V Ojha… - arXiv preprint arXiv …, 2024

SemEval-2024 Task 8 introduces the challenge of identifying machine-generated texts from diverse Large Language Models (LLMs) in various languages and domains. The task comprises three subtasks: binary classification in monolingual …


Fine-tuning large language models: from accuracy enhancement to bias mitigation

As the frontier of artificial intelligence (AI) continues to expand, Large Language Models (LLMs) have emerged as pivotal tools across a broad spectrum of applications. These sophisticated AI systems, powered by vast amounts of data and …


FINETUNING COMMERCIAL LARGE LANGUAGE MODELS WITH LORA FOR ENHANCED ITALIAN LANGUAGE UNDERSTANDING

J Hartsuiker, P Torroni, AE Ziri, DF Alise, F Ruggeri

The world of Artificial intelligence is booming, especially the fields of natural language processing (NLP) and Computer Vision (CV). Recently the chatGPT models from OpenAI have caused another shock-wave. This was a clear sign to the …


Finite State Automata on Multi-Word Units for Efficient Text-Mining

Text mining is crucial for analyzing unstructured and semi-structured textual documents. This paper introduces a fast and precise text mining method based on a finite automaton to extract knowledge domains. Unlike simple words, multi-word …


FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models

G Bhatia, EMB Nagoudi, H Cavusoglu… - arXiv preprint arXiv …, 2024

We introduce FinTral, a suite of state-of-the-art multimodal large language models (LLMs) built upon the Mistral-7b model and tailored for financial analysis. FinTral integrates textual, numerical, tabular, and image data. We enhance FinTral with domain-specific …


Foregrounding Artist Opinions: A Survey Study on Transparency, Ownership, and Fairness in AI Generative Art

J Lovato, J Zimmerman, I Smith, P Dodds, J Karson - arXiv preprint arXiv:2401.15497, 2024

Generative Artificial Intelligence (AI) tools are used to create art-like outputs and aid in the creative process. While these tools have potential benefits for artists, they also have the potential to harm the art workforce and infringe upon artistic and intellectual …


Fostering the Ecosystem of Open Neural Encoders for Portuguese with Albertina PT* Family

R Santos, J Rodrigues, L Gomes, J Silva, A Branco… - arXiv preprint arXiv …, 2024

… The OSCAR subset for Portuguese we use is based on November/December 2022 version of Common Crawl, which is an automatic crawl from the web. Despite being a crawl, the final dataset is of relatively good quality due the filtering performed …


Foundation Models for Autonomous Robots in Unstructured Environments

H Naderi, A Shojaei - arXiv preprint arXiv:2407.14296, 2024

Automating activities through robots in unstructured environments, such as construction sites, has been a long-standing desire. However, the high degree of unpredictable events in these settings has resulted in far less adoption compared to …


Frequently asked questions on erectile dysfunction: evaluating artificial intelligence answers with expert mentorship

M Baturu, M Solakhan, TG Kazaz, O Bayrak - International Journal of Impotence …, 2024

The present study assessed the accuracy of artificiaI intelligence-generated responses to frequently asked questions on erectile dysfunction. A cross-sectional analysis involved 56 erectile dysfunction-related questions searched on Google …


From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models

W Messner, T Greene, J Matalone - arXiv preprint arXiv:2312.17256, 2023

Large language models (LLMs) are able to engage in natural-sounding conversations with humans, showcasing unprecedented capabilities for information retrieval and automated decision support. They have disrupted human-technology …


From Cosine Similarity to Likelihood Ratio: Coupling Representations From Machine Learning (and Other Sources) With Cognitive Models

Modern machine learning models yield vector representations that capture similarity relations between complex items like text and images. These representations can help explain and predict how individuals respond to those items in particular tasks …


From Data Creator to Data Reuser: Distance Matters

CL Borgman, PT Groth - arXiv preprint arXiv:2402.07926, 2024

… Common Crawl, launched in 2007, now spends roughly $200,000 per year to maintain petabytes of openly available web crawl data (Common Crawl, 2023; Roberts, 2023). The use of these data to train large language models has …


From Data Quality to Model Performance: Navigating the Landscape of Deep Learning Model Evaluation

M Akram, WH Moosa - Deep Learning for Multimedia Processing Applications, 2024

Deep learning (DL) technologies have revolutionized the analysis of multimedia data, with natural language processing, visual data analytics, and audio recognition being just a few examples of multimedia applications that have effectively utilized DL. This …


From Development to Dissemination: Social and Ethical Issues with Text-to-Image AI-Generated Art

… Model developed by the Computer Vision & Learning Group (CompVis) lab at Ludwig Maximilian University of Munich was trained with a subset of the LAION-5B dataset [6], comprising of about 2.3 billion CLIP-filtered image-text pairs by parsing …


From Form (s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

X Ohmer, E Bruni, D Hupkes - arXiv preprint arXiv:2404.12145, 2024

… We use the current common crawl statistics6 to compute an estimate of how low- or high-resource these languages are in web-based corpora. Of this corpus, English constitutes 46% of the data, German 5.8%, Italian 2.7%, Dutch 2.2%, and Swedish …


From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients

A Jaiswal, L Yin, Z Zhang, S Liu, J Zhao, Y Tian… - arXiv preprint arXiv …, 2024

… The C4 dataset is a massive collection of Common Crawl’s web crawl corpus, meticulously filtered and cleaned to ensure high-quality language modeling and training. For downstream task finetuning of compressed models, we consider a good …


From Graph Theory for Robust Deep Networks to Graph Learning for Multimodal Cancer Analysis

This dissertation explores the intersection of graph theory and deep learning, focusing on enhancing the robustness of deep neural networks (DNNs) and applying these advancements to complex problems like cancer diagnosis and …


From ML to LLM: Evaluating the Robustness of Phishing Webpage Detection Models against Adversarial Attacks

A Kulkarni, V Balachandran, DM Divakaran, T Das - arXiv preprint arXiv:2407.20361, 2024

Phishing attacks attempt to deceive users into stealing sensitive information, posing a significant cybersecurity threat. Advances in machine learning (ML) and deep learning (DL) have led to the development of numerous phishing webpage detection …


From News to Summaries: Building a Hungarian Corpus for Extractive and Abstractive Summarization

B Barta, D Lakatos, A Nagy, MK Nyist, J Ács - arXiv preprint arXiv:2404.03555, 2024

… We use the freely available Common Crawl dataset4 as a basis for constructing the corpus. It contains petabytes of crawled web pages from the past 25 years and it is available on Amazon S3 in WARC format. Retrieval and deduplication of the raw …


From Pixels to Prose: A Large Dataset of Dense Image Captions

V Singla, K Yue, S Paul, R Shirkavand… - arXiv preprint arXiv …, 2024

Training large vision-language models requires extensive, high-quality image-text pairs. Existing web-scraped datasets, however, are noisy and lack detailed image descriptions. To bridge this gap, we introduce PixelProse, a comprehensive dataset …


From Populism to Platforms: Antitrust Law and the AI Revolution

This chapter, from my book manuscript “The Political Economy of the Fourth Industrial Revolution,” examines how changes to US antitrust law catalyzed the rise of digital platforms and the Artificial Intelligence (AI) Revolution. It traces the …


From Rule-Based Models to Deep Learning Transformers Architectures for Natural Language Processing and Sign Language Translation Systems: Survey, Taxonomy …

N Shahin, L Ismail - arXiv preprint arXiv:2408.14825, 2024

With the growing Deaf and Hard of Hearing population worldwide and the persistent shortage of certified sign language interpreters, there is a pressing need for an efficient, signs-driven, integrated end-to-end translation system, from sign to gloss to …


From Text to Transformation: A Comprehensive Review of Large Language Models' Versatility

P Kaur, GS Kashyap, A Kumar, MT Nafis, S Kumar… - arXiv preprint arXiv …, 2024

This groundbreaking study explores the expanse of Large Language Models (LLMs), such as Generative Pre-Trained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) across varied domains ranging from …


From'Showgirls' to'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMs

M Bartl, S Leavy - arXiv preprint arXiv:2407.04434, 2024

Gender bias is not only prevalent in Large Language Models (LLMs) and their training data, but also firmly ingrained into the structural aspects of language itself. Therefore, adapting linguistic structures within LLM training data to promote gender-inclusivity …


Frozen or Fine-tuned? Analyzing Deep Learning Models and Training Strategies for Optimizing Big Five Personality Traits Prediction from Text

M Soleimani, HB Kashani - 2024 10th International Conference on Artificial …, 2024

In today’s digital age, the comprehension and prediction of human personality traits have assumed paramount significance. This study embarks on the task of forecasting the Big Five personality traits through textual data, harnessing the …


FuLG: 150B Romanian Corpus for Language Model Pretraining

VA Bădoiu, MV Dumitru, AM Gherghescu, A Agache… - arXiv preprint arXiv …, 2024

… billion-token Romanian corpus extracted from CommonCrawl. We present our methodology for … , we delve into filtering Romanian content from CommonCrawl, documenting the process and … Second, we document the process of filtering data …


Fundus: A Simple-to-Use News Scraper Optimized for High Quality Extractions

M Dallabetta, C Dobberstein, A Breiding, A Akbik - arXiv preprint arXiv:2403.15279, 2024

… Specifically, we support the CC-NEWS3 dataset provided by the COMMONCRAWL initiative. At the time of writing, this dataset comprises around 40 terabytes of WARC-formatted data, containing millions of news articles dating back …


Fusing Domain-Specific Content from Large Language Models into Knowledge Graphs for Enhanced Zero Shot Object State Classification

F Gouidis, K Papantoniou, KPT Patkos, A Argyros… - arXiv preprint arXiv …, 2024

Domain-specific knowledge can significantly contribute to addressing a wide variety of vision tasks. However, the generation of such knowledge entails considerable human labor and time costs. This study investigates the potential of Large Language …


Fusion of Deep Learning with Advanced and Traditional Embeddings in Sentiment Analysis

K Kiran, PD Shenoy - 2024 IEEE 9th International Conference for …, 2024

In todays changing landscape the abundance of unprocessed data plays a vital role. It is crucial to curate this dataset and conduct a thorough analysis of the sentiments it captures. The aim of this research study is to explore learning techniques such as Bi …


Fusion of Knowledge: Enhancing AI Reasoning Through Language Models and Knowledge Graphs

Large Language Models (LLMs) and Knowledge Graphs (KGs) have rapidly emerged as important areas in Artificial Intelligence (AI). LLMs leverage vast amounts of unstructured text to understand and generate natural language. KGs are …


FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data

H Sun, R Jin, S Xu, L Pan, M Cui, J Dui, Y Lei, L Yang… - arXiv preprint arXiv …, 2024

Large language models (LLMs) have demonstrated prowess in a wide range of tasks. However, many LLMs exhibit significant performance discrepancies between highand low-resource languages. To mitigate this challenge, we present FuxiTranyu, an …


GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

J Zhao, Z Zhang, B Chen, Z Wang, A Anandkumar… - arXiv preprint arXiv …, 2024

Training Large Language Models (LLMs) presents significant memory challenges, predominantly due to the growing size of weights and optimizer states. Common memory-reduction approaches, such as low-rank adaptation (LoRA), add a trainable …


Gauging Novelty in Crowdfunding Projects: A Theory-Driven Text Analysis Approach

The explosion of crowdfunding projects has called for new ways to screen for novel projects. Building on the multidimensional nature of novelty, we draw on the literature to propose two measures of project novelty that are theoretically sound and …


GEB-1.3 B: Open Lightweight Large Language Model

J Wu, Y Zhu, L Shen, X Lu - arXiv preprint arXiv:2406.09900, 2024

… Data Processing The Common Crawl dataset is crawled from the web and often suffers from quality issues, because of including garbled … Rule Cleaning First, we extract a Chinese corpus from the Common Crawl dataset, by applying specialized …


GenDec: A robust generative Question-decomposition method for Multi-hop reasoning

J Wu, L Yang, Y Ji, W Huang, BF Karlsson, M Okumura - arXiv preprint arXiv …, 2024

Multi-hop QA (MHQA) involves step-by-step reasoning to answer complex questions and find multiple relevant supporting facts. However, Existing large language models'(LLMs) reasoning ability in multi-hop question answering remains exploration, which is …


Gender Bias in Natural Gender Language and Grammatical Gender Language within Children's Literature

There has been much research on the connection between language and gender bias but there is little comparing natural gender language, grammatical gender language, and gender bias. This research is important because it can offer an …


Gender Identity and Representation in the Context of Economic Development in India

… It is a large multi-lingual language model, trained on 2.5TB of filtered CommonCrawl data. It represents some of the most sophisticated pre-trained models prevalently being used in natural language processing for text classification …


Gendered Grammar or Ingrained Bias? Exploring Gender Bias in Icelandic Language Models

SR Friðriksdóttir, H Einarsson - Proceedings of the 2024 Joint International …, 2024

Large language models, trained on vast datasets, exhibit increased output quality in proportion to the amount of data that is used to train them. This data-driven learning process has brought forth a pressing issue where these models may not only reflect …


Gendered Language in Job Advertisements Relates to Gender Sorting in Public Labor Markets: A Multi-Source Analysis

Increasing gender diversity constitutes a desirable goal for policymakers and recruiters in public organizations. However, contemporary research lacks focus on gender sorting, referring to structural self-selection among job seekers in the public …


General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model

H Wei, C Liu, J Chen, J Wang, L Kong, Y Xu, Z Ge… - arXiv preprint arXiv …, 2024

… For the document-level data, we first collect open-source PDF-style files from the Common Crawl and employ the Fitz Python package to extract corresponding dense text content. In such a process, we gain 1.2M full-page PDF-style image-text pairs …


Generating Apparel Images by Using Stable Diffusion with Prompt Recommendation

XZ Liu, HM Hung, LH Chen - 人工知能学会全国大会論文集 第 38 回 (2024), 2024

Fashion is a profound expression of personal identity, style, and culture in daily life. Making oneself look attractive, which often equates to being fashionable or stylish, is always a priority for many people. In this paper, we proposed a clothing …


Generating Character Lines in Four-Panel Manga

M Inaba - Proceedings of the 37th Pacific Asia Conference on …, 2023

Automatic content generation based on natural language processing is an active research area, especially for story generation. Research on story generation has focused on generating consistent text pertaining to characters’ actions and events; …


Generating Probabilistic Scenario Programs from Natural Language

K Elmaaroufi, D Shankar, A Cismaru… - arXiv preprint arXiv …, 2024

… Most of these model’s are fine-tuned variants of a foundational model that was trained on a wider body of text such as Common Crawl. For instance Codex is ”a GPT language model fine-tuned on publicly available code from GitHub” and it’s …


Generation of API Documentation using Large Language Models–Towards Self-explaining APIs

Application Programming Interfaces (APIs) play a crucial role in modern software development. They are key in not only in enabling software reuse and adaptation, which greatly increase productivity, but also in facilitating exploration of new …


Generative adversarial network-based phishing URL detection with variational autoencoder and transformer

JK Sasi, A Balakrishnan - Int J Artif Intell, 2024

Phishing attacks pose a constant threat to online security, necessitating the development of efficient tools for identifying malicious URLs. In this article, we propose a novel approach to detect phishing URLs employing a generative …


Generative AI and Large Language Models for Cyber Security: All Insights You Need

MA Ferrag, F Alwahedi, A Battah, B Cherif, A Mechri… - arXiv preprint arXiv …, 2024

This paper provides a comprehensive review of the future of cybersecurity through Generative AI and Large Language Models (LLMs). We explore LLM applications across various domains, including hardware design security, intrusion detection …


Generative AI and the Continuing Importance of Information Literacy

GAI models are embedded in a variety of applications. These models are trained using existing information, which is then formatted and delivered within the context of the GAI application. Bias, misand disinformation, and ethical use of information …


Generative AI for Math: Part I--MathPile: A Billion-Token-Scale Pretraining Corpus for Math

Z Wang, R Xia, P Liu - arXiv preprint arXiv:2312.17120, 2023

… expanse of Common Crawl snapshots, a venture we aim to pursue in our future work. … We provide a near-duplicate example found in Common Crawl, as shown in 2 (See Table 8 to … StackExchange that appeared in a document from Common …


Generative AI for Unmanned Vehicle Swarms: Challenges, Applications and Opportunities

G Liu, N Van Huynh, H Du, DT Hoang, D Niyato, K Zhu… - arXiv preprint arXiv …, 2024

With recent advances in artificial intelligence (AI) and robotics, unmanned vehicle swarms have received great attention from both academia and industry due to their potential to provide services that are difficult and dangerous to perform by humans …


Generative AI from Theory to Practice: A Case Study of Financial Advice

We identify some of the most pressing issues facing the adoption of large language models (LLMs) in practical settings and propose a research agenda to reach the next technological inflection point in generative AI. We focus on three challenges …


Generative AI in Education From the Perspective of Students, Educators, and Administrators

This research explores how advanced artificial intelligence (AI), like the technology that powers tools such as ChatGPT, is changing the way we teach and learn in schools and universities. Imagine AI helping to summarize thick legal documents …


Generative AI

… • CommonCrawl News: This is a dataset of news stories from websites across the globe. …


Generative AI, Plagiarism, and Copyright Infringement in Legal Documents

AB Cyphert - Minnesota Journal of Law, Science & Technology, 2024

… OpenAI acknowledged in the GPT-3 research paper that GPT-3 was trained on the Common Crawl dataset, “which includes everything from traditional news sites like the New York Times to sites like Reddit.” Cyphert, supra note 1, at 407, citing Liz …


Generative AI-Augmented Decision-Making for Business Information Systems

E Kromidha, RM Davison - IFIP International Conference on Human Choice and …, 2024

… This is only influenced by the data it has been trained on, and its users which is one of its main limitations as ChatGPT was trained on datasets like Wikipedia, BookCorpus for books, and Common Crawl for web content which are mainly in …


Generative AI-Based Text Generation Methods Using Pre-Trained GPT-2 Model

R Pandey, H Waghela, S Rakshit, A Rangari, A Singh… - arXiv preprint arXiv …, 2024

This work delved into the realm of automatic text generation, exploring a variety of techniques ranging from traditional deterministic approaches to more modern stochastic methods. Through analysis of greedy search, beam search, top-k …


GENERATIVE AI-POWERED FRAMEWORK

K VAYADANDE - Deep Learning Model Optimization, Deployment and …, 2024

Artificial intelligence (AI) tools encompass a diverse array of applications and software designed to simulate intelligent behavior, facilitate decision making, and automate tasks across various domains. These tools leverage artificial intelligence …


Generative Artificial Intelligence (AI) for the Data Economy: Use Cases in Nursing

… The core recent AIGC advances stem from training more sophisticated generative models on larger datasets; for example, while the main framework maintains the same as GPT-2, GPT-3’s pre-training data size grows from WebTex (38GB) to …


GENERATIVE ARTIFICIAL INTELLIGENCE (AI) IN EDUCATION: A CROSS-NATIONAL SURVEY ON UNIVERSITY TEACHERS'PERCEPTIONS ON THE USE OF …

… A year and three months later, the GPT-3 emerged with the Common Crawl as the main source of its data. The current GPT-4 started in March 2023 with the “unpublished” data source (Wu et al., 2023, p. 1123). …


Generative Artificial Intelligence-Guided User Studies: An Application for Air Taxi Services

S Xiao, J Li, T Fushimi, Y Ochiai - arXiv preprint arXiv:2406.12296, 2024

User studies are crucial for meeting user needs. In user studies, real experimental scenarios and participants are constructed and recruited. However, emerging and unfamiliar studies face limitations, including safety concerns and iterative efficiency …


Generative Artificial Intelligence: Models, Benefits, Dangers and Detection of AI-Generated Text on Specialized Domains

… However, Radford et al. found that a large amount of document within the hashed pages of Common Crawl was unintelligible. A new approach could be to simply use a curated subset of Common Crawl, but they also created a web scrape from Reddit …


Generative Knowledge Management for Financial Inclusion Through Financial Literacy: A Systematic Review.

S Vadari, C Malladi - IUP Journal of Knowledge Management, 2024

… OpenAI utilized an openly available corpus of webpages called Common Crawl, which in turn derives from a bot that crawls the entire Internet. This Common Crawl dataset contains billions of webpages and is one of the colossal textual datasets …


GeniL: A Multilingual Dataset on Generalizing Language

AM Davani, S Gubbi, S Dev, S Dave, V Prabhakaran - arXiv preprint arXiv …, 2024

… For each language, we queried the Multilingual Common Crawl (mC4) language corpus to collect naturally occurring sentences that contain both the terms in the tuples present in SGM. To ensure a diverse representation of different associations …


German Text Embedding Clustering Benchmark

S Wehrli, B Arnrich, C Irrgang - arXiv preprint arXiv:2401.02709, 2024

This work introduces a benchmark assessing the performance of clustering German text embeddings in different domains. This benchmark is driven by the increasing use of clustering neural text embeddings in tasks that require the grouping of texts (such …


GFWeb: Measuring the Great Firewall's Web Censorship at Scale

NP Hoang, J Dalek, M Crete-Nishihata, N Christin…

… Aggregating the ranking information provided by the Tranco list [62] and the Common Crawl dataset [1], we analyze the popularity of the base censored domains discovered by GFWeb for HTTP(S) filters in comparison to the GFWatch (DNS filter) …


Ghost Sentence: A Tool for Everyday Users to Copyright Data from Large Language Models

S Zhao, L Zhu, R Quan, Y Yang - arXiv preprint arXiv:2403.15740, 2024

Web user data plays a central role in the ecosystem of pre-trained large language models (LLMs) and their fine-tuned variants. Billions of data are crawled from the web and fed to LLMs. How can \textit{\textbf{everyday web users}} confirm if LLMs …


Goldfish: Monolingual Language Models for 350 Languages

TA Chang, C Arnett, Z Tu, BK Bergen - arXiv preprint arXiv:2408.10441, 2024

For many low-resource languages, the only available language models are large multilingual models trained on many languages simultaneously. However, using FLORES perplexity as a metric, we find that these models perform worse than …


GPT‐3‐and DALL‐E‐Powered Applications: A Complete Survey

K Vayadande, CB Pednekar, PA Khune… - How Machine Learning is …, 2024

In the new millennium, the advancement of language and image processing technologies has witnessed remarkable progress, and various applications have demonstrated promising results by utilizing sophisticated models such as GPT‐3 …


GPTs: Concerns, Limitations and (Some) Responses

… The ones of these that are large and containing substantial source material from the Internet (eg from Common Crawl) will have some harms of representation bias. Then, if the training data has bias then so too will LLMs. To address this, there seem …


Grammatical versus Spelling Error Correction: An Investigation into the Responsiveness of Transformer-Based Language Models Using BART and MarianMT

R Raju, PB Pati, SA Gandheesh, GS Sannala… - Journal of Information & …, 2024

Text continues to remain a relevant form of representation for information. Text documents are created either in digital native platforms or through the conversion of other media files such as images and speech. While the digital native text is …


Grammatical vs Spelling Error Correction: An Investigation into the Responsiveness of Transformer-based Language Models using BART and MarianMT

R Raju, PB Pati, SA Gandheesh, GS Sannala, S KS - arXiv preprint arXiv:2403.16655, 2024

… C4 Dataset is an Open-Source dataset obtained with Common Crawl web scrape [30]. This dataset contains millions of collected sentences along with their target sentences. The collected statements, referred to as input sentences in this work …


Granite Code Models: A Family of Open Foundation Models for Code Intelligence

M Mishra, M Stallone, G Zhang, Y Shen, A Prasad… - arXiv preprint arXiv …, 2024

Large Language Models (LLMs) trained on code are revolutionizing the software development process. Increasingly, code LLMs are being integrated into software development environments to improve the productivity of human programmers, and …


Granularity is crucial when applying differential privacy to text: An investigation for neural machine translation

DNL Vu, T Igamberdiev, I Habernal - arXiv preprint arXiv:2407.18789, 2024

… The computation of b is carried out using Common Crawl11 monolingual datasets. For each language and contextual embedding model, one million candidate-reference pairs are created by grouping two random sentences. Due to the random pairing …


Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions

YG Hsieh, CY Hsieh, SY Yeh, L Béthune, HP Ansari… - arXiv preprint arXiv …, 2024

… In a similar vein, Meta-CLIP [68] reproduces the processing of the seminal CLIP paper [50] on a subset of the Common Crawl dataset, … to filter data and remove duplicates, while DFN [17] uses filtering networks trained on high quality data to …


Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients

A Muhamed, O Li, D Woodruff, M Diab, V Smith - arXiv preprint arXiv:2406.17660, 2024

Large language model (LLM) training and finetuning are often bottlenecked by limited GPU memory. While existing projection-based optimization methods address this by projecting gradients into a lower-dimensional subspace to reduce optimizer …


GreekT5: Sequence-to-Sequence Models for Greek News Summarization

N Giarelis, C Mastrokostas, N Karacapilidis - IFIP International Conference on …, 2024

… mBART was pre-trained for multiple languages using the CC25 dataset [15], which contains documents from 25 different languages from Common Crawl. The authors have also published an updated mBART model, which supports 50 natural …


Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation

S Xia, X Wang, J Liang, Y Zhang, W Zhou, J Deng, F Yu… - arXiv preprint arXiv …, 2024

Retrieval-Augmented Generation (RAG) has been widely adopted to enhance Large Language Models (LLMs) in knowledge-intensive tasks. Recently, Attributed Text Generation (ATG) has attracted growing attention, which provides citations to …


Grounded Content Automation: Generation and Verification of Wikipedia in Low-Resouce languages.

In this thesis, we work towards improving the representation of low-resource languages in the digital world by easing the access and participation of these communities to reliable information hubs like Wikipedia. Although the internet has …


Grounding Language in Images and Videos

While machine learning research has traditionally explored image, video and text understanding as separate fields, the surge in multi-modal content in today’s digital landscape underscores the importance of computation models that adeptly navigate …


Growing Smaller Language Models Using Knowledge Distillation from Larger Models

M Featherstone, E Cuthbertson, D Appleyard, S Gittins

… Publicly available datasets, such as The Pile and Common Crawl, were employed to ensure a comprehensive and representative sample of language usage across various domains. Data preprocessing involved several crucial steps …


Hacker News new| past| comments| ask| show| jobs| submit login

I don't think these dichotomies in quality are going to go away. It's the exact same thing in every neural net domain. Self driving is probably the most sophisticated domain besides chat, and there too it's the exact same problem. They can drive a …


Hallucinating (or poorly fed) LLMs? The problem of data accuracy

… This is the case of “Common Crawl”, which consists in a massive collection of web pages and … To give a picture, the Common Crawl dataset constitutes nearly a trillion words. … were publicly available online and memorised in the Common …


Harder Tasks Need More Experts: Dynamic Routing in MoE Models

Q Huang, Z An, N Zhuang, M Tao, C Zhang, Y Jin, K Xu… - arXiv preprint arXiv …, 2024

In this paper, we introduce a novel dynamic expert selection framework for Mixture of Experts (MoE) models, aiming to enhance computational efficiency and model performance by adjusting the number of activated experts based on input difficulty …


Hardware Phi-1.5 B: A Large Language Model Encodes Hardware Domain Specific Knowledge

W Fu, S Li, Y Zhao, H Ma, R Dutta, X Zhang, K Yang…

… The first segment includes data from specialized platforms such as Arxiv , Books, Wikipedia, and StackExchange; the second segment is derived from broader internet content via CommonCrawl and C4. Table II provides a comprehensive …


Harmony in the Australian Domain Space

X Gong, PX McCarthy, MA Rizoiu, P Boldi - arXiv preprint arXiv:2404.10006, 2024

… Though not focused on localized networks, the extensive and all-encompassing web graph from Common Crawl utilized in our study enables the extraction of implicit information on a broader scale. … We utilize the datasets mainly from …


Harnessing Artificial Intelligence in Bariatric Surgery: Comparative Analysis of ChatGPT-4, Bing, and Bard in Generating Clinician-Level Bariatric Surgery …

Y Lee, T Shin, L Tessier, A Javidan, J Jung, D Hong… - Surgery for Obesity and Related …

Background The formulation of clinical recommendations pertaining to bariatric surgery is essential in guiding healthcare professionals. However, the extensive and continuously evolving body of literature in bariatric surgery presents considerable …


Harnessing generative AI for overcoming labeled data challenges in social media NLP

With the introduction of Transformers and Large Language Models, the field of NLP has significantly evolved. Generative AI, a prominent transformer-based technology for crafting human-like content, has proven powerful skills across numerous NLP …


Harnessing Small AI Model Collaboration and Debate Mechanisms in 6G Networks: Distributed Architectures and Layered Privacy Protection

C Han, T Yang, X Sun, Z Cui - ICC 2024-IEEE International Conference on …, 2024

… was used for classification tasks, while OpenSubtitles and CommonCrawl were used for text classification tasks. Our primary evaluation metrics revolve around model accuracy, convergence speed, and computational and communication …


Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings

L Gao - arXiv preprint arXiv:2408.15650, 2024

… For features that require embedding words, we use the 300-dimensional GloVe word embeddings [100] pretrained on the 42 billion token Common Crawl corpus. The GloVe embeddings are provided in decreasing order by frequency, and some …


Harnessing the Power of Metadata for Enhanced Question Retrieval in Community Question Answering

S Ghasemi, A Shakery - IEEE Access, 2024

… GPT-3 was trained on an open source dataset called ‘CommonCrawl’ and other texts from OpenAI such as Wikipedia entries. GPT-4 is the newest version of OpenAI’s language model systems. In this article, we fine-tune GPT-2 and GPT-3 pretrained …


Hate speech detection in low-resourced Indian languages: An analysis of transformer-based monolingual and multilingual models with cross-lingual experiments

K Ghosh, A Senapati - Natural Language Processing

Warning: This paper is based on hate speech detection and may contain examples of abusive/ offensive phrases.Cyberbullying, online harassment, etc., via offensive comments are pervasive across different social media platforms like ™Twitter, ™Facebook …


HCI International 2024 Posters: 26th International Conference on Human-Computer Interaction, HCII 2024, Washington, DC, USA, June 29–July 4, 2024, Proceedings …

Preliminary scientific results, professional news, or work in progress, described in the form of short research papers (4–11 pages long), constitute a popular submission type among the International Conference on Human-Computer …


Headline Generation for Indian Languages

… mT5: We conducted experiments on our dataset by fine-tuning the pre-trained mT5 [35] model, a multilingual variant of T5 [25], which was originally trained on the common crawl dataset, encompassing 101 languages. We use mT5-small model for …


HGT: Leveraging Heterogeneous Graph-enhanced Large Language Models for Few-shot Complex Table Understanding

R Jin, Y Li, G Qi, N Hu, YF Li, J Chen, J Wang, Y Chen… - arXiv preprint arXiv …, 2024

Table understanding (TU) has achieved promising advancements, but it faces the challenges of the scarcity of manually labeled tables and the presence of complex table structures.To address these challenges, we propose HGT, a framework with a …


Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding

H Xu, L Chen, Z Zhao, D Ma, R Cao, Z Zhu, K Yu - arXiv preprint arXiv:2402.18262, 2024

… Common Crawl is a publicly available web crawl dataset that collects webpages from the internet. Instead of using the source code, we collect webpage links from Common Crawl. We traverse all links in a dataset snapshot 4 and categorize and …


Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania

J Rudolph, MF Ismail, S Popenici - Journal of University Teaching and Learning …, 2024

Higher education is currently under a significant transformation due to the emergence of generative artificial intelligence (GenAI) technologies, the hype surrounding GenAI and the increasing influence of educational technology business …


Historical Dutch Spelling Normalization with Pretrained Language Models

A Wolters, A Van Cranenburgh - Computational Linguistics in the Netherlands …, 2024

… These texts are from the common crawl corpus, which contains petabytes of data scraped from the internet. Further pretraining on in-domain data can make the model more suited for the downstream task it will be finetuned for, and since our task …


Historical German Text Normalization Using Type-and Token-Based Language Modeling

A Ehrmanntraut - arXiv preprint arXiv:2409.02841, 2024

Historic variations of spelling poses a challenge for full-text search or natural language processing on historical digitized texts. To minimize the gap between the historic orthography and contemporary spelling, usually an automatic orthographic …


History, Development, and Principles of Large Language Models-An Introductory Survey

Z Chu, S Ni, Z Wang, X Feng, C Li, X Hu, R Xu, M Yang… - arXiv preprint arXiv …, 2024

Language models serve as a cornerstone in natural language processing (NLP), utilizing mathematical methods to generalize language laws and knowledge for prediction and generation. Over extensive research spanning decades, language …


HMoE: Heterogeneous Mixture of Experts for Language Modeling

A Wang, X Sun, R Xie, S Li, J Zhu, Z Yang, P Zhao… - arXiv preprint arXiv …, 2024

Mixture of Experts (MoE) offers remarkable performance and computational efficiency by selectively activating subsets of model parameters. Traditionally, MoE models use homogeneous experts, each with identical capacity. However, varying …


Holographic Embeddings for Text and Graphs A Master's Thesis Presented to The Faculty of the Graduate School of Arts and Sciences

… Static word embeddings such as word2vec, Global Vectors for Word Representation (GloVe), and fastText are trained on large amounts of natural language text sources like Common Crawl and Wikipedia dumps. These models …


Homonym Sense Disambiguation in the Georgian Language

D Melikidze, A Gamkrelidze - arXiv preprint arXiv:2405.00710, 2024

This research proposes a novel approach to the Word Sense Disambiguation (WSD) task in the Georgian language, based on supervised fine-tuning of a pre-trained Large Language Model (LLM) on a dataset formed by filtering the Georgian …


How Chinese are Chinese Language Models? The Puzzling Lack of Language Policy in China's LLMs

AW Wen-Yi, UES Jo, LJ Lin, D Mimno - arXiv preprint arXiv:2407.09652, 2024

… Additionally, similar to the training of international models, sources of pretrained data rely heavily on web pages, with three reports specifically mentioning Common Crawl as their primary data source… Common Crawl 8 (80%), encyclopedia, books …


How Do We Learn What We Cannot Say?

The contributions of this thesis are two-fold. First, this thesis presents UDTube, an easily usable software developed to perform morphological analysis in a multi-task fashion. This work shows the strong performance of UDTube versus the current state-of-the-art …


How Does Code Pretraining Affect Language Model Task Performance?

J Petty, S van Steenkiste, T Linzen - arXiv preprint arXiv:2409.04556, 2024

Large language models are increasingly trained on corpora containing both natural language and non-linguistic data like source code. Aside from aiding programming-related tasks, anecdotal evidence suggests that including code in pretraining corpora may …


How Effective are State Space Models for Machine Translation?

H Pitorro, P Vasylenko, M Treviso, AFT Martins - arXiv preprint arXiv:2407.05489, 2024

… In order to obtain a small high-quality subset for training, we exclude ParaCrawl and CommonCrawl samples from the original dataset and clean the remaining data. Our cleaning process includes three steps. First, we identify and remove samples in …


How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites

Z Chen, W Wang, H Tian, S Ye, Z Gao, E Cui, W Tong… - arXiv preprint arXiv …, 2024

In this report, we introduce InternVL 1.5, an open-source multimodal large language model (MLLM) to bridge the capability gap between open-source and proprietary commercial models in multimodal understanding. We introduce three simple …


How Gender Interacts with Political Values: A Case Study on Czech BERT Models

AA Ali, J Libovický - arXiv preprint arXiv:2403.13514, 2024

Neural language models, which reach state-of-the-art results on most natural language processing tasks, are trained on large text corpora that inevitably contain value-burdened content and often capture undesirable biases, which the models …


How Large Corpora Sizes Influence the Distribution of Low Frequency Text n-grams

JF Silva, JC Cunha - Pacific-Asia Conference on Knowledge Discovery and …, 2024

The prediction of the numbers of distinct word n-grams and their frequency distributions in text corpora is important in domains like information processing and language modelling. With big data corpora, there is an increased application …


How Lexical is Bilingual Lexicon Induction?

H Kohli, H Feng, N Dronen, C McCarter, S Moeini… - arXiv preprint arXiv …, 2024

… derived from Common Crawl and Wikipedia. The plot visualizes the Spearman’s Rank correlation of term frequencies between each of the source (by row) and target (by column) language pairs in the 5k vocabularies in the XLING corpus derived from …


How many news websites block AI crawlers?

In this factsheet we describe the proportion of news websites in ten countries that block AI (artificial intelligence) crawlers. We find that, (i) by the end 2023, 48% of the most widely used news websites across ten countries were blocking OpenAI’s (ChatGPT) …


How Much are LLMs Contaminated? A Comprehensive Survey and the LLMSanitize Library

M Ravaut, B Ding, F Jiao, H Chen, X Li, R Zhao, C Qin… - arXiv preprint arXiv …, 2024

… This includes policies and protocols for data privacy, consent, and use that help prevent the incorporation of contaminated data from unethical sources and the contamination of widely used pre-training data sources (eg, CommonCrawl) …


How much semantic information is available in large language model tokens?

… We first regressed MMLU score on the percentage of the Common Crawl that a language makes up and whether it uses the Latin alphabet… is a significant predictor when added to the model that includes Common Crawl percentage and …


How to Train Data-Efficient LLMs

N Sachdeva, B Coleman, WC Kang, J Ni, L Hong… - arXiv preprint arXiv …, 2024

The training of large language models (LLMs) is expensive. In this paper, we study data-efficient approaches for pre-training LLMs, ie, techniques that aim to optimize the Pareto frontier of model quality and training resource/data consumption. We …


HPLT's First Release of Data and Models

N Arefyev, M Aulamo, P Chen, O de Gibert, B Haddow… - 2024

… Datasets For the first release, we processed 1.85 petabytes of the Internet Archive and CommonCrawl to create monolingual and parallel corpora. We release them under the permissive CC0 licence1 through our project website2, OPUS3, and Hugging …


Human-Centered Interaction in Virtual Worlds: A New Era of Generative Artificial Intelligence and Metaverse

Y Wang, L Wang, KL Siau - International Journal of Human–Computer Interaction, 2024

The metaverse has emerged as an exciting new paradigm for human-computer interaction (HCI) and virtual collaboration. This paper presents a comprehensive review of the metaverse to address the gap in the existing literature where there is a …


Human-Centered Natural Language Processing for Countering Misinformation

As curbing the spread of online misinformation has proven to be challenging, we look to artificial intelligence (AI) and natural language technology for helping individuals and society counter and limit it. Despite current advances, state-of-the-art …


Humkinar: Construction of a Large Scale Web Repository and Information System for Low Resource Urdu Language

… first Common Crawl Corpus (CCC) filter leverages the open-source Common Crawl repository to filter Urdu language webpages. Common Crawl… From August 2018 onward releases, Common Crawl also provides the meta information …


Hybrid Bilingual Question-Answering System with Human-Level Naturalness Text-to-Speech Extension

This project introduces a robust bilingual question-answering and natural Text-to-Speech (TTS) system, primarily designed to support both Vietnamese and English languages. Leveraging pre-trained Large Language Models (LLMs), the system enables …


Hybrid semantic models for building smart and robust home robots

The creation of home robots that can aid human beings in daily mundane chores has been a long-standing goal of robotics research. Some common indoor tasks that service robots can help with include retrieving objects from different locations …


Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers

S Hwang, A Lahoti, T Dao, A Gu - arXiv preprint arXiv:2407.09941, 2024

A wide array of sequence models are built on a framework modeled after Transformers, comprising alternating sequence mixer and channel mixer layers. This paper studies a unifying matrix mixer view of sequence mixers that can be …


HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts

W Kim, S Chun, T Kim, D Han, S Yun - arXiv preprint arXiv:2404.17507, 2024

In an era where the volume of data drives the effectiveness of self-supervised learning, the specificity and clarity of data semantics play a crucial role in model training. Addressing this, we introduce HYPerbolic Entailment filtering (HYPE), a …


Hypertext Entity Extraction in Webpage

Y Yang, T Liu, B Shao, H Zhao, L Shou, M Gong… - arXiv preprint arXiv …, 2024

… Common Crawl5 is a widely used large-scale dataset for IE. However, they all overlook rich hypertext features and contain too much noise. Moreover, their annotations are often determined by rules or automated tools, lacking accurate manual annotations. …


I'm In! A Comparative Analysis of Business Pitch Competition

E Barron, I Bernatovic, LE Ruiz - DIANA INTERNATIONAL RESEARCH …, 2023

Methods Based on a perceived theoretical gap in research, entrepreneurship scholars have called for increased research to understand how institutions (including governments and academia), influence the construction of intersectional …


Identification of patients' smoking status using an explainable AI approach: a Danish electronic health records case study

A Ebrahimi, MBH Henriksen, CL Brasen, O Hilberg… - BMC Medical Research …, 2024

Smoking is a critical risk factor responsible for over eight million annual deaths worldwide. It is essential to obtain information on smoking habits to advance research and implement preventive measures such as screening of high-risk …


Identification of Trolling Memes in Kannada and Tulu-Under-resourced Dravidian Languages

… Trained on a diverse dataset comprising Common Crawl and Wikipedia content across 157 languages, fastText employs Continuous Bag-of-Words (CBOW) with position weights, generating embeddings in a low-dimensional space of size 300 [44] …


Identifying the sources of ideological bias in GPT models through linguistic variation in output

C Walker, JC Timoneda - arXiv preprint arXiv:2409.06043, 2024

Extant work shows that generative AI models such as GPT-3.5 and 4 perpetuate social stereotypes and biases. One concerning but less explored source of bias is ideology. Do GPT models take ideological stances on politically sensitive topics? In …


iDRAMA-rumble-2024: A Dataset of Podcasts from Rumble Spanning 2020 to 2022

U Balci, J Patel, B Balci, J Blackburn - 2024

Rumble has emerged as a prominent platform hosting controversial figures facing restrictions on YouTube. Despite this, the academic community’s engagement with Rumble has been minimal. To help researchers address this gap, we introduce a …


If You Don't Understand It, Don't Use It: Eliminating Trojans with Filters Between Layers

A Hernandez - arXiv preprint arXiv:2407.06411, 2024

… In this model an LLM is pre-trained on a large web corpus such as Common Crawl, the model server tunes the model and inserts it into a (business) mission-critical system, and the model user uses this system. This system can be a chat-bot, code-completion …


IGOT: Information Gain Optimized Tokenizer on Domain Adaptive Pretraining

D Feng, Y Zhang, Z Xu - arXiv preprint arXiv:2405.09857, 2024

Pretrained Large Language Models (LLM) such as ChatGPT, Claude, etc. have demonstrated strong capabilities in various fields of natural language generation. However, there are still many problems when using LLM in specialized domain-specific …


Impact of COVID-19 Pandemic on Social Determinants of Health Issues of Marginalized Black and Asian Communities: A Social Media Analysis Empowered by …

C Whitfield, Y Liu, M Anwar - Journal of Racial and Ethnic Health Disparities, 2024

Purpose This study aims to understand the impact of the COVID-19 pandemic on social determinants of health (SDOH) of marginalized racial/ethnic US population groups, specifically African Americans and Asians, by leveraging natural language …


Improved methodology for longitudinal Web analytics using Common Crawl

HS Thompson - 16th ACM Web Science Conference 2024, 2024

Common Crawl is a multi-petabyte longitudinal dataset containing over 100 billion web pages which is widely used as a source of language data for sequence model training and in web science research. Each of its constituent archives is on the order …


Improving and Understanding Clarifying Question Generation in Conversational Search

D Ortega, S Söhnel, NT Vu - International Conference on Text, Speech, and …, 2024

Conversational information-seeking systems (CISs), such as chatbots and virtual personal assistants, encounter difficulty when processing ambiguous user requests (URs) and generate an accurate response, especially when multiple search results match …


Improving Arithmetical Reasoning of Language Models

This work focuses on improving language models for solving math word problems. First, we clean and transform existing datasets to create Calc-X, a standardized collection of 300,000 math problems with step-by-step solutions. We demonstrate …


Improving Bengali and Hindi Large Language Models

A Shahriar, D Barbosa - Proceedings of the 2024 Joint International Conference …, 2024

Despite being widely spoken worldwide, Bengali and Hindi are low-resource languages. The state-of-the-art in modeling such languages uses BERT and the Wordpiece tokenizer. We observed that the Wordpiece tokenizer often breaks words …


Improving Cross-lingual Representation for Semantic Retrieval with Code-switching

M Maimaiti, Y Zheng, J Zhang, F Huang, Y Zhang… - arXiv preprint arXiv …, 2024

Semantic Retrieval (SR) has become an indispensable part of the FAQ system in the task-oriented question-answering (QA) dialogue scenario. The demands for a cross-lingual smart-customer-service system for an e-commerce platform or some particular …


Improving Parameter-Efficient Cross-Lingual Transfer for Low-Resource Languages

… The success of the XLM-R can largely be attributed to the quality and size of its training corpora the model is pretrained on more than two terabytes of the filtered Common Crawl data. Notable differences in comparison with mBERT (beyond much …


Improving Pretraining Data Using Perplexity Correlations

T Thrush, C Potts, T Hashimoto - arXiv preprint arXiv:2409.05816, 2024

Quality pretraining data is often seen as the key to high-performance language models. However, progress in understanding pretraining data has been slow due to the costly pretraining runs required for data selection experiments. We present a …


Improving the Consistency in Cross-Lingual Cross-Modal Retrieval with 1-to-K Contrastive Learning

Z Nie, R Zhang, Z Feng, H Huang, X Liu - arXiv preprint arXiv:2406.18254, 2024

… For instance, XLM-R, CCR𝑘 ’s cross-lingual encoder, is trained on the 2.5TB CommonCrawl Corpus encompassing 100 languages. Discrepancies in data sizes between high-resource and low-resource languages within this corpus, like the …


INCLURE: a Dataset and Toolkit for Inclusive French Translation

P Lerner, C Grouin - The 17th Workshop on Building and Using …, 2024

… Indeed, this corpus mostly contains transcripts of political speeches, whose oral style differs from the text typically found in OSCAR/CommonCrawl. Exceptions are six examples used to illustrate the use of the inclusive neutralization process …


Incorporating Geo-Diverse Knowledge into Prompting for Increased Geographical Robustness in Object Recognition

K Buettner, S Malakouti, XL Li, A Kovashka - arXiv preprint arXiv:2401.01482, 2024

Existing object recognition models have been shown to lack robustness in diverse geographical scenarios due to significant domain shifts in design and context. Class representations need to be adapted to more accurately reflect an object concept …


Incorporating Human Judgment in AI-Assisted Content Development: The HEAT Heuristic

G Verhulsdonck, J Weible, DM Stambler, T Howard… - Technical Communication, 2024

Purpose: As technical and professional communicators (TPCs) use AI to develop content, inaccuracies due to AI limitations are introduced; it is vital TPCs evaluate AI-generated content to improve accuracy and human-centeredness. In this article, we present a …


IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context

NR Sahoo, PP Kulkarni, N Asad, A Ahmad, T Goyal… - arXiv preprint arXiv …, 2024

The pervasive influence of social biases in language data has sparked the need for benchmark datasets that capture and evaluate these biases in Large Language Models (LLMs). Existing efforts predominantly focus on English language and the …


INDIC QA BENCHMARK: A Multilingual Benchmark to Evaluate Question Answering capability of LLMs for Indic Languages

AK Singh, R Murthy, J Sen, G Ramakrishnan - arXiv preprint arXiv:2407.13522, 2024

… To address this, we sampled numerous Wikipedia and Common Crawl pages, focusing on paragraphs rich in cultural nuances and domain diversity. Using these paragraphs, we employed large language models (LLMs) to generate QA pairs, thus …


IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages

MSUR Khan, P Mehta, A Sankar, U Kumaravelan… - arXiv preprint arXiv …, 2024

Despite the considerable advancements in English LLMs, the progress in building comparable models for other languages has been hindered due to the scarcity of tailored resources. Our work aims to bridge this divide by introducing an expansive …


Indiscriminate Data Poisoning Attacks on Pre-trained Feature Extractors

Y Lu, MYR Yang, G Kamath, Y Yu - arXiv preprint arXiv:2402.12626, 2024

… To collect such large-scale datasets, practitioners usually extract the desired data by crawling on the internet (eg, web pages using Common Crawl1). However, using outsourced data raises an imminent security risk (Kumar et al. 2020; Nelson et al …


Individual-vs. Multiple-Objective Strategies for Targeted Sentiment Analysis in Finances Using the Spanish MTSA 2023 Corpus

R Pan, JA García-Díaz, R Valencia-García - Electronics, 2024

… complexity; (vi) RoBERTuito [22], a pretrained language model for user-generated content in Spanish, trained following RoBERTa guidelines on 500 million tweets; and (vii) XLM-RoBERTa [17], a multilingual version of RoBERTa, trained with data …


IndoBerea: Evolving Semantic Search in Theological Context

FVP Samosir, S Mendrofa - 2023 Eighth International Conference on Informatics …, 2023

This paper presents IndoBerea, a semantic search model pre-trained on an Indonesian Bible dataset and based on SentenceTransformers and IndoBERT. It aims to enhance theological research by providing contextually relevant verses in …


INDUS: Effective and Efficient Language Models for Scientific Applications

B Bhattacharjee, A Trivedi, M Muraoka… - arXiv preprint arXiv …, 2024

Large language models (LLMs) trained on general domain corpora showed remarkable results on natural language processing (NLP) tasks. However, previous research demonstrated LLMs trained using domain-focused corpora perform better …