Physical Sciences and Mathematics Commons^™

A Psychometric Analysis Of Natural Language Inference Using Transformer Language Models, Antonio Laverghetta Jr.

USF Tampa Graduate Theses and Dissertations

Large language models (LLMs) are poised to transform both academia and industry. But the excitement around these generative AIs has also been met with concern for the true extent of their capabilities. This dissertation helps to address these questions by examining the capabilities of LLMs using the tools of psychometrics. We focus on analyzing the capabilities of LLMs on the task of natural language inference (NLI), a foundational benchmark often used to evaluate new models. We demonstrate that LLMs can reliably predict the psychometric properties of NLI items were those items administered to humans. Through a series of experiments, we …

Text Augmentation For Semantic Frame Induction And Parsing, Saba Anwar, Artem Shelmanov, Nikolay Arefyev, Alexander Panchenko, Chris Biemann

Natural Language Processing Faculty Publications

Semantic frames are formal structures describing situations, actions or events, e.g., Commerce buy, Kidnapping, or Exchange. Each frame provides a set of frame elements or semantic roles corresponding to participants of the situation and lexical units (LUs)—words and phrases that can evoke this particular frame in texts. For example, for the frame Kidnapping, two key roles are Perpetrator and the Victim, and this frame can be evoked with lexical units abduct, kidnap, or snatcher. While formally sound, the scarce availability of semantic frame resources and their limited lexical coverage hinders the wider adoption of frame semantics across languages and domains. …

Yet Another Model For Arabic Dialect Identification, Ajinkya Kulkarni, Hanan Al Darmaki

Natural Language Processing Faculty Publications

In this paper, we describe a spoken Arabic dialect identification (ADI) model for Arabic that consistently outperforms previously published results on two benchmark datasets: ADI-5 and ADI-17. We explore two architectural variations: ResNet and ECAPA-TDNN, coupled with two types of acoustic features: MFCCs and features exratected from the pre-trained self-supervised model UniSpeech-SAT Large, as well as a fusion of all four variants. We find that individually, ECAPA-TDNN network outperforms ResNet, and models with UniSpeech-SAT features outperform models with MFCCs by a large margin. Furthermore, a fusion of all four variants consistently outperforms individual models. Our best models outperform previously reported …

Decentralized Science (Desci): A New Paradigm For Diverse And Sustainable Scientific Development, Feiyue Wang, Wenwen Ding

Bulletin of Chinese Academy of Sciences (Chinese Version)

The rise of artificial intelligence for science (AI4S) has made it particularly important and urgent to ensure the openness, fairness, impartiality, diversity, and sustainability of scientific systems. This is significant to the discourse power and leadership of countries in global innovation and industrial revolution, and also affects the security, stability, and sustainable development of a community with a shared future for mankind. To address these challenges, AI4S needs to adopt new scientific organizational and operational methods. Decentralized science (DeSci) has emerged to vitalize AI4S and provide strong support, effectively addressing issues such as information silos, biases, unfair distribution, and monopolies …

Metaverse Key Requirements And Platforms Survey, Akbobek Abilkaiyrkyzy, Ahmed Elhagry, Fedwa Laamarti, Abdulmotaleb El Saddik

Computer Vision Faculty Publications

The growing interest in the metaverse has led to an abundance of platforms, each with its own unique features and limitations. This paper's objective is two-fold. First, we aim at providing an objective analysis of requirements that need to be fulfilled by metaverse platforms. We survey a broad set of criteria including interoperability, immersiveness, persistence, multimodal and social interaction, scalability, level of openness, configurability, market access, security, and blockchain integration, among others. Second, we review a wide range of existing metaverse platforms, and we critically evaluate their ability to meet the requirements listed. We identify their limitations, which must be …

Improving Human-Automation Collaboration In Motion Planning, Torin J. Adamson

Computer Science ETDs

Human-automation collaboration is becoming a part of everyday life as AI helps us drive, make decisions, and solve a variety of other tasks. However, safe and effective collaboration systems depend on factors in trust, communication, and more. Existing studies to explore these are typically carried out in laboratory settings, providing robust data under tight environmental control. However, human behavior evolves over time, driven by external factors that cannot be fully captured in single participation sessions. These factors form the "human context", contextualizing the behavioral data for a more complete understanding. In this thesis, video game adaptations upon conventional subject studies …

Ai As A License Review Assistant, Nat Gustafson-Sundell

Library Services Publications

I will present the steps we have taken to develop a prototype AI assistant for license review. I’ll explain our criteria for the selection of an AI tool for this project. We reviewed ChatGPT, Claude 2, Bard, and PDF readers. My goal was to develop an initial prototype in a Jupyter Notebook environment so I could easily re-load context information, including a license checklist, but I’ll explain why I revised this goal, instead to linger over license review interactions with ChatBots. I’ll discuss early results, demonstrate example license review interactions, and outline my next steps.

Artificial Intelligence History, And Libraries: History And Legacy Of Library Contributions To Machine Learning, Wilhelmina Randtke

Library Faculty Presentations

Machine learning seems to be newly everywhere. It's not new, so much as faster processing makes it newly useful. Imagine an automated cataloging program that takes 300 years to run, versus one that takes a week to run. Increased processing speed is a substantive change. This presentation overviews the history of libraries and artificial intelligence. First, teasing out past applications of machine learning in libraries. High quality results and concrete applications of artificial intelligence in libraries have been explored and published for decades. Over time, faster processing allows use at scale. Second, how library and metadata work contributes to machine …

Local Model Agnostic Xai Methodologies Applied To Breast Cancer Malignancy Predictions, Heather Hartley

Electronic Thesis and Dissertation Repository

This thesis examines current state-of-the-art Explainable Artificial Intelligence (XAI) methodologies applicable to breast cancer diagnostics, as well as local model-agnostic XAI methodologies more broadly. It is well known that AI is underutilized in healthcare due to the fact that black box AI methods are largely uninterpretable. The potential for AI to positively affect health care outcomes is massive, and AI adoption by medical practitioners and the community at large will translate to more desirable patient outcomes. The development of XAI is crucial to furthering the integration of AI within healthcare, as it will allow medical practitioners and regulatory bodies to …

Artificial Intelligence And Human Hope, Michael Paulus

SPU Works

Slides from a book talk at Folio: The Seattle Atheneum on Artificial Intelligence and the Apocalyptic Imagination: Artificial Agency and Human Hope.

Investigating Continual Learning Strategies In Neural Networks, Christopher Tam, Luiz Fernando Capretz

Electrical and Computer Engineering Publications

This paper explores the role of continual learning strategies when neural networks are confronted with learning tasks sequentially. We analyze the stability-plasticity dilemma with three factors in mind: the type of network architecture used, the continual learning scenario defined and the continual learning strategy implemented. Our results show that complementary learning systems and neural volume significantly contribute towards memory retrieval and consolidation in neural networks. Finally, we demonstrate how regularization strategies such as elastic weight consolidation are more well-suited for larger neural networks whereas rehearsal strategies such as gradient episodic memory are better suited for smaller neural networks.

Evocative And Provocative Image-Making In The Age Of Generative Ai, Julian Kilker

Tradition Innovations in Arts, Design, and Media Higher Education

Editorial for inaugural AI-focused special issue of Tradition-Innovations in Arts, Design, and Media Higher Education, published under the auspices of the Alliance for the Arts in Research Universities (a2ru). Discusses three articles by five authors in this issue: (1) Choreographing Shadows: Interdisciplinary collaboration to orchestrate ethical image-making by Mark Burchick and Diana Pasulka; (2) Giving Up Control: Hybrid AI-augmented workflows for image-making by Joshua Vermillion; and (3) Hands are Hard: Unlearning how we talk about machine learning in the arts by Adam Hyland and Oscar Keyes.

Editing this special issue explored several key questions: What does “innovation” mean when …

Healthcare Ai: A Revised Quebec Framework For Nursing Education, Maggie Lattuca, Diane Maratta, Ute Beffert, Annie Chevrier, Laura Winer

Quality Advancement in Nursing Education - Avancées en formation infirmière

Artificial Intelligence Health Technologies (AIHT) are taking their place in the practice of nursing. However, the curricula have not evolved to include competencies required of nursing graduates to incorporate their impact on theory and practice. This project was born of an identified need by nurse educators to articulate new competencies grounded in the literature and expert knowledge. Based on extensive literature reviews and an iterative process of expert validation, this paper provides recommendations for five new competencies that will be needed for nurses to use AIHT responsibly, ethically, and intelligently in the best interests of patient care. The methodology started …

Dtitd: An Intelligent Insider Threat Detection Framework Based On Digital Twin And Self-Attention Based Deep Learning Models, Zhi Qiang Wang, Abdulmotaleb El Saddik

Computer Vision Faculty Publications

Recent statistics and studies show that the loss generated by insider threats is much higher than that generated by external attacks. More and more organizations are investing in or purchasing insider threat detection systems to prevent insider risks. However, the accurate and timely detection of insider threats faces significant challenges. In this study, we proposed an intelligent insider threat detection framework based on Digital Twins and self-attentions based deep learning models. First, this paper introduces insider threats and the challenges in detecting them. Then this paper presents recent related works on solving insider threat detection problems and their limitations. Next, …

Adapting The Adapters For Code-Switching In Multilingual Asr, Atharva Kulkarni, Ajinkya Kulkarni, Miguel Couceiro, Hanan Al Darmaki

Natural Language Processing Faculty Publications

Recently, large pre-trained multilingual speech models have shown potential in scaling Automatic Speech Recognition (ASR) to many low-resource languages. Some of these models employ language adapters in their formulation, which helps to improve monolingual performance and avoids some of the drawbacks of multi-lingual modeling on resource-rich languages. However, this formulation restricts the usability of these models on code-switched speech, where two languages are mixed together in the same utterance. In this work, we propose ways to effectively fine-tune such models on code-switched speech, by assimilating information from both language adapters at each language adaptation point in the network. We also …

Graph Transformer Network For Flood Forecasting With Heterogeneous Covariates, Jimeng Shi, Vitalii Stebliankin, Zhaonan Wang, Shaowen Wang, Giri Narasimhan

I-GUIDE Forum

Floods can be very destructive causing heavy damage to life, property, and livelihoods. Global climate change and the consequent sea-level rise have increased the occurrence of extreme weather events, resulting in elevated and frequent flood risk. Therefore, accurate and timely flood forecasting in coastal river systems is critical to facilitate good flood management. However, the computational tools currently used are either slow or inaccurate. In this paper, we propose a Flood prediction tool using Graph Transformer Network (FloodGTN) for river systems. More specifically, FloodGTN learns the spatio-temporal dependencies of water levels at different monitoring stations using Graph Neural Networks (GNNs) …

Semantic Lung Segmentation From Chest X-Ray Images Using Seg-Net Deep Cnn Model, Dathar Abas Hasan, Umed Hayder Jader

Polytechnic Journal

Implementing an accurate image segmentation to extract the lung shape from X-ray images is a vital step in designing a CAD system that diagnoses various types of chest diseases. Lung segmentation is a complex process due to the blurred regions that separate the lung area and the rest of the image. The conventional image segmentation techniques do not meet the ambitions to achieve precise lung segmentation. In this paper, we utilized the Seg-Net semantic segmentation model as a practical approach to distinguish the lung region pixels in X-ray images. The model involves an encoder network that extracts the data from …

Curriculum Design Of Artificial Intelligence And Sustainability In Secondary School, Jinyi Cai, Mei-Po Kwan, Chunyu Hou, Dong Liu, Yeung Yam

I-GUIDE Forum

Artificial Intelligence is revolutionizing numerous sectors with its transformative power, while at the same time, there is an increasing sense of urgency to address sustainability challenges. Despite the significance of both areas, secondary school curriculums still lack comprehensive integration of AI and sustainability education. This paper presents a curriculum designed to bridge this gap. The curriculum integrates progressive objectives, computational thinking competencies and system thinking components across five modules—awareness, knowledge, interaction, empowerment and ethics—to cater to varying learner levels. System thinking components help students understand sustainability in a holistic manner. Computational thinking competencies aim to cultivate computational thinkers to guide …

Reducing Uncertainty In Sea-Level Rise Prediction: A Spatial-Variability-Aware Approach, Subhankar Ghosh, Shuai An, Arun Sharma, Jayant Gupta, Shashi Shekhar, Aneesh Subramanian

I-GUIDE Forum

Given multi-model ensemble climate projections, the goal is to accurately and reliably predict future sea-level rise while lowering the uncertainty. This problem is important because sea-level rise affects millions of people in coastal communities and beyond due to climate change's impacts on polar ice sheets and the ocean. This problem is challenging due to spatial variability and unknowns such as possible tipping points (e.g., collapse of Greenland or West Antarctic ice-shelf), climate feedback loops (e.g., clouds, permafrost thawing), future policy decisions, and human actions. Most existing climate modeling approaches use the same set of weights globally, during either regression or …

Peatmoss: Mining Pre-Trained Models In Open-Source Software, Wenxin Jiang, Jason Jones, Jerin Yasmin, Nicholas Synovic, Rajiv Sashti, Sophie Chen, George K. Thiruvathukal, Yuan Tian, James C. Davis

Computer Science: Faculty Publications and Other Works

Developing and training deep learning models is expensive, so software engineers have begun to reuse pre-trained deep learning models (PTMs) and fine-tune them for downstream tasks. Despite the widespread use of PTMs, we know little about the corresponding software engineering behaviors and challenges. To enable the study of software engineering with PTMs, we present the PeaTMOSS dataset: Pre-Trained Models in Open-Source Software. PeaTMOSS has three parts: a snapshot of (1) 281,638 PTMs, (2) 27,270 open-source software repositories that use PTMs, and (3) a mapping between PTMs and the projects that use them. We challenge PeaTMOSS miners to discover software engineering …

Synthesizing Sentience: Integrating Large Language Models And Autonomous Agents For Emulating Human Cognitive Complexity, Jay Ratican, James Hutson, Daniel Plate

Faculty Scholarship

The paper aims to present a novel methodology for emulating the intricacies of human cognitive complexity by ingeniously integrating large language models with autonomous agents. Grounded in the theoretical framework of the modular mind theory-originally espoused by Fodor and later refined by scholars such as Joanna Bryson—the study seeks to venture into the untapped potential of large language models and autonomous agents in mirroring human cognition. Recent advancements in artificial intelligence, exemplified by the inception of autonomous agents like Age in GPT, auto GPT, and baby AGI, underscore the transformative capacities of these technologies in diverse applications. Moreover, empirical studies …

An Ai-Based Framework For Translating American Sign Language To English And Vice Versa, Vijayendra D. Avina, Md Amiruzzaman, Stefanie Amiruzzaman, Linh B. Ngo, M. Ali Akber Dewan

Computer Science Faculty Publications

Abstract: In this paper, we propose a framework to convert American Sign Language (ASL) to English and English to ASL. Within this framework, we use a deep learning model along with the rolling average prediction that captures image frames from videos and classifies the signs from the image frames. The classified frames are then used to construct ASL words and sentences to support people with hearing impairments. We also use the same deep learning model to capture signs from the people with deaf symptoms and convert them into ASL words and English sentences. Based on this framework, we developed a …

Exploring Approaches To Engage K-12 Students In Learning Computational Thinking Using Collaborative Robots, Zoila Anuri Kanu

College of Engineering Summer Undergraduate Research Program

Minority students are largely underrepresented in the STEM field. The goal for this project was to develop a program which promotes the inclusion of computation skills among students and help them work collaboratively with the use of human – robot interaction. Robots are such a strong tool that can be used to enhance computational thinking and engage students towards a technical field. Through workshops and readings about computational thinking we worked on building a block-based program that introduces the uses of robots as teaching tool for computational thinking.

Object Recognition With Deep Neural Networks In Low-End Systems, Lillian Davis

Mahurin Honors College Capstone Experience/Thesis Projects

Object recognition is an important area in computer vision. Object recognition has been advanced significantly by deep learning that unifies feature extraction and classification. In general, deep neural networks, such as Convolution Neural Networks (CNNs), are trained in high-performance systems. Aiming to extend the reach of deep learning to personal computing, I propose a study of deep learning-based object recognition in low-end systems, such as laptops. This research includes how differing layer configurations and hyperparameter values used in CNNs can either create or resolve the issue of overfitting and affect final accuracy levels of object recognition systems. The main contribution …

A System For The Detection Of Adversarial Attacks In Computer Vision Via Performance Metrics, Sarah Reynolds

Doctoral Dissertations and Master's Theses

Adversarial attacks, or attacks committed by an adversary to hijack a system, are prevalent in the deep learning tasks of computer vision and are one of the greatest threats to these models' safe and accurate use. These attacks force the trained model to misclassify an image, using pixel-level changes undetectable to the human eye. Various defenses against these attacks exist and are detailed in this work. The work of previous researchers has established that when adversarial attacks occur, different node patterns in a Deep Neural Network (DNN) are activated within the model. Additionally, it is known that CPU and GPU …

Voucher Abuse Detection With Prompt-Based Fine-Tuning On Graph Neural Networks, Zhihao Wen, Yuan Fang, Yihan Liu, Yang Guo, Shuji Hao

Research Collection School Of Computing and Information Systems

Voucher abuse detection is an important anomaly detection problem in E-commerce. While many GNN-based solutions have emerged, the supervised paradigm depends on a large quantity of labeled data. A popular alternative is to adopt self-supervised pre-training using label-free data, and further fine-tune on a downstream task with limited labels. Nevertheless, the "pre-train, fine-tune" paradigm is often plagued by the objective gap between pre-training and downstream tasks. Hence, we propose VPGNN, a prompt-based fine-tuning framework on GNNs for voucher abuse detection. We design a novel graph prompting function to reformulate the downstream task into a similar template as the pretext task …

Posmlp-Video: Spatial And Temporal Relative Position Encoding For Efficient Video Recognition, Yanbin Hao, Diansong Zhou, Zhicai Wang, Chong-Wah Ngo, Xiangnan He, Meng Wang

Research Collection School Of Computing and Information Systems

In recent years, vision Transformers and MLPs have demonstrated remarkable performance in image understanding tasks. However, their inherently dense computational operators, such as self-attention and token-mixing layers, pose significant challenges when applied to spatio-temporal video data. To address this gap, we propose PosMLP-Video, a lightweight yet powerful MLP-like backbone for video recognition. Instead of dense operators, we use efficient relative positional encoding (RPE) to build pairwise token relations, leveraging small-sized parameterized relative position biases to obtain each relation score. Specifically, to enable spatio-temporal modeling, we extend the image PosMLP’s positional gating unit to temporal, spatial, and spatio-temporal variants, namely PoTGU, …

Understanding The Effect Of Counterfactual Explanations On Trust And Reliance On Ai For Human-Ai Collaborative Clinical Decision Making, Min Hun Lee, Chong Jun Chew

Research Collection School Of Computing and Information Systems

Artificial intelligence (AI) is increasingly being considered to assist human decision-making in high-stake domains (e.g. health). However, researchers have discussed an issue that humans can over-rely on wrong suggestions of the AI model instead of achieving human AI complementary performance. In this work, we utilized salient feature explanations along with what-if, counterfactual explanations to make humans review AI suggestions more analytically to reduce overreliance on AI and explored the effect of these explanations on trust and reliance on AI during clinical decision-making. We conducted an experiment with seven therapists and ten laypersons on the task of assessing post-stroke survivors' quality …

Objectfusion: Multi-Modal 3d Object Detection With Object-Centric Fusion, Q. Cai, Y. Pan, T. Yao, Chong-Wah Ngo, T. Mei

Research Collection School Of Computing and Information Systems

Recent progress on multi-modal 3D object detection has featured BEV (Bird-Eye-View) based fusion, which effectively unifies both LiDAR point clouds and camera images in a shared BEV space. Nevertheless, it is not trivial to perform camera-to-BEV transformation due to the inherently ambiguous depth estimation of each pixel, resulting in spatial misalignment between these two multi-modal features. Moreover, such transformation also inevitably leads to projection distortion of camera image features in BEV space. In this paper, we propose a novel Object-centric Fusion (ObjectFusion) paradigm, which completely gets rid of camera-to-BEV transformation during fusion to align object-centric features across different modalities for …

Residual Pattern Learning For Pixel-Wise Out-Of-Distribution Detection In Semantic Segmentation, Y Liu, Choubo Ding, Yu Tian, Guansong Pang, Vasileios Belagiannis, Ian Reid, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

Semantic segmentation models classify pixels into a set of known ("in-distribution") visual classes. When deployed in an open world, the reliability of these models depends on their ability to not only classify in-distribution pixels but also to detect out-of-distribution (OoD) pixels. Historically, the poor OoD detection performance of these models has motivated the design of methods based on model re-training using synthetic training images that include OoD visual objects. Although successful, these re-trained methods have two issues: 1) their in-distribution segmentation accuracy may drop during re-training, and 2) their OoD detection accuracy does not generalise well to new contexts (e.g., …