Physical Sciences and Mathematics Commons^™

Let’S Think Outside The Box: Exploring Leap-Of-Thought In Large Language Models With Multimodal Humor Generation, Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, Pan Zhou

Research Collection School Of Computing and Information Systems

Chain-of-Thought (CoT) [2, 3] guides large language models (LLMs) to reason step-by-step, and can motivate their logical reasoning ability. While effective for logical tasks, CoT is not conducive to creative problem-solving which often requires out-of-box thoughts and is crucial for innovation advancements. In this paper, we explore the Leap-of-Thought (LoT) abilities within LLMs — a nonsequential, creative paradigm involving strong associations and knowledge leaps. To this end, we study LLMs on the popular Oogiri game which needs participants to have good creativity and strong associative thinking for responding unexpectedly and humorously to the given image, text, or both, and thus …

Learning Dynamic Multimodal Network Slot Concepts From The Web For Forecasting Environmental, Social And Governance Ratings, Meng Kiat Gary Ang, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

Dynamic multimodal networks are networks with node attributes from different modalities where the at- tributes and network relationships evolve across time, i.e., both networks and multimodal attributes are dynamic; for example, dynamic relationship networks between companies that evolve across time due to changes in business strategies and alliances, which are associated with dynamic company attributes from multiple modalities such as textual online news, categorical events, and numerical financial-related data. Such information can be useful in predictive tasks involving companies. Environmental, social, and gov- ernance (ESG) ratings of companies are important for assessing the sustainability risks of companies. The process of …

Few-Shot Learner Parameterization By Diffusion Time-Steps, Zhongqi Yue, Pan Zhou, Richang Hong, Hanwang Zhang, Sun Qianru

Research Collection School Of Computing and Information Systems

Even when using large multi-modal foundation models, few-shot learning is still challenging—if there is no proper inductive bias, it is nearly impossible to keep the nuanced class attributes while removing the visually prominent attributes that spuriously correlate with class labels. To this end, we find an inductive bias that the time-steps of a Diffusion Model (DM) can isolate the nuanced class attributes, i.e., as the forward diffusion adds noise to an image at each time-step, nuanced attributes are usually lost at an earlier time-step than the spurious attributes that are visually prominent. Building on this, we propose Time-step Few-shot (TiF) …

Gts: Gpu-Based Tree Index For Fast Similarity Search, Yifan Zhu, Ruiyao Ma, Baihua Zheng, Xiangyu Ke, Lu Chen, Yunjun Gao

Research Collection School Of Computing and Information Systems

Similarity search, the task of identifying objects most similar to a given query object under a specific metric, has gathered significant attention due to its practical applications. However, the absence of coordinate information to accelerate similarity search and the high computational cost of measuring object similarity hinder the efficiency of existing CPU-based methods. Additionally, these methods struggle to meet the demand for high throughput data management. To address these challenges, we propose GTS, a GPU-based tree index designed for the parallel processing of similarity search in general metric spaces, where only the distance metric for measuring object similarity is known. …

Navigating The Ethical Terrain Of Ai In Higher Education: Strategies For Mitigating Bias And Promoting Fairness, Emily Barnes, James Hutson

Faculty Scholarship

Artificial intelligence (AI) and machine learning (ML) are transforming higher education by enhancing personalized learning and academic support, yet they pose significant ethical challenges, particularly in terms of inherent biases. This review critically examines the integration of AI in higher education, underscoring the dual aspects of its potential to innovate educational paradigms and the essential need to address ethical implications to avoid perpetuating existing inequalities. The researchers employed a methodological approach that analyzed case studies and literature as primary data collection methods, focusing on strategies to mitigate biases through technical solutions, diverse datasets, and strict adherence to ethical guidelines. Their …

Strategic Integration Of Ai In Higher Education And Industry: The Ai8-Point Model, Emily Barnes, James Hutson

Faculty Scholarship

The AI8-Point Model, derived from extensive experience in technology, AI, and higher education administration, addresses the critical need for cost-effective, high-impact strategies tailored to higher education. Despite the transformative potential of AI in enhancing student engagement, optimizing processes, and improving educational outcomes, institutions often struggle with practical implementation. The AI8-Point Model fills this gap by offering strategies that balance cost and impact. Visualized as a circle divided into four quadrants, the model encompasses phases of student engagement and institutional interaction: pre-enrollment beyond institutional control, pre-enrollment within institutional control, post-enrollment within institutional control, and post-enrollment beyond institutional control. Each quadrant contains …

On Coresets For Fair Clustering In Metric And Euclidean Spaces And Their Applications, Sayan Bandyapadhyay, Fedor V. Fomin, Kirill Simonov

Computer Science Faculty Publications and Presentations

Fair clustering is a constrained clustering problem where we need to partition a set of colored points. The fraction of points of each color in every cluster should be more or less equal to the fraction of points of this color in the dataset. The problem was recently introduced by Chierichetti et al. (2017) [1]. We propose a new construction of coresets for fair clustering for Euclidean and general metrics based on random sampling. For the Euclidean space Rd, we provide the first coreset whose size does not depend exponentially on the dimension d. The question of whether such constructions …

Auditory Ace Mobile Application Capstone Review, Layla Smith

University Honors Theses

This paper describes the development process and outcomes of my 2023-2024 Capstone Project, Auditory Ace, a self-directed auditory training mobile application for individuals with cochlear implants. Recognizing the limitations of current market offerings, Dr. Timothy Anderson created a Capstone project proposal to develop an accessible auditory training mobile application. The Capstone team that took on this proposal consisted of Darya Haines, Dustin Huynh, Jordan Nguyen, Nihar Koppolu, Scott Thorkelson, Sienna Day, and myself, Layla Smith. This paper is structured to follow the Agile software development methodology, which we used to develop Auditory Ace, reviewing in detail every major choice we …

Virtual Field Environments Capstone Software Review, Ashton Sawyer

University Honors Theses

This is a review of the Virtual Field Environments computer science capstone project, sponsored by geology professor Rick Hugo. The tool aims to create and render VFEs, interactable 360° environments hosted on the web that are used as virtual field trips for K-12 students. This essay discusses the development process, including understanding requirements, tool and technology selection, problem-solving, and decision-making strategies. It also highlights the differences between the capstone and the other core computer science courses, and how those differences help to prepare students for the workforce. The project was completed over the course of twenty weeks by a team …

To Protect Or To Hide: An Investigation On Corporate Redacted Disclosure Motives Under New Fast Act Regulation, Yan Ma, Qian Mao, Nan Hu

Research Collection School Of Computing and Information Systems

China adopted amendments allowing companies to redact filings without prior approval in 2016. Leveraging this change as a quasi-nature experiment, we explore whether managers utilize redacted information to withhold bad information in the more lenient regulatory environment. Our investigation uncovers a significant shift in managerial behavior: Since 2016, managers incline to employ redactions to obscure negative news rather than safeguarding proprietary data. Furthermore, we find that the poorer firm performance and a higher cost of equity are associated with the redacted disclosures after 2016, suggesting that investors perceive an increase in firm-specific risk attributed to withholding bad news through redactions.

Friendly Sharpness-Aware Minimization, Tao Li, Pan Zhou, Zhengbao He, Xinwen Cheng, Xiaolin Huang

Research Collection School Of Computing and Information Systems

Sharpness-Aware Minimization (SAM) has been instrumental in improving deep neural network training by minimizing both training loss and loss sharpness. Despite the practical success, the mechanisms behind SAM’s generalization enhancements remain elusive, limiting its progress in deep learning optimization. In this work, we investigate SAM’s core components for generalization improvement and introduce “Friendly-SAM” (F-SAM) to further enhance SAM’s generalization. Our investigation reveals the key role of batch-specific stochastic gradient noise within the adversarial perturbation, i.e., the current minibatch gradient, which significantly influences SAM’s generalization performance. By decomposing the adversarial perturbation in SAM into full gradient and stochastic gradient noise components, …

Diffusion Time-Step Curriculum For One Image To 3d Generation, Xuanyu Yi, Zike Wu, Qingshan Xu, Pan Zhou, Joo Hwee Lim, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Score distillation sampling (SDS) has been widely adopted to overcome the absence of unseen views in reconstructing 3D objects from a single image. It leverages pretrained 2D diffusion models as teacher to guide the reconstruction of student 3D models. Despite their remarkable success, SDS-based methods often encounter geometric artifacts and texture saturation. We find out the crux is the overlooked indiscriminate treatment of diffusion time-steps during optimization: it unreasonably treats the studentteacher knowledge distillation to be equal at all time-steps and thus entangles coarse-grained and fine-grained modeling. Therefore, we propose the Diffusion Time-step Curriculum one-image-to-3D pipeline (DTC123), which involves both …

Detecting Foot Strikes During Running With Earbuds, Changshuo Hu, Thivya Kandappu, Jake Stuchbury-Wass, Yang Liu, Anthony Tang, Cecelia Mascolo, Dong Ma

Research Collection School Of Computing and Information Systems

Running is a widely embraced form of aerobic exercise, offering various physical and mental benefits. However, improper running gaits (i.e., the way of foot landing) can pose safety risks and impact running efficiency. As many runners lack the knowledge or continuous attention to manage their foot strikes during running, in this work, we present a portable and non-invasive running gait monitoring system. Specifically, we leverage the in-ear microphone on wireless earbuds to capture the vibrations generated by foot strikes. Landing with different parts of the foot (e.g., forefoot and heel) generates distinct vibration patterns, and thus we utilize machine learning …

How Is Our Mobility Affected As We Age? Findings From A 934 Users Field Study Of Older Adults Conducted In An Urban Asian City, Yi Zhen Tan, Ngoc Doan Thu Tran, Sapphire Lin, Fang Zhao, Yee Sien Ng, Dong Ma, Jeonggil Ko, Rajesh Krishna Balan

Research Collection School Of Computing and Information Systems

In this paper, we analyze the results of a large study involving 934 older adults living in an urban Asian city that collected their mobility patterns, in the form of logged GPS data, along with a multitude of demographic and health data. We show that mobility, in terms of average distance travelled per day, is greatly affected by age and by employment status. In addition, other factors such as type of day, household size, physical and financial conditions and the onset of retirement also play a significant role in determining the mobility of an individual. These results will have high …

Ethical Considerations Toward Protestware, Marc Cheong, Raula Kula, Christoph Treude

Research Collection School Of Computing and Information Systems

This article looks into possible scenarios where developers might consider turning their free and open source software into protestware. Using different frameworks commonly used in artificial intelligence (AI) ethics, we extend the applications of AI ethics to the study of protestware.

Consistent3d: Towards Consistent High-Fidelity Text-To-3d Generation With Deterministic Sampling Prior, Zike Wu, Pan Zhou, Xuanyu Yi, Xiaoding Yuan, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Score distillation sampling (SDS) and its variants have greatly boosted the development of text-to-3D generation, but are vulnerable to geometry collapse and poor textures yet. To solve this issue, we first deeply analyze the SDS and find that its distillation sampling process indeed corresponds to the trajectory sampling of a stochastic differential equation (SDE): SDS samples along an SDE trajectory to yield a less noisy sample which then serves as a guidance to optimize a 3D model. However, the randomness in SDE sampling often leads to a diverse and unpredictable sample which is not always less noisy, and thus is …

Closest Pairs Search Over Data Stream, Rui Zhu Zhu, Bin Wang, Xiaochun Yang, Baihua Zheng

Research Collection School Of Computing and Information Systems

��-closest pair (KCP for short) search is a fundamental problem in database research. Given a set of��-dimensional streaming data S, KCP search aims to retrieve �� pairs with the shortest distances between them. While existing works have studied continuous 1-closest pair query (i.e., �� = 1) over dynamic data environments, which allow for object insertions/deletions, they require high computational costs and cannot easily support KCP search with �� > 1. This paper investigates the problem of KCP search over data stream, aiming to incrementally maintain as few pairs as possible to support KCP search with arbitrarily ��. To achieve this, we …

Improving Interpretable Embeddings For Ad-Hoc Video Search With Generative Captions And Multi-Word Concept Bank, Jiaxin Wu, Chong-Wah Ngo, Wing-Kwong Chan

Research Collection School Of Computing and Information Systems

Aligning a user query and video clips in cross-modal latent space and that with semantic concepts are two mainstream approaches for ad-hoc video search (AVS). However, the effectiveness of existing approaches is bottlenecked by the small sizes of available video-text datasets and the low quality of concept banks, which results in the failures of unseen queries and the out-of-vocabulary problem. This paper addresses these two problems by constructing a new dataset and developing a multi-word concept bank. Specifically, capitalizing on a generative model, we construct a new dataset consisting of 7 million generated text and video pairs for pre-training. To …

Perceptions And Aspirations Of Undergraduate Computer Science Students Towards Generative Ai: A Qualitative Inquiry, James Hutson, Theresa Jeevanjee

Faculty Scholarship

This article presents a comprehensive study conducted during the spring semester of 2024, aimed at exploring undergraduate computer science students’ perceptions, awareness, and understanding of generative artificial intelligence (GAI) tools within the context of their Artificial Intelligence (AI) courses. The research methodology employed qualitative techniques, including human-subject research and focus groups, to delve into students’ insights on the evolution of AI as delineated in the seminal textbook by Russell and Norvig. The study-initiated discussions on the historical development of AI, prompting students to reflect on the aspects that intrigued them the most, and to identify which historical concepts and methodologies, …

Predictive Power Of Machine Learning Models On Degree Completion Among Adult Learners, Emily Barnes, James Hutson, Karriem Perry

Faculty Scholarship

The integration of machine learning (ML) into higher education has been recognized as a transformative force for adult learners, a growing demographic facing unique educational challenges. This study evaluates the predictive power of three ML models—Random Forest, Gradient-Boosting Machine, and Decision Trees—in forecasting degree completion among this group. Utilizing a dataset from the academic years 2013-14 to 2021-22, which includes demographic and academic performance metrics, the study employs accuracy, precision, recall, and F1 score to assess the efficacy of these models. The results indicate that the Gradient-Boosting Machine model outperforms others in predicting degree completion, suggesting that ML can significantly …

Design And Implementation Of A Vision-Based Deep-Learning Protocol For Kinematic Feature Extraction With Application To Stroke Rehabilitation, Juan Diego Luna Inga

Master's Theses

Stroke is a leading cause of long-term disability, affecting thousands of individuals annually and significantly impairing their mobility, independence, and quality of life. Traditional methods for assessing motor impairments are often costly and invasive, creating substantial barriers to effective rehabilitation. This thesis explores the use of DeepLabCut (DLC), a deep-learning-based pose estimation tool, to extract clinically meaningful kinematic features from video data of stroke survivors with upper-extremity (UE) impairments.

To conduct this investigation, a specialized protocol was developed to tailor DLC for analyzing movements characteristic of UE impairments in stroke survivors. This protocol was validated through comparative analysis using peak …

Evaluating Methods For Assessing Interpretability Of Deep Neural Networks (Dnns), Emily Barnes, James Hutson

Faculty Scholarship

The interpretability of deep neural networks (DNNs) is a critical focus in artificial intelligence (AI) and machine learning (ML), particularly as these models are increasingly deployed in high-stakes applications such as healthcare, finance, and autonomous systems. In the context of these technologies, interpretability refers to the extent to which a human can understand the cause of a decision made by a model. This article evaluates various methods for assessing the interpretability of DNNs, recognizing the significant challenges posed by their complex and opaque nature. The review encompasses both quantitative metrics and qualitative evaluations, aiming to identify effective strategies that enhance …

Present Case Studies Highlighting Practical Implications Of Architectural Design Choices, Emily Barnes, James Hutson

Faculty Scholarship

The interpretability of deep neural networks (DNNs) has become a crucial focus within artificial intelligence and machine learning, particularly as these models are increasingly used in high-stakes applications such as healthcare, finance, and autonomous driving. This article explores the impact of architectural design choices on the interpretability of DNNs, emphasizing the importance of transparency, trust, and accountability in AI systems. By presenting case studies and experimental results, the article highlights how different architectural elements—such as layer types, network depth, connectivity patterns, and attention mechanisms—affect model interpretability and performance. The discussion is structured into three main sections: real-world applications, architectural trade-offs, …

Architectural Elements Contributing To Interpretability Of Deep Neural Networks (Dnns), Emily Barnes, James Hutson

Faculty Scholarship

The interpretability of Deep Neural Networks (DNNs) has become a critical focus in artificial intelligence and machine learning, particularly as DNNs are increasingly used in high-stakes applications like healthcare, finance, and autonomous driving. Interpretability refers to the extent to which humans can understand the reasons behind a model's decisions, which is essential for trust, accountability, and transparency. However, the complexity and depth of DNN architectures often compromise interpretability as these models function as "black boxes." This article reviews key architectural elements of DNNs that affect their interpretability, aiming to guide the design of more transparent and trustworthy models. The primary …

Navigating The Complexities Of Ai: The Critical Role Of Interpretability And Explainability In Ensuring Transparency And Trust, Emily Barnes, James Hutson

Faculty Scholarship

The interpretability and explainability of deep neural networks (DNNs) are paramount in artificial intelligence (AI), especially when applied to high-stakes fields such as healthcare, finance, and autonomous driving. The need for this study arises from the growing integration of AI into critical areas where transparency, trust, and ethical decision-making are essential. This paper explores the impact of architectural design choices on DNN interpretability, focusing on how different architectural elements like layer types, network depth, connectivity patterns, and attention mechanisms affect model transparency. Methodologically, the study employs a comprehensive review of case studies and experimental results to analyze the balance between …

Performance Interference Detection For Cloud-Native Applications Using Unsupervised Machine Learning Models, Eli Bakshi

Master's Theses

Contemporary cloud-native applications frequently adopt the microservice architecture, where applications are deployed within multiple containers that run on cloud virtual machines (VMs). These applications are typically hosted on public cloud platforms, where VMs from multiple cloud subscribers compete for the same physical resources on a cloud server. When a cloud subscriber application running on a VM competes for shared physical resources from other applications running on the same VM or from other VMs co-located on the same cloud server, performance interference may occur when the performance of an application degrades due to shared resource contention. Detecting such interference is crucial …

Morp: Monocular Orientation Regression Pipeline, Jacob Gunderson

Master's Theses

Orientation estimation of objects plays a pivotal role in robotics, self-driving cars, and augmented reality. Beyond mere position, accurately determining the orientation of objects is essential for constructing precise models of the physical world. While 2D object detection has made significant strides, the field of orientation estimation still faces several challenges. Our research addresses these hurdles by proposing an efficient pipeline which facilitates rapid creation of labeled training data and enables direct regression of object orientation from a single image. We start by creating a digital twin of a physical object using an iPhone, followed by generating synthetic images using …

D-Hacking, Emily Black, Talia B. Gillis, Zara Hall

Faculty Scholarship

Recent regulatory efforts, including Executive Order 14110 and the AI Bill of Rights, have focused on mitigating discrimination in AI systems through novel and traditional application of anti-discrimination laws. While these initiatives rightly emphasize fairness testing and mitigation, we argue that they pay insufficient attention to robust bias measurement and mitigation — and that without doing so, the frameworks cannot effectively achieve the goal of reducing discrimination in deployed AI models. This oversight is particularly concerning given the instability and brittleness of current algorithmic bias mitigation and fairness optimization methods, as highlighted by growing evidence in the algorithmic fairness literature. …

Interpretable Learning In Multivariate Big Data Analysis For Network Monitoring, José Camacho, Katarzyna Wasielewska, Rasmus Bro, David Kotz

Dartmouth Scholarship

There is an increasing interest in the development of new data-driven models useful to assess the performance of communication networks. For many applications, like network monitoring and troubleshooting, a data model is of little use if it cannot be interpreted by a human operator. In this paper, we present an extension of the Multivariate Big Data Analysis (MBDA) methodology, a recently proposed interpretable data analysis tool. In this extension, we propose a solution to the automatic derivation of features, a cornerstone step for the application of MBDA when the amount of data is massive. The resulting network monitoring approach allows …

Automated Sensor Node Malicious Activity Detection With Explainability Analysis, Md Zubair, Helge Janicke, Ahmad Mohsin, Leandros Maglaras, Iqbal H. Sarker

Research outputs 2022 to 2026

Cybersecurity has become a major concern in the modern world due to our heavy reliance on cyber systems. Advanced automated systems utilize many sensors for intelligent decision-making, and any malicious activity of these sensors could potentially lead to a system-wide collapse. To ensure safety and security, it is essential to have a reliable system that can automatically detect and prevent any malicious activity, and modern detection systems are created based on machine learning (ML) models. Most often, the dataset generated from the sensor node for detecting malicious activity is highly imbalanced because the Malicious class is significantly fewer than the …