Physical Sciences and Mathematics Commons^™

Assessing Job Vulnerability And Employment Growth In The Era Of Large Language Models (Llms), Prudence P. Brou

Dissertations, Theses, and Capstone Projects

This paper explores the impact of Large Language Models (LLMs) and artificial intelligence (AI) on white-collar occupations in the context of job vulnerability and employment growth. Utilizing the Kaggle dataset "Occupation Salary and Likelihood of Automation," the study employs a data-driven approach to analyze trends across states. Through interactive data visualization, the project aims to provide actionable insights for affected workers, businesses, and policymakers navigating the changing dynamics of the workforce amidst technological advancements.

Boring But Demanding: Using Secondary Tasks To Counter The Driver Vigilance Decrement For Partially Automated Driving, Scott Mishler, Jing Chen

Psychology Faculty Publications

Objective

We investigated secondary–task–based countermeasures to the vigilance decrement during a simulated partially automated driving (PAD) task, with the goal of understanding the underlying mechanism of the vigilance decrement and maintaining driver vigilance in PAD.

Background

Partial driving automation requires a human driver to monitor the roadway, but humans are notoriously bad at monitoring tasks over long periods of time, demonstrating the vigilance decrement in such tasks. The overload explanations of the vigilance decrement predict the decrement to be worse with added secondary tasks due to increased task demands and depleted attentional resources, whereas the underload explanations predict the vigilance …

Violet: Visual Analytics For Explainable Quantum Neural Networks, Shaolun Ruan, Zhiding Liang, Qiang Guan, Paul Robert Griffin, Xiaolin Wen, Yanna Lin, Yong Wang

Research Collection School Of Computing and Information Systems

With the rapid development of Quantum Machine Learning, quantum neural networks (QNN) have experienced great advancement in the past few years, harnessing the advantages of quantum computing to significantly speed up classical machine learning tasks. Despite their increasing popularity, the quantum neural network is quite counter-intuitive and difficult to understand, due to their unique quantum-specific layers (e.g., data encoding and measurement) in their architecture. It prevents QNN users and researchers from effectively understanding its inner workings and exploring the model training status. To fill the research gap, we propose VIOLET , a novel visual analytics approach to improve the explainability …

Poster: Profiling Event Vision Processing On Edge Devices, Ila Nitin Gokarn, Archan Misra

Research Collection School Of Computing and Information Systems

As RGB camera resolutions and frame-rates improve, their increased energy requirements make it challenging to deploy fast, efficient, and low-power applications on edge devices. Newer classes of sensors, such as the biologically inspired neuromorphic event-based camera, capture only changes in light intensity per-pixel to achieve operational superiority in sensing latency (O(μs)), energy consumption (O(mW)), high dynamic range (140dB), and task accuracy such as in object tracking, over traditional RGB camera streams. However, highly dynamic scenes can yield an event rate of up to 12MEvents/second, the processing of which could overwhelm …

Predicting Mild Cognitive Impairment Through Ambient Sensing And Artificial Intelligence, Ah-Hwee Tan, Weng Yan Ying, Budhitama Subagdja, Anni Huang, Shanthoshigaa D, Tony Chin-Ian Tay, Iris Rawtaer

Research Collection School Of Computing and Information Systems

This paper reports an emerging application leveraging ambient and artificial intelligence techniques for in-home sensing and cognitive health assessment. The application involves a prospective longitudinal study, wherein non-pervasive sensing devices are installed in homes of over 63 real users undergoing clinical cognitive assessment, and digital signals of the users’ activities and behaviour are transmitted to a central cloud-based data server for further processing and analysis. Based on the sensor readings, we identify a set of digital biomarkers covering four key aspects of daily living, namely physical, activity, cognitive, and sleep, and develop a suite of customized feature extraction methods for …

Criticality Aware Canvas-Based Visual Perception At The Edge, Ila Gokarn

Research Collection School Of Computing and Information Systems

Efficient and effective machine perception remains a formidable challenge in sustaining high fidelity and high throughput of perception tasks on affordable edge devices. This is especially due to the continuing increase in resolution of sensor streams (e.g., video input streams generated by 4K/8K cameras and neuromorphic event cameras that produce ≥ 10 MEvents/second) and computational complexity of Deep Neural Network (DNN) models, which overwhelms edge platforms, adversely impacting machine perception efficiency. Given the insufficiency of the available computation resources, a question then arises on whether selected regions/components of the perception task can be prioritized (and executed preferentially) to achieve highest …

Enhancing Robustness Of Machine Learning Models Against Adversarial Attacks, Ronak Guliani

University Honors Theses

Machine learning models are integral for numerous applications, but they remain increasingly vulnerable to adversarial attacks. These attacks involve subtle manipulation of input data to deceive models, presenting a critical threat to their dependability and security. This thesis addresses the need for strengthening these models against such adversarial attacks. Prior research has primarily focused on identifying specific types of adversarial attacks on a limited range of ML algorithms. However, there is a gap in the evaluation of model resilience across algorithms and in the development of effective defense mechanisms. To bridge this gap, this work adopts a two-phase approach. First, …

Semantic Structuring Of Digital Documents: Knowledge Graph Generation And Evaluation, Erik E. Luu

Master's Theses

In the era of total digitization of documents, navigating vast and heterogeneous data landscapes presents significant challenges for effective information retrieval, both for humans and digital agents. Traditional methods of knowledge organization often struggle to keep pace with evolving user demands, resulting in suboptimal outcomes such as information overload and disorganized data. This thesis presents a case study on a pipeline that leverages principles from cognitive science, graph theory, and semantic computing to generate semantically organized knowledge graphs. By evaluating a combination of different models, methodologies, and algorithms, the pipeline aims to enhance the organization and retrieval of digital documents. …

Accessible Real-Time Eye-Gaze Tracking For Neurocognitive Health Assessments, A Multimodal Web-Based Approach, Daniel C. Tisdale

Master's Theses

We introduce a novel integration of real-time, predictive eye-gaze tracking models into a multimodal dialogue system tailored for remote health assessments. This system is designed to be highly accessible requiring only a conventional webcam for video input along with minimal cursor interaction and utilizes engaging gaze-based tasks that can be performed directly in a web browser. We have crafted dynamic subsystems that capture high-quality data efficiently and maintain quality through instances of user attrition and incomplete calls. Additionally, these subsystems are designed with the foresight to allow for future re-analysis using improved predictive models, as well as enable the creation …

An Experimental Study Of Supervised Machine Learning Techniques For Minor Class Prediction Utilizing Kernel Density Estimation: Factors Impacting Model Performance, Abdullah Mana Alfarwan

Dissertations

This dissertation examined classification outcome differences among four popular individual supervised machine learning (ISML) models (logistic regression, decision tree, support vector machine, and multilayer perceptron) when predicting minor class membership within imbalanced datasets. The study context and the theoretical population sampled focus on one aspect of the larger problem of student retention and dropout prediction in higher education (HE): identification.

This study differs from current literature by implementing an experimental design approach with simulated student data that closely mirrors HE situational and student data. Specifically, this study tested the predictive ability of the four ISML classification models (CLS) under experimentally …

Navigating The Ethical Terrain Of Ai In Higher Education: Strategies For Mitigating Bias And Promoting Fairness, Emily Barnes, James Hutson

Faculty Scholarship

Artificial intelligence (AI) and machine learning (ML) are transforming higher education by enhancing personalized learning and academic support, yet they pose significant ethical challenges, particularly in terms of inherent biases. This review critically examines the integration of AI in higher education, underscoring the dual aspects of its potential to innovate educational paradigms and the essential need to address ethical implications to avoid perpetuating existing inequalities. The researchers employed a methodological approach that analyzed case studies and literature as primary data collection methods, focusing on strategies to mitigate biases through technical solutions, diverse datasets, and strict adherence to ethical guidelines. Their …

Strategic Integration Of Ai In Higher Education And Industry: The Ai8-Point Model, Emily Barnes, James Hutson

Faculty Scholarship

The AI8-Point Model, derived from extensive experience in technology, AI, and higher education administration, addresses the critical need for cost-effective, high-impact strategies tailored to higher education. Despite the transformative potential of AI in enhancing student engagement, optimizing processes, and improving educational outcomes, institutions often struggle with practical implementation. The AI8-Point Model fills this gap by offering strategies that balance cost and impact. Visualized as a circle divided into four quadrants, the model encompasses phases of student engagement and institutional interaction: pre-enrollment beyond institutional control, pre-enrollment within institutional control, post-enrollment within institutional control, and post-enrollment beyond institutional control. Each quadrant contains …

Perceptions And Aspirations Of Undergraduate Computer Science Students Towards Generative Ai: A Qualitative Inquiry, James Hutson, Theresa Jeevanjee

Faculty Scholarship

This article presents a comprehensive study conducted during the spring semester of 2024, aimed at exploring undergraduate computer science students’ perceptions, awareness, and understanding of generative artificial intelligence (GAI) tools within the context of their Artificial Intelligence (AI) courses. The research methodology employed qualitative techniques, including human-subject research and focus groups, to delve into students’ insights on the evolution of AI as delineated in the seminal textbook by Russell and Norvig. The study-initiated discussions on the historical development of AI, prompting students to reflect on the aspects that intrigued them the most, and to identify which historical concepts and methodologies, …

Predictive Power Of Machine Learning Models On Degree Completion Among Adult Learners, Emily Barnes, James Hutson, Karriem Perry

Faculty Scholarship

The integration of machine learning (ML) into higher education has been recognized as a transformative force for adult learners, a growing demographic facing unique educational challenges. This study evaluates the predictive power of three ML models—Random Forest, Gradient-Boosting Machine, and Decision Trees—in forecasting degree completion among this group. Utilizing a dataset from the academic years 2013-14 to 2021-22, which includes demographic and academic performance metrics, the study employs accuracy, precision, recall, and F1 score to assess the efficacy of these models. The results indicate that the Gradient-Boosting Machine model outperforms others in predicting degree completion, suggesting that ML can significantly …

Design And Implementation Of A Vision-Based Deep-Learning Protocol For Kinematic Feature Extraction With Application To Stroke Rehabilitation, Juan Diego Luna Inga

Master's Theses

Stroke is a leading cause of long-term disability, affecting thousands of individuals annually and significantly impairing their mobility, independence, and quality of life. Traditional methods for assessing motor impairments are often costly and invasive, creating substantial barriers to effective rehabilitation. This thesis explores the use of DeepLabCut (DLC), a deep-learning-based pose estimation tool, to extract clinically meaningful kinematic features from video data of stroke survivors with upper-extremity (UE) impairments.

To conduct this investigation, a specialized protocol was developed to tailor DLC for analyzing movements characteristic of UE impairments in stroke survivors. This protocol was validated through comparative analysis using peak …

Evaluating Methods For Assessing Interpretability Of Deep Neural Networks (Dnns), Emily Barnes, James Hutson

Faculty Scholarship

The interpretability of deep neural networks (DNNs) is a critical focus in artificial intelligence (AI) and machine learning (ML), particularly as these models are increasingly deployed in high-stakes applications such as healthcare, finance, and autonomous systems. In the context of these technologies, interpretability refers to the extent to which a human can understand the cause of a decision made by a model. This article evaluates various methods for assessing the interpretability of DNNs, recognizing the significant challenges posed by their complex and opaque nature. The review encompasses both quantitative metrics and qualitative evaluations, aiming to identify effective strategies that enhance …

Present Case Studies Highlighting Practical Implications Of Architectural Design Choices, Emily Barnes, James Hutson

Faculty Scholarship

The interpretability of deep neural networks (DNNs) has become a crucial focus within artificial intelligence and machine learning, particularly as these models are increasingly used in high-stakes applications such as healthcare, finance, and autonomous driving. This article explores the impact of architectural design choices on the interpretability of DNNs, emphasizing the importance of transparency, trust, and accountability in AI systems. By presenting case studies and experimental results, the article highlights how different architectural elements—such as layer types, network depth, connectivity patterns, and attention mechanisms—affect model interpretability and performance. The discussion is structured into three main sections: real-world applications, architectural trade-offs, …

Architectural Elements Contributing To Interpretability Of Deep Neural Networks (Dnns), Emily Barnes, James Hutson

Faculty Scholarship

The interpretability of Deep Neural Networks (DNNs) has become a critical focus in artificial intelligence and machine learning, particularly as DNNs are increasingly used in high-stakes applications like healthcare, finance, and autonomous driving. Interpretability refers to the extent to which humans can understand the reasons behind a model's decisions, which is essential for trust, accountability, and transparency. However, the complexity and depth of DNN architectures often compromise interpretability as these models function as "black boxes." This article reviews key architectural elements of DNNs that affect their interpretability, aiming to guide the design of more transparent and trustworthy models. The primary …

Navigating The Complexities Of Ai: The Critical Role Of Interpretability And Explainability In Ensuring Transparency And Trust, Emily Barnes, James Hutson

Faculty Scholarship

The interpretability and explainability of deep neural networks (DNNs) are paramount in artificial intelligence (AI), especially when applied to high-stakes fields such as healthcare, finance, and autonomous driving. The need for this study arises from the growing integration of AI into critical areas where transparency, trust, and ethical decision-making are essential. This paper explores the impact of architectural design choices on DNN interpretability, focusing on how different architectural elements like layer types, network depth, connectivity patterns, and attention mechanisms affect model transparency. Methodologically, the study employs a comprehensive review of case studies and experimental results to analyze the balance between …

D-Hacking, Emily Black, Talia B. Gillis, Zara Hall

Faculty Scholarship

Recent regulatory efforts, including Executive Order 14110 and the AI Bill of Rights, have focused on mitigating discrimination in AI systems through novel and traditional application of anti-discrimination laws. While these initiatives rightly emphasize fairness testing and mitigation, we argue that they pay insufficient attention to robust bias measurement and mitigation — and that without doing so, the frameworks cannot effectively achieve the goal of reducing discrimination in deployed AI models. This oversight is particularly concerning given the instability and brittleness of current algorithmic bias mitigation and fairness optimization methods, as highlighted by growing evidence in the algorithmic fairness literature. …

Context In Computer Vision: A Taxonomy, Multi-Stage Integration, And A General Framework, Xuan Wang

Dissertations, Theses, and Capstone Projects

Contextual information has been widely used in many computer vision tasks, such as object detection, video action detection, image classification, etc. Recognizing a single object or action out of context could be sometimes very challenging, and context information may help improve the understanding of a scene or an event greatly. However, existing approaches design specific contextual information mechanisms for different detection tasks.

In this research, we first present a comprehensive survey of context understanding in computer vision, with a taxonomy to describe context in different types and levels. Then we proposed MultiCLU, a new multi-stage context learning and utilization framework, …

Morp: Monocular Orientation Regression Pipeline, Jacob Gunderson

Master's Theses

Orientation estimation of objects plays a pivotal role in robotics, self-driving cars, and augmented reality. Beyond mere position, accurately determining the orientation of objects is essential for constructing precise models of the physical world. While 2D object detection has made significant strides, the field of orientation estimation still faces several challenges. Our research addresses these hurdles by proposing an efficient pipeline which facilitates rapid creation of labeled training data and enables direct regression of object orientation from a single image. We start by creating a digital twin of a physical object using an iPhone, followed by generating synthetic images using …

Sensing With Integrity: Responsible Sensor Systems In An Era Of Ai, David Eisenberg

Dissertations

Deep and machine learning now offer immense benefits for consumer choice, decision-making, medicine, mental health and education, smart cities, and intelligent transportation and driver safety. However, as communication and Internet technology further advances, these benefits have the potential to be outweighed by compromises to privacy, personal freedom, consumer trust, and discrimination. While ethical consequences for personal freedom and equity rise from these technological advances, the issue may not be the technology itself but a lack of regulation and policy that allow abuses to occur. A first study examines how emerging sensor-based technologies, limited to only accelerometer and gyroscope data from …

Charting A Path To The Quintuple Aim: Harnessing Ai To Address Social Determinants Of Health, Yash Shah, Zachary Goldberg, Erika Harness, David Nash

College of Population Health Faculty Papers

The Quintuple Aim seeks to improve healthcare by addressing social determinants of health (SDOHs), which are responsible for 70-80% of medical outcomes. SDOH-related concerns have traditionally been addressed through referrals to social workers and community-based organizations (CBOs), but these pathways have had limited success in connecting patients with resources. Given that health inequity is expected to cost the United States nearly USD 300 billion by 2050, new artificial intelligence (AI) technology may aid providers in addressing SDOH. In this commentary, we present our experience with using ChatGPT to obtain SDOH management recommendations for archetypal patients in Philadelphia, PA. ChatGPT identified …

Try It Together - Qualitative Coding With Atlas.Ti, Danping Dong, Bryan Leow

AI for Research Week

This hands-on session introduces Atlas.ti, a well-established qualitative data analysis tool for analyzing your transcripts and textual data. The session will cover coding data, extracting insights, creating visualizations, and exploring the tool's latest AI features.

Try It Together: Transcribing Your Audio With Whisper Api, Bella Ratmelia

AI for Research Week

In this hands-on session, we will explore using the Whisper API to transcribe audio recordings from interviews, focus groups, and speeches. The session will delve into best practices and address common issues that may arise during the transcription process.

Singleadv: Single-Class Target-Specific Attack Against Interpretable Deep Learning Systems, Eldor Abdukhamidov, Mohammed Abuhamad, George K. Thiruvathukal, Hyoungshick Kim, Tamer Abuhmed

Computer Science: Faculty Publications and Other Works

In this paper, we present a novel Single-class target-specific Adversarial attack called SingleADV. The goal of SingleADV is to generate a universal perturbation that deceives the target model into confusing a specific category of objects with a target category while ensuring highly relevant and accurate interpretations. The universal perturbation is stochastically and iteratively optimized by minimizing the adversarial loss that is designed to consider both the classifier and interpreter costs in targeted and non-targeted categories. In this optimization framework, ruled by the first- and second-moment estimations, the desired loss surface promotes high confidence and interpretation score of adversarial samples. By …

Detecting Drifts In Data Streams Using Kullback-Leibler (Kl) Divergence Measure For Data Engineering Applications, Jeomoan Francis Kurian, Mohamed Allali

Engineering Faculty Articles and Research

The exponential growth of data coupled with the widespread application of artificial intelligence(AI) presents organizations with challenges in upholding data accuracy, especially within data engineering functions. While the Extraction, Transformation, and Loading process addresses error-free data ingestion, validating the content within data streams remains a challenge. Prompt detection and remediation of data issues are crucial, especially in automated analytical environments driven by AI. To address these issues, this study focuses on detecting drifts in data distributions and divergence within data fields processed from different sample populations. Using a hypothetical banking scenario, we illustrate the impact of data drift on automated …

Automatic Measurement Of Dialogue Engagingness In Multilingual Settings, Amila Ferron

Dissertations and Theses

Expansive use of large language models (LLMs) as dialogue systems brings increased importance to the evaluation of the responses they generate. Although evaluation of qualities such as coherence and fluency are readily possible with well-established automatic metrics, engagingness is often measured with human evaluation -- a process that can be costly and slows the pace of development. Existing automatic metrics for engagingness have low to moderate correlation with human annotations, evaluate the response without the conversation history, are complicated to implement, or are designed for a specific dataset. Moreover, they have been tested exclusively on English conversations. Given that dialogue …

Academic Search And Discovery Tools In The Age Of Ai And Large Language Models: An Overview Of The Space, Aaron Tay

AI for Research Week

In the ever-evolving landscape of academic research, “AI tools” for literature search and synthesis are currently getting a lot of attention. These tools promise to ramp up productivity, enabling us to accomplish more in less time or absorb more knowledge without drowning in endless reading. With the sheer number of these systems increasing daily, it's natural to wonder: are they really worth our time and money? And if they are, how should we go about picking the right one from the multitude of options?

In this talk, I will share my views on how the space has developed over two …