Physical Sciences and Mathematics Commons^™

Toward Intuitive 3d Interactions In Virtual Reality: A Deep Learning- Based Dual-Hand Gesture Recognition Approach, Trudi Di Qi, Franceli L. Cibrian, Meghna Raswan, Tyler Kay, Hector M. Camarillo-Abad, Yuxin Wen

Engineering Faculty Articles and Research

Dual-hand gesture recognition is crucial for intuitive 3D interactions in virtual reality (VR), allowing the user to interact with virtual objects naturally through gestures using both handheld controllers. While deep learning and sensor-based technology have proven effective in recognizing single-hand gestures for 3D interactions, research on dual-hand gesture recognition for VR interactions is still underexplored. In this work, we introduce CWT-CNN-TCN, a novel deep learning model that combines a 2D Convolution Neural Network (CNN) with Continuous Wavelet Transformation (CWT) and a Temporal Convolution Network (TCN). This model can simultaneously extract features from the time-frequency domain and capture long-term dependencies using …

Next-Generation Crop Monitoring Technologies: Case Studies About Edge Image Processing For Crop Monitoring And Soil Water Property Modeling Via Above-Ground Sensors, Nipuna Chamara

Dissertations and Doctoral Documents from University of Nebraska-Lincoln, 2023–

Artificial Intelligence (AI) has advanced rapidly in the past two decades. Internet of Things (IoT) technology has advanced rapidly during the last decade. Merging these two technologies has immense potential in several industries, including agriculture.

We have identified several research gaps in utilizing IoT technology in agriculture. One problem was the digital divide between rural, unconnected, or limited connected areas and urban areas for utilizing images for decision-making, which has advanced with the growth of AI. Another area for improvement was the farmers' demotivation to use in-situ soil moisture sensors for irrigation decision-making due to inherited installation difficulties. As Nebraska …

In-Between Frame Generation For 2d Animation Using Generative Adversarial Networks, Francisco Arriaga Pazos

Open Access Theses & Dissertations

Traditional 2D animation remains a largely manual process where each frame in a video is hand-drawn, as no robust algorithmic solutions exist to assist in this process. This project introduces a system that generates intermediate frames in an uncolored 2D animated video sequence using Generative Adversarial Networks (GAN), a deep learning approach widely used for tasks within the creative realm. We treat the task as a frame interpolation problem, and show that adding a GAN dynamic to a system significantly improves the perceptual fidelity of the generated images, as measured by perceptual oriented metrics that aim to capture human judgment …

Dlvs4audio2sheet: Deep Learning-Based Vocal Separation For Audio Into Music Sheet Conversion, Nicole Teo, Zhaoxia Wang, Ezekiel Ghe, Yee Sen Tan, Kevan Oktavio, Alexander Vincent Lewi, Allyne Zhang, Seng-Beng Ho

Research Collection School Of Computing and Information Systems

While manual transcription tools exist, music enthusiasts, including amateur singers, still encounter challenges when transcribing performances into sheet music. This paper addresses the complex task of translating music audio into music sheets, particularly challenging in the intricate field of choral arrangements where multiple voices intertwine. We propose DLVS4Audio2Sheet, a novel method leveraging advanced deep learning models, Open-Unmix and Band-Split Recurrent Neural Networks (BSRNN), for vocal separation. DLVS4Audio2Sheet segments choral audio into individual vocal sections and selects the optimal model for further processing, aiming towards audio into music sheet conversion. We evaluate DLVS4Audio2Sheet’s performance using these deep learning algorithms and assess …

Classification Of Remote Sensing Image Data Using Rsscn-7 Dataset, Satya Priya Challa

Electronic Theses, Projects, and Dissertations

A novel technique for remote sensing image scene classification is employed using the Compact Vision Transformer (CVT) architecture. This model strengthens the power of deep learning and self-attention algorithms to significantly intensify the accuracy and efficiency of scene classification in remote sensing imagery. Through extensive training and evaluation of the RSSCNN7 dataset, our CVT-based model has achieved an impressive accuracy rate of 87.46% on the original dataset. This remarkable result underscores the prospect of CVT models in the domain of remote sensing and underscores their applicability in real-world scenarios. Our report furnishes an elaborate account of the model's architecture, training …

Incremental Image Dehazing Algorithm Based On Multiple Transfer Attention, Jinyang Wei, Keping Wang, Yi Yang, Shumin Fei

Journal of System Simulation

Abstract: In order to improve the processing ability of the depth-neural network dehazing algorithm to the supplementary data set, and to make the network differently process the image features of different importance to improve the dehazing ability of the network, an incremental dehazing algorithm based on multiple migration of attention is proposed. The teacher's attention generation network in the form of Encoder-Decoder extracts the multiple attention of labels and haze, which is used it as the label of the characteristic migration media network to constrain the network training to form the migration media attention as close as possible to the …

Mathematically Rigorous Deep Learning Paradigms For Data-Driven Scientific Modeling, Owen Nicholas Davis

Mathematics & Statistics ETDs

This dissertation explores the crucial role of data-driven modeling in science and engineering, with a focus on developing surrogate models to accelerate large-scale computational tasks, aiding in both outer-loop functions like uncertainty quantification and expensive inner-loop tasks within broader computational frameworks. Challenges arise with increased problem dimension and sparse, noisy training data, particularly significant when constructing surrogates for very expensive computational models where acquiring sufficient high-fidelity training data is unfeasible. In such scenarios, training surrogates from an ensemble of multifidelity information sources of varying accuracy and cost becomes essential. We emphasize neural network-based modeling paradigms, which are flexible in integrating …

Expanding Analytical Capabilities In Intrusion Detection Through Ensemble-Based Multi-Label Classification, Ehsan Hallaji, Roozbeh Razavi-Far, Mehrdad Saif

Electrical and Computer Engineering Publications

Intrusion detection systems are primarily designed to flag security breaches upon their occurrence. These systems operate under the assumption of single-label data, where each instance is assigned to a single category. However, when dealing with complex data, such as malware triage, the information provided by the IDS is limited. Consequently, additional analysis becomes necessary, leading to delays and incurring additional computational costs. Existing solutions to this problem typically merge these steps by considering a unified, but large, label set encompassing both intrusion and analytical labels, which adversely affects efficiency and performance. To address these challenges, this paper presents a novel …

Dronlomaly: Runtime Log-Based Anomaly Detector For Dji Drones, Wei Minn, Naing Tun Yan, Lwin Khin Shar, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

We present an automated tool for realtime detection of anomalous behaviors while a DJI drone is executing a flight mission. The tool takes sensor data logged by drone at fixed time intervals and performs anomaly detection using a Bi-LSTM model. The model is trained on baseline flight logs from a successful mission physically or via a simulator. The tool has two modules --- the first module is responsible for sending the log data to the remote controller station, and the second module is run as a service in the remote controller station powered by a Bi-LSTM model, which receives the …

Enhancing Source Code Representations For Deep Learning With Static Analysis, Xueting Guan, Christoph Treude

Research Collection School Of Computing and Information Systems

Deep learning techniques applied to program analysis tasks such as code classification, summarization, and bug detection have seen widespread interest. Traditional approaches, however, treat programming source code as natural language text, which may neglect significant structural or semantic details. Additionally, most current methods of representing source code focus solely on the code, without considering beneficial additional context. This paper explores the integration of static analysis and additional context such as bug reports and design patterns into source code representations for deep learning models. We use the Abstract Syntax Tree-based Neural Network (ASTNN) method and augment it with additional context information …

Computational Modeling And Analysis Of Facial Expressions And Gaze For Discovery Of Candidate Behavioral Biomarkers For Children And Young Adults With Autism Spectrum Disorder, Megan Anita Witherow

Electrical & Computer Engineering Theses & Dissertations

Facial expression production and perception in autism spectrum disorder (ASD) suggest the potential presence of behavioral biomarkers that may stratify individuals on the spectrum into prognostic or treatment subgroups. High-speed internet and the ease of technology have enabled remote, scalable, affordable, and timely access to medical care, such as measurements of ASDrelated behaviors in familiar environments to complement clinical observation. Machine and deep learning (DL)-based analysis of video tracking (VT) of expression production and eye tracking (ET) of expression perception may aid stratification biomarker discovery for children and young adults with ASD. However, there are open challenges in 1) facial …

Historical Perspectives In Volatility Forecasting Methods With Machine Learning, Zhiang Qiu, Clemens Kownatzki, Fabien Scalzo, Eun Sang Cha

Seaver College Research And Scholarly Achievement Symposium

Volatility forecasting in the financial market plays a pivotal role across a spectrum of disciplines, such as risk management, option pricing, and market making. However, volatility forecasting is challenging because volatility can only be estimated, and different factors influence volatility, ranging from macroeconomic indicators to investor sentiments. While recent works suggest advances in machine learning and artificial intelligence for volatility forecasting, a comprehensive benchmark of current statistical and learning-based methods for such purposes is lacking. Thus, this paper aims to provide a comprehensive survey of the historical evolution of volatility forecasting with a comparative benchmark of key landmark models. We …

Medical Image Super-Resolution For Smart Healthcare Applications: A Comprehensive Survey, Sabina Umirzakova, Shabir Ahmad, Latif U. Khan, Taegkeun Whangbo

Machine Learning Faculty Publications

The digital transformation in healthcare, propelled by the integration of deep learning models and the Internet of Things (IoT), is creating unprecedented opportunities for improving patient care. However, the utilization of low-resolution images, often generated by IoT devices, introduces biases in the deep learning models, thereby affecting the overall clinical decision-making process. While super-resolution techniques have been extensively employed to transform low-resolution images into high-resolution counterparts, the challenge of achieving highly accurate image restoration remains unresolved. This is especially critical in the medical imaging domain, where even minor inaccuracies can lead to significant biases in model training and, consequently, impact …

Understanding The Impact Of Trade Policy Effect Uncertainty On Firm-Level Innovation Investment: A Deep Learning Approach, Daniel Chang, Nan Hu, Peng Liang, Morgan Swink

Research Collection School Of Computing and Information Systems

Integrating the real options perspective and resource dependence theory, this study examines how firms adjust their innovation investments to trade policy effect uncertainty (TPEU), a less studied type of firm specific, perceived environmental uncertainty in which managers have difficulty predicting how potential policy changes will affect business operations. To develop a text-based, context-dependent, time-varying measure of firm-level perceived TPEU, we apply Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art deep learning approach. We apply BERT to analyze the texts of mandatory Management Discussion and Analysis (MD&A) sections of annual reports for a sample of 22,669 firm-year observations from 3,181 unique …

The Impact Of Artificial Intelligence And Machine Learning On Organizations Cybersecurity, Mustafa Abdulhussein

Doctoral Dissertations and Projects

As internet technology proliferate in volume and complexity, the ever-evolving landscape of malicious cyberattacks presents unprecedented security risks in cyberspace. Cybersecurity challenges have been further exacerbated by the continuous growth in the prevalence and sophistication of cyber-attacks. These threats have the capacity to disrupt business operations, erase critical data, and inflict reputational damage, constituting an existential threat to businesses, critical services, and infrastructure. The escalating threat is further compounded by the malicious use of artificial intelligence (AI) and machine learning (ML), which have increasingly become tools in the cybercriminal arsenal. In this dynamic landscape, the emergence of offensive AI introduces …

Anomaly Detection On Small Wind Turbine Blades Using Deep Learning Algorithms, Bridger Altice, Edwin Nazario, Mason Davis, Mohammad Shekaramiz, Todd K. Moon, Mohammad A. S. Masoum

Electrical and Computer Engineering Faculty Publications

Wind turbine blade maintenance is expensive, dangerous, time-consuming, and prone to misdiagnosis. A potential solution to aid preventative maintenance is using deep learning and drones for inspection and early fault detection. In this research, five base deep learning architectures are investigated for anomaly detection on wind turbine blades, including Xception, Resnet-50, AlexNet, and VGG-19, along with a custom convolutional neural network. For further analysis, transfer learning approaches were also proposed and developed, utilizing these architectures as the feature extraction layers. In order to investigate model performance, a new dataset containing 6000 RGB images was created, making use of indoor and …

Motion Magnification-Inspired Feature Manipulation For Deepfake Detection, Aydamir Mirzayev, Hamdi Di̇bekli̇oğlu

Turkish Journal of Electrical Engineering and Computer Sciences

Recent advances in deep learning, increased availability of large-scale datasets, and improvement of accelerated graphics processing units facilitated creation of an unprecedented amount of synthetically generated media content with impressive visual quality. Although such technology is used predominantly for entertainment, there is widespread practice of using deepfake technology for malevolent ends. This potential for malicious use necessitates the creation of detection methods capable of reliably distinguishing manipulated video content. In this work we aim to create a learning-based detection method for synthetically generated videos. To this end, we attempt to detect spatiotemporal inconsistencies by leveraging a learning-based magnification-inspired feature manipulation …

Automated Identification Of Vehicles In Very High-Resolution Uav Orthomosaics Using Yolov7 Deep Learning Model, Esra Yildirim, Umut Güneş Seferci̇k, Taşkın Kavzoğlu

Turkish Journal of Electrical Engineering and Computer Sciences

The utilization of remote sensing products for vehicle detection through deep learning has gained immense popularity, especially due to the advancement of unmanned aerial vehicles (UAVs). UAVs offer millimeter-level spatial resolution at low flight altitudes, which surpasses traditional airborne platforms. Detecting vehicles from very high-resolution UAV data is crucial in numerous applications, including parking lot and highway management, traffic monitoring, search and rescue missions, and military operations. Obtaining UAV data at desired periods allows the detection and tracking of target objects even several times during a day. Despite challenges such as diverse vehicle characteristics, traffic congestion, and hardware limitations, the …

Blood Cell Image Segmentation And Classification: A Systematic Review, Muhammad Shahzad, Farman Ali, Syed Hamad Shirazi, Assad Rasheed, Awais Ahmad, Babar Shah, Daehan Kwak

All Works

Background Blood diseases such as leukemia, anemia, lymphoma, and thalassemia are hematological disorders that relate to abnormalities in the morphology and concentration of blood elements, specifically white blood cells (WBC) and red blood cells (RBC). Accurate and efficient diagnosis of these conditions significantly depends on the expertise of hematologists and pathologists. To assist the pathologist in the diagnostic process, there has been growing interest in utilizing computer-aided diagnostic (CAD) techniques, particularly those using medical image processing and machine learning algorithms. Previous surveys in this domain have been narrowly focused, often only addressing specific areas like segmentation or classification but lacking …

Dataset Of Arabic Spam And Ham Tweets, Sanaa Kaddoura, Safaa Henno

All Works

This data article provides a dataset of 132421 posts and their corresponding information collected from Twitter social media. The data has two classes, ham or spam, where ham indicates non-spam clean tweets. The main target of this dataset is to study a way to classify whether a post is a spam or not automatically. The data is in Arabic language only, which makes the data essential to the researchers in Arabic natural language processing (NLP) due to the lack of resources in this language. The data is made publicly available to allow researchers to use it as a benchmark for …

Conic Challenge: Pushing The Frontiers Of Nuclear Detection, Segmentation, Classification And Counting, Simon Graham, Quoc Dang Vu, Mostafa Jahanifar, Martin Weigert, Uwe Schmidt, Wenhua Zhang, Jun Zhang, Sen Yang, Jinxi Xiang, Xiyue Wang, Josef Lorenz Rumberger, Elias Baumann, Peter Hirsch, Lihao Liu, Chenyang Hong, Angelica I. Aviles-Rivero, Ayushi Jain, Heeyoung Ahn, Yiyu Hong, Hussam Azzuni, Min Xu

Computer Vision Faculty Publications

Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and …

Catnet: Cross-Modal Fusion For Audio-Visual Speech Recognition, Xingmei Wang, Jianchen Mi, Boquan Li, Yixu Zhao, Jiaxiang Meng

Research Collection School Of Computing and Information Systems

Automatic speech recognition (ASR) is a typical pattern recognition technology that converts human speeches into texts. With the aid of advanced deep learning models, the performance of speech recognition is significantly improved. Especially, the emerging Audio–Visual Speech Recognition (AVSR) methods achieve satisfactory performance by combining audio-modal and visual-modal information. However, various complex environments, especially noises, limit the effectiveness of existing methods. In response to the noisy problem, in this paper, we propose a novel cross-modal audio–visual speech recognition model, named CATNet. First, we devise a cross-modal bidirectional fusion model to analyze the close relationship between audio and visual modalities. Second, …

Action Recognition Model Of Directed Attention Based On Cosine Similarity, Chen Li, Ming He, Chen Dong, Wei Li

Journal of System Simulation

Abstract: Aiming at the lack of directionality of traditional dot product attention, this paper proposes a directed attention model (DAM) based on cosine similarity. To effectively represent the direction relationship between the spatial and temporal features of video frames, the paper defines the relationship function in the attention mechanism using the cosine similarity theory, which can remove the absolute value of the relationship between features. To reduce the computational burden of the attention mechanism, the operation is decomposed from two dimensions of time and space. The computational complexity is further optimized by combining linear attention operation. The experiment is divided …

Open System Neural Networks, Bradley Hatch

Theses and Dissertations

Recent advances in self-supervised learning have made it possible to reuse information-rich models that have been generally pre-trained on massive amounts of data for other downstream tasks. But the pre-training process can be drastically different from the fine-tuning training process, which can lead to inefficient learning. We address this disconnect in training dynamics by structuring the learning process like an open system in thermodynamics. Open systems can achieve a steady state when low-entropy inputs are converted to high-entropy outputs. We modify the the model and the learning process to mimic this behavior, and attend more to elements of the input …

Road Extraction On Remote Sensing Imagery: Historical Mapping Of The Brazilian Amazon, Jonas Paiva Botelho Jr

MSU Graduate Theses

This work proposes an artificial intelligence model based on U-Net architecture to map road networks in the Brazilian Amazon. Over the years, the Amazon region has been heavily exploited, leading to increased deforestation rates, contributing to CO2 emissions, amplifying global warming, and causing a disturbance in local fauna and flora. The expansion into the forest by illegal miners, loggers, and land grabbers can be tracked down by the construction of roads, which we can refer to as the arteries of deforestation. Previous works on the matter proposed algorithms that use high-resolution imagery to map roads precisely. However, this work approach …

An Enhanced Deep Autoencoder For Flight Delay Prediction, Desmond B. Bisandu Phd, Dan Andrei Soviani-Sitoiu Msc, Irene Moulitsas Phd

Journal of Aviation/Aerospace Education & Research

Accurate and timely flight delay prediction cannot be overemphasized because of the ever-increasing demand for air travel and its importance in deploying intelligent transportation systems. Nonetheless, there has not been a universal solution to the problem, as more intelligent flight decision systems are required for the aviation industry's future growth. Existing flight delay classification and prediction approaches are mainly shallow traffic models and do not satisfy many applications in the real world. Our motivation to rethink the deep architecture model for predicting flight delays emanates from the problem. In this research, we proposed a technique that modified stacked autoencoder architecture …

A Memory Efficient Deep Recurrent Q-Learning Approach For Autonomous Wildfire Surveillance, Jeremy A. Cantor

UNF Graduate Theses and Dissertations

Previous literature demonstrates that autonomous UAVs (unmanned aerial vehicles) have the po- tential to be utilized for wildfire surveillance. This advanced technology empowers firefighters by providing them with critical information, thereby facilitating more informed decision-making processes. This thesis applies deep Q-learning techniques to the problem of control policy design under the objective that the UAVs collectively identify the maximum number of locations that are under fire, assuming the UAVs can share their observations. The prohibitively large state space underlying the control policy motivates a neural network approximation, but prior work used only convolutional layers to extract spatial fire information from …

Machine Learning As A Tool For Early Detection: A Focus On Late-Stage Colorectal Cancer Across Socioeconomic Spectrums, Hadiza Galadima, Rexford Anson-Dwamena, Ashley Johnson, Ghalib Bello, Georges Adunlin, James Blando

Community & Environmental Health Faculty Publications

Purpose: To assess the efficacy of various machine learning (ML) algorithms in predicting late-stage colorectal cancer (CRC) diagnoses against the backdrop of socio-economic and regional healthcare disparities. Methods: An innovative theoretical framework was developed to integrate individual- and census tract-level social determinants of health (SDOH) with sociodemographic factors. A comparative analysis of the ML models was conducted using key performance metrics such as AUC-ROC to evaluate their predictive accuracy. Spatio-temporal analysis was used to identify disparities in late-stage CRC diagnosis probabilities. Results: Gradient boosting emerged as the superior model, with the top predictors for late-stage CRC diagnosis being anatomic site, …

Nonuniform Sampling-Based Breast Cancer Classification, Santiago Posso

Theses and Dissertations--Electrical and Computer Engineering

The emergence of deep learning models and their success in visual object recognition have fueled the medical imaging community's interest in integrating these algorithms to improve medical diagnosis. However, natural images, which have been the main focus of deep learning models and mammograms, exhibit fundamental differences. First, breast tissue abnormalities are often smaller than salient objects in natural images. Second, breast images have significantly higher resolutions but are generally heavily downsampled to fit these images to deep learning models. Models that handle high-resolution mammograms require many exams and complex architectures. Additionally, spatially resizing mammograms leads to losing discriminative details essential …

Advancing Deep Learning With Graph-Based Structural Insights: From Graph Classification To Semantic Segmentation, Xin Ma

Computer Science and Engineering Dissertations

Deep learning has profoundly transformed machine learning by offering sophisticated data representations, yet effectively incorporating structural information remains a challenge. Structural data, whether explicit or implicit, has the potential to significantly enhance the performance of deep learning tasks. This research investigates the benefits of structural information across three crucial tasks: classification, clustering, and segmentation. For explicit structural data, where inputs are directly represented as graphs, we investigate graph-level classification in brain connectivity networks. We introduce the Multi-resolution Edge Network (MENET), a novel framework designed to identify disease-specific connectomic benchmarks with high discriminatory power across diagnostic categories. MENET leverages graph-level representations …