Physical Sciences and Mathematics Commons^™

From Text Classification To Image Clustering, Problems Less Optimized, Amirhossein Herandi

Computer Science and Engineering Theses

Machine Learning is thriving. Every industry is using its techniques in some way to improve their efficiency and revenue. However, the focus on research is not divided equally between all of the different areas and problems that this field can tackle and analyze. Currently, Computer Vision is the one area that is being focused very extensively by researchers and companies alike, and as a result has seen an amazing boost in the recent years. This ranges from the well-known problems of classification that use discriminative models all the way to more novel problems that use generative models such as style …

Design And Implementation Of A Domain Specific Language For Deep Learning, Xiao Bing Huang

Theses and Dissertations

\textit {Deep Learning} (DL) has found great success in well-diversified areas such as machine vision, speech recognition, big data analysis, and multimedia understanding recently. However, the existing state-of-the-art DL frameworks, e.g. Caffe2, Theano, TensorFlow, MxNet, Torch7, and CNTK, are programming libraries with fixed user interfaces, internal representations, and execution environments. Modifying the code of DL layers or data structure is very challenging without in-depth understanding of the underlying implementation. The optimization of the code and execution in these tools is often limited and relies on the specific DL computation graph manipulation and scheduling that lack systematic and universal strategies. Furthermore, …

Parameterizing And Aggregating Activation Functions In Deep Neural Networks, Luke Benjamin Godfrey

Graduate Theses and Dissertations

The nonlinear activation functions applied by each neuron in a neural network are essential for making neural networks powerful representational models. If these are omitted, even deep neural networks reduce to simple linear regression due to the fact that a linear combination of linear combinations is still a linear combination. In much of the existing literature on neural networks, just one or two activation functions are selected for the entire network, even though the use of heterogenous activation functions has been shown to produce superior results in some cases. Even less often employed are activation functions that can adapt their …

Comparative Study Of Deep Learning Models For Network Intrusion Detection, Brian Lee, Sandhya Amaresh, Clifford Green, Daniel Engels

SMU Data Science Review

In this paper, we present a comparative evaluation of deep learning approaches to network intrusion detection. A Network Intrusion Detection System (NIDS) is a critical component of every Internet connected system due to likely attacks from both external and internal sources. A NIDS is used to detect network born attacks such as Denial of Service (DoS) attacks, malware replication, and intruders that are operating within the system. Multiple deep learning approaches have been proposed for intrusion detection systems. We evaluate three models, a vanilla deep neural net (DNN), self-taught learning (STL) approach, and Recurrent Neural Network (RNN) based Long Short …

A Multi-Step Nonlinear Dimension-Reduction Approach With Applications To Bigdata, R. Krishnan, V. A. Samaranayake, Jagannathan Sarangapani

Mathematics and Statistics Faculty Research & Creative Works

In this paper, a multi-step dimension-reduction approach is proposed for addressing nonlinear relationships within attributes. In this work, the attributes in the data are first organized into groups. In each group, the dimensions are reduced via a parametric mapping that takes into account nonlinear relationships. Mapping parameters are estimated using a low rank singular value decomposition (SVD) of distance covariance. Subsequently, the attributes are reorganized into groups based on the magnitude of their respective singular values. The group-wise organization and the subsequent reduction process is performed for multiple steps until a singular value-based user-defined criterion is satisfied. Simulation analysis is …

Speech Emotion Recognition Using Convolutional Neural Networks, Somayeh Shahsavarani

Department of Computer Science and Engineering: Dissertations, Theses, and Student Research

Automatic speech recognition is an active field of study in artificial intelligence and machine learning whose aim is to generate machines that communicate with people via speech. Speech is an information-rich signal that contains paralinguistic information as well as linguistic information. Emotion is one key instance of paralinguistic information that is, in part, conveyed by speech. Developing machines that understand paralinguistic information, such as emotion, facilitates the human-machine communication as it makes the communication more clear and natural. In the current study, the efficacy of convolutional neural networks in recognition of speech emotions has been investigated. Wide-band spectrograms of the …

Attributed Social Network Embedding, Lizi Liao, Xiangnan He, Hanwang Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Embedding network data into a low-dimensional vector space has shown promising performance for many real-world applications, such as node classification and entity retrieval. However, most existing methods focused only on leveraging network structure. For social networks, besides the network structure, there also exists rich information about social actors, such as user profiles of friendship networks and textual content of citation networks. These rich attribute information of social actors reveal the homophily effect, exerting huge impacts on the formation of social networks. In this paper, we explore the rich evidence source of attributes in social networks to improve network embedding. We …

Patent Keyword Extraction Algorithm Based On Distributed Representation For Patent Classification, Jie Hu, Shaobo Li, Yong Yao, Liya Yu, Guanci Yang, Jianjun Hu

Faculty Publications

Many text mining tasks such as text retrieval, text summarization, and text comparisons depend on the extraction of representative keywords from the main text. Most existing keyword extraction algorithms are based on discrete bag-of-words type of word representation of the text. In this paper, we propose a patent keyword extraction algorithm (PKEA) based on the distributed Skip-gram model for patent classification. We also develop a set of quantitative performance measures for keyword extraction evaluation based on information gain and cross-validation, based on Support Vector Machine (SVM) classification, which are valuable when human-annotated keywords are not available. We used a standard …

Object Localization, Segmentation, And Classification In 3d Images, Allan Zelener

Dissertations, Theses, and Capstone Projects

We address the problem of identifying objects of interest in 3D images as a set of related tasks involving localization of objects within a scene, segmentation of observed object instances from other scene elements, classifying detected objects into semantic categories, and estimating the 3D pose of detected objects within the scene. The increasing availability of 3D sensors motivates us to leverage large amounts of 3D data to train machine learning models to address these tasks in 3D images. Leveraging recent advances in deep learning has allowed us to develop models capable of addressing these tasks and optimizing these tasks jointly …

Representation Learning With Convolutional Neural Networks, Haotian Xu

Wayne State University Dissertations

Deep learning methods have achieved great success in the areas of Computer Vision and Natural Language Processing. Recently, the rapidly developing field of deep learning is concerned with questions surrounding how we can learn meaningful and effective representations of data. This is because the performance of machine learning approaches is heavily dependent on the choice and quality of data representation, and different kinds of representation entangle and hide the different explanatory factors of variation behind the data.

In this dissertation, we focus on representation learning with deep neural networks for different data formats including text, 3D polygon shapes, and brain …

Continuous Restricted Boltzmann Machines, Robert W. Harrison

EBCS Articles

Restricted Boltzmann machines are a generative neural network. They summarize their input data to build a probabilistic model that can then be used to reconstruct missing data or to classify new data. Unlike discrete Boltzmann machines, where the data are mapped to the space of integers or bitstrings, continuous Boltzmann machines directly use floating point numbers and therefore represent the data with higher fidelity. The primary limitation in using Boltzmann machines for big-data problems is the efficiency of the training algorithm. This paper describes an efficient deterministic algorithm for training continuous machines.

Existing And Potential Statistical And Computational Approaches For The Analysis Of 3d Ct Images Of Plant Roots, Zheng Xu, Camilo Valdes, Jennifer Clarke

Department of Statistics: Faculty Publications

Scanning technologies based on X-ray Computed Tomography (CT) have been widely used in many scientific fields including medicine, nanosciences and materials research. Considerable progress in recent years has been made in agronomic and plant science research thanks to X-ray CT technology. X-ray CT image-based phenotyping methods enable high-throughput and non-destructive measuring and inference of root systems, which makes downstream studies of complex mechanisms of plants during growth feasible. An impressive amount of plant CT scanning data has been collected, but how to analyze these data efficiently and accurately remains a challenge. We review statistical and computational approaches that have been …

Review Of Deep Learning Methods In Robotic Grasp Detection, Shehan Caldera, Alexander Rassau, Douglas Chai

Research outputs 2014 to 2021

For robots to attain more general-purpose utility, grasping is a necessary skill to master. Such general-purpose robots may use their perception abilities to visually identify grasps for a given object. A grasp describes how a robotic end-effector can be arranged to securely grab an object and successfully lift it without slippage. Traditionally, grasp detection requires expert human knowledge to analytically form the task-specific algorithm, but this is an arduous and time-consuming approach. During the last five years, deep learning methods have enabled significant advancements in robotic vision, natural language processing, and automated driving applications. The successful results of these methods …

A Comparison Of Classical Versus Deep Learning Techniques For Abusive Content Detection On Social Media Sites, Hao Che, Susan Mckeever, Sarah Jane Delany

Conference papers

The automated detection of abusive content on social media websites faces a variety of challenges including imbalanced training sets, the identification of an appropriate feature representation and the selection of optimal classifiers. Classifiers such as support vector machines (SVM), combined with bag of words or ngram feature representation, have traditionally dominated in text classification for decades. With the recent emergence of deep learning and word embeddings, an increasing number of researchers have started to focus on deep neural networks. In this paper, our aim is to explore cutting-edge techniques in automated abusive content detection. We use two deep learning approaches: …

Deep Learning Methods For Visual Object Recognition, Zeyad Hailat

Wayne State University Dissertations

Convolutional neural networks (CNNs) attain state-of-the-art performance on various classification tasks assuming a sufficiently large number of labeled training examples. Unfortunately, curating sufficiently large labeled training dataset requires human involvement, which is expensive, time-consuming, and susceptible to noisy labels. Semi-supervised learning methods can alleviate the aforementioned problems by employing one of two techniques. First, utilizing a limited number of labeled data in conjunction with sufficiently large unlabeled data to construct a classification model. Second, exploiting sufficiently large noisy label training data to learn a classification model. In this dissertation, we proposed a few new methods to mitigate the aforementioned problems. …

Automated Tree-Level Forest Quantification Using Airborne Lidar, Hamid Hamraz

Theses and Dissertations--Computer Science

Traditional forest management relies on a small field sample and interpretation of aerial photography that not only are costly to execute but also yield inaccurate estimates of the entire forest in question. Airborne light detection and ranging (LiDAR) is a remote sensing technology that records point clouds representing the 3D structure of a forest canopy and the terrain underneath. We present a method for segmenting individual trees from the LiDAR point clouds without making prior assumptions about tree crown shapes and sizes. We then present a method that vertically stratifies the point cloud to an overstory and multiple understory tree …

Quantitative Behavior Tracking Of Xenopus Laevis Tadpoles For Neurobiology Research, Alexander Hansen Hamme

Senior Projects Fall 2018

Xenopus laevis tadpoles are a useful animal model for neurobiology research because they provide a means to study the development of the brain in a species that is both physiologically well-understood and logistically easy to maintain in the laboratory. For behavioral studies, however, their individual and social swimming patterns represent a largely untapped trove of data, due to the lack of a computational tool that can accurately track multiple tadpoles at once in video feeds. This paper presents a system that was developed to accomplish this task, which can reliably track up to six tadpoles in a controlled environment, thereby …

Recurrent Neural Networks And Their Applications To Rna Secondary Structure Inference, Devin Willmott

Theses and Dissertations--Mathematics

Recurrent neural networks (RNNs) are state of the art sequential machine learning tools, but have difficulty learning sequences with long-range dependencies due to the exponential growth or decay of gradients backpropagated through the RNN. Some methods overcome this problem by modifying the standard RNN architecure to force the recurrent weight matrix W to remain orthogonal throughout training. The first half of this thesis presents a novel orthogonal RNN architecture that enforces orthogonality of W by parametrizing with a skew-symmetric matrix via the Cayley transform. We present rules for backpropagation through the Cayley transform, show how to deal with the Cayley …

Estimating Meteorological Visibility Range Under Foggy Weather Conditions: A Deep Learning Approach, Hazar Chaabani, Naoufel Werghi, Faouzi Kamoun, Bilal Taha, Fatma Outay, Ansar Ul Haque Yasar

All Works

© 2018 The Authors. Published by Elsevier Ltd. Systems capable of estimating visibility distances under foggy weather conditions are extremely useful for next-generation cooperative situational awareness and collision avoidance systems. In this paper, we present a brief review of noticeable approaches for determining visibility distance under foggy weather conditions. We then propose a novel approach based on the combination of a deep learning method for feature extraction and an SVM classifier. We present a quantitative evaluation of the proposed solution and show that our approach provides better performance results compared to an earlier approach that was based on the combination …

Deep Convolutional Neural Networks Enable Discrimination Of Heterogeneous Digital Pathology Images, Pegah Khosravi, Ehsan Kazemi, Marcin Imielinski, Olivier Elemento, Iman Hajirasouliha

Publications and Research

Pathological evaluation of tumor tissue is pivotal for diagnosis in cancer patients and automated image analysis approaches have great potential to increase precision of diagnosis and help reduce human error.

In this study, we utilize several computational methods based on convolutional neural networks (CNN) and build a stand-alone pipeline to effectively classify different histopathology images across different types of cancer.

In particular, we demonstrate the utility of our pipeline to discriminate between two subtypes of lung cancer, four biomarkers of bladder cancer, and five biomarkers of breast cancer. In addition, we apply our pipeline to discriminate among four immunohistochemistry …

Deep Learning Based Brain Tumor Classification And Detection System, Ali̇ Ari, Davut Hanbay

Turkish Journal of Electrical Engineering and Computer Sciences

The brain cancer treatment process depends on the physician's experience and knowledge. For this reason, using an automated tumor detection system is extremely important to aid radiologists and physicians to detect brain tumors. The proposed method has three stages, which are preprocessing, the extreme learning machine local receptive fields (ELM-LRF) based tumor classification, and image processing based tumor region extraction. At first, nonlocal means and local smoothing methods were used to remove possible noises. In the second stage, cranial magnetic resonance (MR) images were classified as benign or malignant by using ELM-LRF. In the third stage, the tumors were segmented. …

Estimating Left Ventricular Volume With Roi-Based Convolutional Neural Network, Feng Zhu

Turkish Journal of Electrical Engineering and Computer Sciences

The volume of the human left ventricular (LV) chamber is an important indicator for diagnosing heart disease. Although LV volume can be measured manually with cardiac magnetic resonance imaging (MRI), the process is difficult and time-consuming for experienced cardiologists. This paper presents an end-to-end segmentation-free method that estimates LV volume from MRI images directly. The method initially uses Fourier transform and a regression filter to calculate the region of interest that contains the LV chambers. Then convolutional neural networks are trained to estimate the end-diastolic volume (EDV) and end-systolic volume (ESV). The resulting models accurately estimate the EDV and ESV …

Smart Augmented Reality Instructional System For Mechanical Assembly, Ze-Hao Lai

Masters Theses

"Quality and efficiency are pivotal indicators of a manufacturing company. Many companies are suffering from shortage of experienced workers across the production line to perform complex assembly tasks such as assembly of an aircraft engine. This could lead to a significant financial loss. In order to further reduce time and error in an assembly, a smart system consisting of multi-modal Augmented Reality (AR) instructions with the support of a deep learning network for tool detection is introduced. The multi-modal smart AR is designed to provide on-site information including various visual renderings with a fine-tuned Region-based Convolutional Neural Network, which is …

Deep Recurrent Learning For Efficient Image Recognition Using Small Data, Mahbubul Alam

Electrical & Computer Engineering Theses & Dissertations

Recognition is fundamental yet open and challenging problem in computer vision. Recognition involves the detection and interpretation of complex shapes of objects or persons from previous encounters or knowledge. Biological systems are considered as the most powerful, robust and generalized recognition models. The recent success of learning based mathematical models known as artificial neural networks, especially deep neural networks, have propelled researchers to utilize such architectures for developing bio-inspired computational recognition models. However, the computational complexity of these models increases proportionally to the challenges posed by the recognition problem, and more importantly, these models require a large amount of data …

Visual Odometry Using Convolutional Neural Networks, Alec Graves, Steffen Lim, Thomas Fagan, Kevin Mcfall Phd.

The Kennesaw Journal of Undergraduate Research

Visual odometry is the process of tracking an agent's motion over time using a visual sensor. The visual odometry problem has only been recently solved using traditional, non-machine learning techniques. Despite the success of neural networks at many related problems such as object recognition, feature detection, and optical flow, visual odometry still has not been solved with a deep learning technique. This paper attempts to implement several Convolutional Neural Networks to solve the visual odometry problem and compare slight variations in data preprocessing. The work presented is a step toward reaching a legitimate neural network solution.

How To Best Apply Neural Networks In Geosciences: Towards Optimal "Averaging" In Dropout Training, Afshin Gholamy, Justin Parra, Vladik Kreinovich, Olac Fuentes, Elizabeth Y. Anthony

Departmental Technical Reports (CS)

The main objectives of geosciences is to find the current state of the Earth -- i.e., solve the corresponding inverse problems -- and to use this knowledge for predicting the future events, such as earthquakes and volcanic eruptions. In both inverse and prediction problems, often, machine learning techniques are very efficient, and at present, the most efficient machine learning technique is deep neural training. To speed up this training, the current learning algorithms use dropout techniques: they train several sub-networks on different portions of data, and then "average" the results. A natural idea is to use arithmetic mean for this …

Online Learning With Nonlinear Models, Doyen Sahoo

Dissertations and Theses Collection (Open Access)

Recent years have witnessed the success of two broad categories of machine learning algorithms: (i) Online Learning; and (ii) Learning with nonlinear models. Typical machine learning algorithms assume that the entire data is available prior to the training task. This is often not the case in the real world, where data often arrives sequentially in a stream, or is too large to be stored in memory. To address these challenges, Online Learning techniques evolved as a promising solution to having highly scalable and efficient learning methodologies which could learn from data arriving sequentially. Next, as the real world data exhibited …

A Deep Learníng-Based Data Minimization Algorithm For Big Genomics Data In Support Of Lot And Secure Smart Health Services, Mohammed Aledhari

Dissertations

In the age of Big Genomics Data, institutes such as the National Human Genome Research Institute (NHGRI),1000-Genomes project, and the international cancer sequencing consortium are faced with the challenge of sharing large volumes of data between internationallydispersed sample collectors, data analyzers, and researchers, a process that up until now has been plagued by unreliable transfers and slow connection speeds. These occur due to the inherent throughput bottlenecks of traditional transfer technologies. One suggested solution is using the cloud as an infrastructure to solve the store and analysis challenges. However, the transfer and share of the genomics datasets between biological laboratories …

Intent Recognition In Smart Living Through Deep Recurrent Neural Networks, Xiang Zhang, Lina Yao, Chaoran Huang, Quan Z. Sheng, Xianzhi Wang

Research Collection School Of Computing and Information Systems

Electroencephalography (EEG) signal based intent recognition has recently attracted much attention in both academia and industries, due to helping the elderly or motor-disabled people controlling smart devices to communicate with outer world. However, the utilization of EEG signals is challenged by low accuracy, arduous and time-consuming feature extraction. This paper proposes a 7-layer deep learning model to classify raw EEG signals with the aim of recognizing subjects’ intents, to avoid the time consumed in pre-processing and feature extraction. The hyper-parameters are selected by an Orthogonal Array experiment method for efficiency. Our model is applied to an open EEG dataset provided …

Hierarchical Fusion Based Deep Learning Framework For Lung Nodule Classification, Kazim Sekeroglu

LSU Doctoral Dissertations

Lung cancer is the leading cancer type that causes the mortality in both men and women. Computer aided detection (CAD) and diagnosis systems can play a very important role for helping the physicians in cancer treatments. This dissertation proposes a CAD framework that utilizes a hierarchical fusion based deep learning model for detection of nodules from the stacks of 2D images. In the proposed hierarchical approach, a decision is made at each level individually employing the decisions from the previous level. Further, individual decisions are computed for several perspectives of a volume of interest (VOI). This study explores three different …