Physical Sciences and Mathematics Commons^™

Multimodal Sensing And Data Processing For Speaker And Emotion Recognition Using Deep Learning Models With Audio, Video And Biomedical Sensors, Farnaz Abtahi

Dissertations, Theses, and Capstone Projects

The focus of the thesis is on Deep Learning methods and their applications on multimodal data, with a potential to explore the associations between modalities and replace missing and corrupt ones if necessary. We have chosen two important real-world applications that need to deal with multimodal data: 1) Speaker recognition and identification; 2) Facial expression recognition and emotion detection.

The first part of our work assesses the effectiveness of speech-related sensory data modalities and their combinations in speaker recognition using deep learning models. First, the role of electromyography (EMG) is highlighted as a unique biometric sensor in improving audio-visual speaker …

Risk-Sensitive Stochastic Orienteering Problems For Trip Optimization In Urban Environments, Pradeep Varakantham, Akshat Kumar, Hoong Chuin Lau, William Yeoh

Research Collection School Of Computing and Information Systems

Orienteering Problems (OPs) are used to model many routing and trip planning problems. OPs are a variantof the well-known traveling salesman problem where the goal is to compute the highest reward path thatincludes a subset of vertices and has an overall travel time less than a specified deadline. However, the applicabilityof OPs is limited due to the assumption of deterministic and static travel times. To that end, Campbellet al. extended OPs to Stochastic OPs (SOPs) to represent uncertain travel times (Campbell et al. 2011). Inthis article, we make the following key contributions: (1) We extend SOPs to Dynamic SOPs (DSOPs), …

On The Use Of Semantic-Based Aig To Automatically Generate Programming Exercises, Laura Zavala, Benito Mendoza

Publications and Research

In introductory programming courses, proficiency is typically achieved through substantial practice in the form of relatively small assignments and quizzes. Unfortunately, creating programming assignments and quizzes is both, time-consuming and error-prone. We use Automatic Item Generation (AIG) in order to address the problem of creating numerous programming exercises that can be used for assignments or quizzes in introductory programming courses. AIG is based on the use of test-item templates with embedded variables and formulas which are resolved by a computer program with actual values to generate test-items. Thus, hundreds or even thousands of test-items can be generated with a single …

Gradient Estimation For Attractor Networks, Thomas Flynn

Dissertations, Theses, and Capstone Projects

It has been hypothesized that neural network models with cyclic connectivity may be more powerful than their feed-forward counterparts. This thesis investigates this hypothesis in several ways. We study the gradient estimation and optimization procedures for several variants of these networks. We show how the convergence of the gradient estimation procedures are related to the properties of the networks. Then we consider how to tune the relative rates of gradient estimation and parameter adaptation to ensure successful optimization in these models. We also derive new gradient estimators for stochastic models. First, we port the forward sensitivity analysis method to the …

Sosiel: A Cognitive, Multi-Agent, And Knowledge-Based Platform For Modeling Boundedly-Rational Decision-Making, Garry Sotnik

Dissertations and Theses

Decision-related activities, such as bottom-up and top-down policy development, analysis, and planning, stand to benefit from the development and application of computer-based models that are capable of representing spatiotemporal social human behavior in local contexts. This is especially the case with our efforts to understand and search for ways to mitigate the context-specific effects of climate change, in which case such models need to include interacting social and ecological components. The development and application of such models has been significantly hindered by the challenges in designing artificial agents whose behavior is grounded in both empirical evidence and theory and in …

Feature Based Calibration Of A Network Of Kinect Sensors, Xiaoyang Li

Electronic Thesis and Dissertation Repository

The availability of affordable depth sensors in conjunction with common RGB cameras, such as the Microsoft Kinect, can provide robots with a complete and instantaneous representation of the current surrounding environment. However, in the problem of calibrating multiple camera systems, traditional methods bear some drawbacks, such as requiring human intervention. In this thesis, we propose an automatic and reliable calibration framework that can easily estimate the extrinsic parameters of a Kinect sensor network. Our framework includes feature extraction, Random Sample Consensus and camera pose estimation from high accuracy correspondences. We also implement a robustness analysis of position estimation algorithms. The …

Resistance Is Futile: Embracing The Era Of The Augmented Worker, Nathaniel Barr, Kelly Peters

Publications and Scholarship

The prospect of A.I.-augmented workers is both promising and unsettling: How can employees and firms ensure that they get the benefits of A.I. without erasing uniquely human strengths?

Ai For Ground Robots For Autonomous Coverage Of Designated Areas, Danxue Huang

Summer Community of Scholars Posters (RCEU and HCR Combined Programs)

No abstract provided.

Exploring Natural User Abstractions For Shared Perceptual Manipulator Task Modeling & Recovery, Senglee Koh

Electronic Theses and Dissertations

State-of-the-art domestic robot assistants are essentially autonomous mobile manipulators capable of exerting human-scale precision grasps. To maximize utility and economy, non-technical end-users would need to be nearly as efficient as trained roboticists in control and collaboration of manipulation task behaviors. However, it remains a significant challenge given that many WIMP-style tools require superficial proficiency in robotics, 3D graphics, and computer science for rapid task modeling and recovery. But research on robot-centric collaboration has garnered momentum in recent years; robots are now planning in partially observable environments that maintain geometries and semantic maps, presenting opportunities for non-experts to cooperatively control task …

Elephant 2000: A Programming Language For Remembering The Past And Building On It, Kerry J. Holmes

Williams Honors College, Honors Research Projects

Elephant 2000 is a programming language to specify programs that accept user speech as text inputs and outputs speech text. The inputs and outputs are based on Dialogue Act theory which describes several forms of speech outputs, such as requests, questions, and answers. The language also relies on Named Entity Recognition to determine what types of objects a user references. These entities include persons, locations, times and so on. Using these attributes of user speech, a program is able to perform simple rule matching and pattern recognition to respond to input. The result is a programming language with English like …

Deep Learning Of 2-D Images Representing N-D Data In General Line Coordinates, Dmytro Dovhalets, Boris Kovalerchuk, Szilárd Vajda, Răzvan Andonie

Computer Science Faculty Scholarship

While knowledge discovery and n-D data visualization procedures are often efficient, the loss of information, occlusion, and clutter continue to be a challenge. General Line Coordinates (GLC) is a rather new technique to deal with such artifacts. GLC-Linear, which is one of the methods in GLC, allows transforming n-D numerical data to their visual representation as polylines losslessly. The method proposed in this paper uses these 2-D visual representations as input to Convolutional Neural Network (CNN) classifiers. The obtained classification accuracies are close to the ones obtained by other machine learning algorithms. The main benefit of the method is the …

Magic Triangle – Human, Exoskeleton, And Collaborative Robot Scenario, R. A. Goehlich, M. H. Rutsch, I. Krohne

Publications

The incidence of musculoskeletal disorders in workplaces with difficult ergonomic conditions is increasing. Today, there is a growing market for technical support systems that avoid repetitive strain on the musculoskeletal system. We have been observing two (parallel) lines of development: on the one hand, the development of exoskeletons supporting shop floor operators and, on the other hand, the development of collaborative robots for the creation of hybrid teams. The focus of our research is the combined application of exoskeletons AND collaborative robots for shop floor operators in the aerospace industry. Our approach is to analyze various scenarios to understand which …

Ai Education Matters: Teaching Hidden Markov Models, Todd W. Neller

Computer Science Faculty Publications

In this column, we share resources for learning about and teaching Hidden Markov Models (HMMs). HMMs find many important applications in temporal pattern recognition tasks such as speech/handwriting/gesture recognition and robot localization. In such domains, we may have a finite state machine model with known state transition probabilities, state output probabilities, and state outputs, but lack knowledge of the states generating such outputs. HMMs are useful in framing problems where external sequential evidence is used to derive underlying state information (e.g. intended words and gestures). [excerpt]

Don't Take This Personally: Sentiment Analysis For Identification Of "Subtweeting" On Twitter, Noah L. Segal-Gould

Senior Projects Spring 2018

The purpose of this project is to identify subtweets. The Oxford English Dictionary defines "subtweet" as a "[Twitter post] that refers to a particular user without directly mentioning them, typically as a form of furtive mockery or criticism." This paper details a process for gathering a labeled ground truth dataset, training a classifier, and creating a Twitter bot which interacts with subtweets in real time. The Naive Bayes classifier trained in this project classifies tweets as subtweets and non-subtweets with an average F1 score of 72%.

Predictive Analytics In The Criminal Justice System: Media Depictions And Framing, Kar Mun Cheng

Honors Program Theses

Artificial intelligence and algorithms are increasingly becoming commonplace in crime-fighting efforts. For instance, predictive policing uses software to predetermine criminals and areas where crime is most likely to happen. Risk assessment software are employed in sentence determination and other courtroom decisions, and they are also being applied towards prison overpopulation by assessing which inmates can be released. Public opinion on the use of predictive software is divided: many police and state officials support it, crediting it with lowering crime rates and improving public safety. Others, however, have questioned its effectiveness, citing civil liberties concerns as well as the possibility of …

Smu Master Of It In Business Launches New Artificial Intelligence Track, Singapore Management University

SMU Press Releases

The Singapore Management University’s School of Information Systems (SIS) has launched a new Artificial Intelligence (AI) track under its Master of IT in Business (MITB) programme. Geared towards nurturing graduates who are ready for the revolutionary change from AI in data science, the AI track equips a new generation of IT business leaders in careers that bridge AI with business.

Librarians' Perceptions Of Artificial Intelligence And Its Potential Impact On The Profession, Barbara A. Wood, David Evans

Faculty Articles

The subject of artificial intelligence (AI) is being discussed everywhere in the media. Stephen Hawking, Elon Musk, and Bill Gates regularly sound the alarm about AI as an existential threat to humankind. Open a newspaper, turn on the television, or log on to the internet, and you will find a plethora of information and opinions on AI and its potential impact on human endeavors. In addition to being a hot topic in the media, the scholarly literature in medicine and law is replete with AI research. It acknowledges AI as a transformative, if not disruptive, game changer. AI is being …

Deep Neural Networks For Multi-Label Text Classification: Application To Coding Electronic Medical Records, Anthony Rios

Theses and Dissertations--Computer Science

Coding Electronic Medical Records (EMRs) with diagnosis and procedure codes is an essential task for billing, secondary data analyses, and monitoring health trends. Both speed and accuracy of coding are critical. While coding errors could lead to more patient-side financial burden and misinterpretation of a patient’s well-being, timely coding is also needed to avoid backlogs and additional costs for the healthcare facility. Therefore, it is necessary to develop automated diagnosis and procedure code recommendation methods that can be used by professional medical coders.

The main difficulty with developing automated EMR coding methods is the nature of the label space. The …

Review Of Deep Learning Methods In Robotic Grasp Detection, Shehan Caldera, Alexander Rassau, Douglas Chai

Research outputs 2014 to 2021

For robots to attain more general-purpose utility, grasping is a necessary skill to master. Such general-purpose robots may use their perception abilities to visually identify grasps for a given object. A grasp describes how a robotic end-effector can be arranged to securely grab an object and successfully lift it without slippage. Traditionally, grasp detection requires expert human knowledge to analytically form the task-specific algorithm, but this is an arduous and time-consuming approach. During the last five years, deep learning methods have enabled significant advancements in robotic vision, natural language processing, and automated driving applications. The successful results of these methods …

Towards Dynamic Interaction-Based Reputation Models, Almas Melnikov, Manuel Mazzara, Victor Rivera, Jooyoung Lee, Luca Longo

Articles

In this paper, we investigate how dynamic properties of reputation can influence the quality of users’ ranking. Reputation systems should be based on rules that can guarantee high level of trust and help identify unreliable units. To understand the effectiveness of dynamic properties in the evaluation of reputation, we propose our own model (DIB-RM) that utilizes three factors: forgetting, cumulative, and activity period. In order to evaluate the model, we use data from StackOverflow which also has its own reputation model. We estimate similarity of ratings between DIB-RM and the StackOverflow reputation model to test our hypothesis. We use two …

Resource Optimization In Wireless Sensor Networks For An Improved Field Coverage And Cooperative Target Tracking, Husam Sweidan

Dissertations, Master's Theses and Master's Reports

There are various challenges that face a wireless sensor network (WSN) that mainly originate from the limited resources a sensor node usually has. A sensor node often relies on a battery as a power supply which, due to its limited capacity, tends to shorten the life-time of the node and the network as a whole. Other challenges arise from the limited capabilities of the sensors/actuators a node is equipped with, leading to complication like a poor coverage of the event, or limited mobility in the environment. This dissertation deals with the coverage problem as well as the limited power and …

The Stance On Artificial Intelligence, Edward Gonzalez-Olmedo

Nebraska College Preparatory Academy: Senior Capstone Projects

Mary Shelley’s novel Frankenstein brings up the question: When do we draw the line on technological Artificial Intelligence (AI) advancement? The importance of this is that we need to be able to control the technology that we produce. Humans always strive for an advancement in technology, and AI is one of the more advanced concepts. Mary Shelley introduces this problem with Frankenstein when he creates a creature which cannot be controlled by its creator. Humans have begun to tinker with this new technology, yet mankind does not fully understand AI.

● Human desire is inevitable. ● The creation of AI …

Sports Analytics With Computer Vision, Colby T. Jeffries

Senior Independent Study Theses

Computer vision in sports analytics is a relatively new development. With multi-million dollar systems like STATS’s SportVu, professional basketball teams are able to collect extremely fine-detailed data better than ever before. This concept can be scaled down to provide similar statistics collection to college and high school basketball teams. Here we investigate the creation of such a system using open-source technologies and less expensive hardware. In addition, using a similar technology, we examine basketball free throws to see whether a shooter’s form has a specific relationship to a shot’s outcome. A system that learns this relationship could be used to …

A Recurrent Neural Network Architecture For Biomedical Event Trigger Classification, Jeevith Bopaiah

Theses and Dissertations--Computer Science

A “biomedical event” is a broad term used to describe the roles and interactions between entities (such as proteins, genes and cells) in a biological system. The task of biomedical event extraction aims at identifying and extracting these events from unstructured texts. An important component in the early stage of the task is biomedical trigger classification which involves identifying and classifying words/phrases that indicate an event. In this thesis, we present our work on biomedical trigger classification developed using the multi-level event extraction dataset. We restrict the scope of our classification to 19 biomedical event types grouped under four broad …

New Trends In Second Language Learning And Teaching Through The Lens Of Ict, Networked Learning, And Artificial Intelligence, Jaya Kannan, Pilar Munday

Languages Faculty Publications

In the last few decades, Information and Communications Technology (ICT) applications have been shaping the field of Computer Assisted Language Learning (CALL). Mobile Assisted Language Learning (MALL) paved the way for ubiquitous learning. The advent of new technologies in the early 21st century also added a social dimension to ICT that allowed for Networked Learning (NL). Given that language learning is fundamentally a socio-cultural experience, networked learning capabilities have provided the potential for language learning in community settings. This has revitalized the earlier frameworks provided by CALL. NL has empowered language learners today to connect globally, to access Open Educational …

Neutrosophic Operational Research - Vol. 3., Florentin Smarandache, Mohamed Abdel Basset, Victor Chang

Branch Mathematics and Statistics Faculty and Staff Publications

Foreword John R. Edwards This book is an excellent exposition of the use of Data Envelopment Analysis (DEA) to generate data analytic insights to make evidence-based decisions, to improve productivity, and to manage cost-risk and benefitopportunity in public and private sectors. The design and the content of the book make it an up-to-date and timely reference for professionals, academics, students, and employees, in particular those involved in strategic and operational decisionmaking processes to evaluate and prioritize alternatives to boost productivity growth, to optimize the efficiency of resource utilization, and to maximize the effectiveness of outputs and impacts to stakeholders. It …

Exploring The Functional And Geometric Bias Of Spatial Relations Using Neural Language Models, Simon Dobnik, Mehdi Ghanimifard, John D. Kelleher

Conference papers

The challenge for computational models of spatial descriptions for situated dialogue systems is the integration of information from different modalities. The semantics of spatial descriptions are grounded in at least two sources of information: (i) a geometric representation of space and (ii) the functional interaction of related objects that. We train several neural language models on descriptions of scenes from a dataset of image captions and examine whether the functional or geometric bias of spatial descriptions reported in the literature is reflected in the estimated perplexity of these models. The results of these experiments have implications for the creation of …

Leveraging Overhead Imagery For Localization, Mapping, And Understanding, Scott Workman

Theses and Dissertations--Computer Science

Ground-level and overhead images provide complementary viewpoints of the world. This thesis proposes methods which leverage dense overhead imagery, in addition to sparsely distributed ground-level imagery, to advance traditional computer vision problems, such as ground-level image localization and fine-grained urban mapping. Our work focuses on three primary research areas: learning a joint feature representation between ground-level and overhead imagery to enable direct comparison for the task of image geolocalization, incorporating unlabeled overhead images by inferring labels from nearby ground-level images to improve image-driven mapping, and fusing ground-level imagery with overhead imagery to enhance understanding. The ultimate contribution of this thesis …

Minimally Actuated Walking: Identifying Core Challenges To Economical Legged Locomotion Reveals Novel Solutions, Ryan T. Schroeder, John Ea Bertram

Research outputs 2014 to 2021

Terrestrial organisms adept at locomotion employ strut-like legs for economical and robust movement across the substrate. Although it is relatively easy to observe and analyze details of the solutions these organic systems have arrived at, it is not as easy to identify the problems these movement strategies have solved. As such, it is useful to investigate fundamental challenges that effective legged locomotion overcomes in order to understand why the mechanisms employed by biological systems provide viable solutions to these challenges. Such insight can inform the design and development of legged robots that may eventually match or exceed animal performance. In …

Crop Height Estimation With Unmanned Aerial Vehicles, Carrick Detweiler, David Anthony, Sebastian Elbaum

School of Computing: Faculty Publications

An unmanned aerial vehicle (UAV) can be configured for crop height estimation. In some examples, the UAV includes an aerial propulsion system, a laser scanner configured to face downwards while the UAV is in flight, and a control system. The laser scanner is configured to scan through a two-dimensional scan angle and is characterized by a maxi mum range. The control system causes the UAV to fly over an agricultural field and maintain, using the aerial propulsion system and the laser scanner, a distance between the UAV and a top of crops in the agricultural field to within a programmed …