Physical Sciences and Mathematics Commons^™

Maximum Entropy Beyond Selecting Probability Distributions, Thach N. Nguyen, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Traditionally, the Maximum Entropy technique is used to select a probability distribution in situations when several different probability distributions are consistent with our knowledge. In this paper, we show that this technique can be extended beyond selecting probability distributions, to explain facts, numerical values, and even types of functional dependence.

Almost All Diophantine Sets Are Undecidable, Vladik Kreinovich

Departmental Technical Reports (CS)

The known 1970 solution to the 10th Hilbert problem says that no algorithm is possible that would decide whether a given Diophantine equation has a solution. In set terms, this means that not all Diophantine sets are decidable. In a posting to the Foundations of Mathematica mailing list, Timothy Y. Chow asked for possible formal justification for his impression that most Diophantine equations are not decidable. One such possible justification is presented in this paper.

Is It Legitimate Statistics Or Is It Sexism: Why Discrimination Is Not Rational, Martha Osegueda Escobar, Vladik Kreinovich, Thach N. Nguyen

Departmental Technical Reports (CS)

While in the ideal world, everyone should have the same chance to succeed in a given profession, in reality, often the probability of success is different for people of different gender and/or ethnicity. For example, in the US, the probability of a female undergraduate student in computer science to get a PhD is lower than a similar probability for a male student. At first glance, it may seem that in such a situation, if we try to maximize our gain and we have a limited amount of resources, it is reasonable to concentrate on students with the higher probability of …

Does The Universe Really Expand Faster Than The Speed Of Light: Kinematic Analysis Based On Special Relativity And Copernican Principle, Reynaldo Martinez, Vladik Kreinovich

Departmental Technical Reports (CS)

In the first approximation, the Universe's expansion is described by the Hubble's law v = H * R, according to which the relative speed v of two objects in the expanding Universe grows linearly with the distance R between them. This law can be derived from the Copernican principle, according to which, cosmology-wise, there is no special location in the Universe, and thus, the expanding Universe should look the same from every starting point. The problem with the Hubble's formula is that for large distance, it leads to non-physical larger-than-speed-of-light velocities. Since the Universe's expansion is a consequence of Einstein's …

Efficient Parameter-Estimating Algorithms For Symmetry-Motivated Models: Econometrics And Beyond, Vladik Kreinovich, Anh H. Ly, Olga Kosheleva, Songsak Sriboonchitta

Departmental Technical Reports (CS)

It is known that symmetry ideas can explain the empirical success of many non-linear models. This explanation makes these models theoretically justified and thus, more reliable. However, the models remain non-linear and thus, identification or the model's parameters based on the observations remains a computationally expensive nonlinear optimization problem. In this paper, we show that symmetry ideas can not only help to select and justify a nonlinear model, they can also help us design computationally efficient almost-linear algorithms for identifying the model's parameters.

Practical Need For Algebraic (Equality-Type) Solutions Of Interval Equations And For Extended-Zero Solutions, Ludmila Dymova, Pavel Sevastjanov, Andrzej Pownuk, Vladik Kreinovich

Departmental Technical Reports (CS)

One of the main problems in interval computations is solving systems of equations under interval uncertainty. Usually, interval computation packages consider united, tolerance, and control solutions. In this paper, we explain the practical need for algebraic (equality-type) solutions, when we look for solutions for which both sides are equal. In situations when such a solution is not possible, we provide a justification for extended-zero solutions, in which we ignore intervals of the type [−a, a].

What Is The Optimal Bin Size Of A Histogram: An Informal Description, Afshin Gholamy, Vladik Kreinovich

Departmental Technical Reports (CS)

A natural way to estimate the probability density function of an unknown distribution from the sample of data points is to use histograms. The accuracy of the estimate depends on the size of the histogram's bins. There exist heuristic rules for selecting the bin size. In this paper, we show that these rules indeed provide the optimal value of the bin size.

Granular Approach To Data Processing Under Probabilistic Uncertainty, Andrzej Pownuk, Vladik Kreinovich

Departmental Technical Reports (CS)

In many real-life situations, uncertainty can be naturally described as a combination of several components, components which are described by probabilistic, fuzzy, interval, etc. granules. In such situations, to process this uncertainty, it is often beneficial to take this granularity into account by processing these granules separately and then combining the results.

In this paper, we show that granular computing can help even in situations when there is no such natural decomposition into granules: namely, we can often speed up processing of uncertainty if we first (artificially) decompose the original uncertainty into appropriate granules.

How To Use Absolute-Error-Minimizing Software To Minimize Relative Error: Practitioner's Guide, Afshin Gholamy, Vladik Kreinovich

Departmental Technical Reports (CS)

In many engineering and scientific problems, there is a need to find the parameters of a dependence from the experimental data. There exist several software packages that find the values for these parameters -- values for which the mean square value of the absolute approximation error is the smallest. In practice, however, we are often interested in minimizing the mean square value of the relative approximation error. In this paper, we show how we can use the absolute-error-minimizing software to minimize the relative error.

A Thought On Refactoring Java Loops Using Java 8 Streams, Khandoker Rahad, Zejing Cao, Yoonsik Cheon

Departmental Technical Reports (CS)

Java 8 has introduced a new abstraction called a stream to represent an immutable sequence of elements and to provide a variety of operations to be executed on the elements in series or in parallel. By processing a collection of data in a declarative way, it enables one to write more concise and clean code that can also leverage multi-core architectures without needing a single line of multithread code to be written. In this document, we describe our preliminary work on systematically refactoring loops with Java 8 streams to produce more concise and clean code. Our idea is to adapt …

How To Get Beyond Uniform When Applying Maxent To Interval Uncertainty, Songsak Sriboonchitta, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, the Maximum Entropy (MaxEnt) approach leads to reasonable distributions. However, in an important case when all we know is that the value of a random variable is somewhere within the interval, this approach leads to a uniform distribution on this interval -- while our intuition says that we should have a distribution whose probability density tends to 0 when we approach the interval's endpoints. In this paper, we show that in most cases of interval uncertainty, we have additional information, and if we account for this additional information when applying MaxEnt, we get distributions which are …

How To Estimate Statistical Characteristics Based On A Sample: Nonparametric Maximum Likelihood Approach Leads To Sample Mean, Sample Variance, Etc., Vladik Kreinovich, Thongchai Dumrongpokaphan

Departmental Technical Reports (CS)

In many practical situations, we need to estimate different statistical characteristics based on a sample. In some cases, we know that the corresponding probability distribution belongs to a known finite-parametric family of distributions. In such cases, a reasonable idea is to use the Maximum Likelihood method to estimate the corresponding parameters, and then to compute the value of the desired statistical characteristic for the distribution with these parameters.

In some practical situations, we do not know any family containing the unknown distribution. We show that in such nonparametric cases, the Maximum Likelihood approach leads to the use of sample mean, …

How To Gauge Accuracy Of Processing Big Data: Teaching Machine Learning Techniques To Gauge Their Own Accuracy, Vladik Kreinovich, Thongchai Dumrongpokaphan, Hung T. Nguyen, Olga Kosheleva

Departmental Technical Reports (CS)

When the amount of data is reasonably small, we can usually fit this data to a simple model and use the traditional statistical methods both to estimate the parameters of this model and to gauge this model's accuracy. For big data, it is often no longer possible to fit them by a simple model. Thus, we need to use generic machine learning techniques to find the corresponding model. The current machine learning techniques estimate the values of the corresponding parameters, but they usually do not gauge the accuracy of the corresponding general non-linear model. In this paper, we show how …

Kuznets Curve: A Simple Dynamical System-Based Explanation, Thongchai Dumrongpokaphan, Vladik Kreinovich

Departmental Technical Reports (CS)

In the 1950s, a future Nobelist Simon Kuznets discovered the following phenomenon: as a country's economy improves, inequality first grows but then decreases. In this paper, we provide a simple dynamical system-based explanation for this empirical phenomenon.

In Education, Delayed Feedback Is Often More Efficient Than Immediate Feedback: A Geometric Explanation, Francisco Zapata, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Feedback is important in education. It is commonly believed that immediate feedback is very important. That is why instructors stay often late at night grading students' assignments -- to make sure that the students get their feedback as early as possible. However, surprisingly, experiments show that in many cases, delayed feedback is more efficient that the immediate one. In this paper, we provide a simple geometric explanation of this seemingly counter-intuitive empirical phenomenon.

Are Permanent Or Temporary Teams More Efficient: A Possible Explanation Of The Empirical Data, Francisco Zapata, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

It is known that in education, stable (long-term) student teams are more effective than temporary (short-term) ones. It turned out that the same phenomenon is true for workers working on a long-term project. However, somewhat surprisingly, for small-scale projects, the opposite is true: teams without any prior collaboration experience are more successful. Moreover, it turns out that if combine in a team members with prior collaboration experience and members without such experience, the efficiency of the team gets even lower. In this paper, we provide a possible explanation for this strange empirical phenomenon.

A Bad Plan Is Better Than No Plan: A Theoretical Justification Of An Empirical Observation, Songsak Sriboonchitta, Vladik Kreinovich

Departmental Technical Reports (CS)

In his 2014 book "Zero to One", a software mogul Peter Thiel lists the lessons he learned from his business practice. Most of these lessons make intuitive sense, with one exception -- his observation that "a bad plan is better than no plan" seems to be counterintuitive. In this paper, we provide a possible theoretical explanation for this somewhat counterintuitive empirical observation.

Taking Into Account Interval (And Fuzzy) Uncertainty Can Lead To More Adequate Statistical Estimates, Ligang Sun, Hani Dbouk, Steffen Schön, Vladik Kreinovich

Departmental Technical Reports (CS)

Traditional statistical data processing techniques (such as Least Squares) assume that we know the probability distributions of measurement errors. Often, we do not have full information about these distributions. In some cases, all we know is the bound of the measurement error; in such cases, we can use known interval data processing techniques. Sometimes, this bound is fuzzy; in such cases, we can use known fuzzy data processing techniques.

However, in many practical situations, we know the probability distribution of the random component of the measurement error and we know the upper bound -- numerical or fuzzy -- on the …

Entropy As A Measure Of Average Loss Of Privacy, Luc Longpre, Vladik Kreinovich, Thongchai Dumrongpokaphan

Departmental Technical Reports (CS)

Privacy means that not everything about a person is known, that we need to ask additional questions to get the full information about the person. It therefore seems to reasonable to gauge the degree of privacy in each situation by the average number of binary ("yes"-"no") questions that we need to ask to determine the full information -- which is exactly Shannon's entropy. The problem with this idea is that it is possible, by asking two binary questions -- and thus, strictly speaking, getting only two bits of information -- to sometimes learn a large amount of information. In this …

Maximum Entropy As A Feasible Way To Describe Joint Distributions In Expert Systems, Thongchai Dumrongpokaphan, Vladik Kreinovich, Hung T. Nguyen

Departmental Technical Reports (CS)

In expert systems, we elicit the probabilities of different statements from the experts. However, to adequately use the expert system, we also need to know the probabilities of different propositional combinations of the experts' statements -- i.e., we need to know the corresponding joint distribution. The problem is that there are exponentially many such combinations, and it is not practically possible to elicit all their probabilities from the experts. So, we need to estimate this joint distribution based on the available information. For this purpose, many practitioners use heuristic approaches -- e.g., the t-norm approach of fuzzy logic. However, this …

Possible Explanation Of Empirical Values Of The Matern Smoothness Parameter For The Temporal Covariance Of Gps Measurements, Gaël Kermarrec, Steffen Schön, Vladik Kreinovich

Departmental Technical Reports (CS)

The measurement errors of GPS measurements are largely due to the atmosphere, and the unpredictable part of these errors are due to the unpredictable (random) atmospheric phenomena, i.e., to turbulence. Turbulence-generated measurement errors should correspond to the smoothness parameter ν = 5/6 in the Matern covariance model. Because of this, we expected the empirical values of this smoothness parameter to be close to 5/6. When we estimated ν based on measurement results, we indeed got values close to 5/6, but interestingly, all our estimates were actually close to 1 (and slightly larger than 1). In this paper, we provide a …

Why Student Distributions? Why Matern's Covariance Model? A Symmetry-Based Explanation, Steffen Schön, Gaël Kermarrec, Boris Kargoll, Ingo Neumann, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In this paper, we show that empirical successes of Student distribution and of Matern's covariance models can be indirectly explained by a natural requirement of scale invariance -- that fundamental laws should not depend on the choice of physical units. Namely, while neither the Student distributions nor Matern's covariance models are themselves scale-invariant, they are the only one which can be obtained by applying a scale-invariant combination function to scale-invariant functions.

Efficient Algorithms For Synchroning Localization Sensors Under Interval Uncertainty, Raphael Voges, Bernardo Wagner, Vladik Kreinovich

Departmental Technical Reports (CS)

In this paper, we show that a practical need for synchronization of localization sensors leads to an interval-uncertainty problem. In principle, this problem can be solved by using the general linear programming algorithms, but this would take a long time -- and this time is not easy to decrease, e.g., by parallelization since linear programming is known to be provably hard to parallelize. To solve the corresponding problem, we propose more efficient and easy-to-parallelize algorithms.

Markowitz Portfolio Theory Helps Decrease Medicines' Side Effect And Speed Up Machine Learning, Thongchai Dumrongpokaphan, Vladik Kreinovich

Departmental Technical Reports (CS)

In this paper, we show that, similarly to the fact that distributing the investment between several independent financial instruments decreases the investment risk, using a combination of several medicines can decrease the medicines' side effects. Moreover, the formulas for optimal combinations of medicine are the same as the formulas for the optimal portfolio, formulas first derived by the Nobel-prize winning economist H. M. Markowitz. A similar application to machine learning explains a recent success of a modified neural network in which the input neurons are also directly connected to the output ones.

Why Some Physicists Are Excited About The Undecidability Of The Spectral Gap Problem And Why Should We, Vladik Kreinovich

Departmental Technical Reports (CS)

Since Turing's time, many problems have been proven undecidable. It is interesting though that, arguably, none of the working physicist problems had been ever proven undecidable -- until T. Cubitt, D. Perez-Garcia and M. M. Wolf proved recently that, for a physically reasonable class of systems, no algorithm can decide whether a given system has a spectral gap. We explain the spectral gap problem, its importance for physics and possible consequences of this exciting new result.

How Accurate Are Expert Estimations Of Correlation?, Michael Beer, Zitong Gong, Francisco Alejandro Diaz De La O, Vladik Kreinovich

Departmental Technical Reports (CS)

In many practical situations, it is important to know the correlation between different quantities -- finding correlations helps find the causes of different phenomena, and helps to find way to improve the situation. Often, there is not enough empirical data to experimentally determine all possible correlation. In such cases, a natural idea is to supplement this situation with expert estimates. Expert estimates are rather crude. So, to decide whether to act based on these estimates, it is desirable to know how accurate are expert estimates. In this paper, we propose several techniques for gauging this accuracy.

How Better Are Predictive Models: Analysis On The Practically Important Example Of Robust Interval Uncertainty, Vladik Kreinovich, Hung T. Nguyen, Songsak Sriboonchitta, Olga Kosheleva

Departmental Technical Reports (CS)

One of the main applications of science and engineering is to predict future value of different quantities of interest. In the traditional statistical approach, we first use observations to estimate the parameters of an appropriate model, and then use the resulting estimates to make predictions. Recently, a relatively new predictive approach has been actively promoted, the approach where we make predictions directly from observations. It is known that in general, while the predictive approach requires more computations, it leads to more accurate predictions. In this paper, on the practically important example of robust interval uncertainty, we analyze how more accurate …

Quantum Ideas In Economics Beyond Quantum Econometrics, Vladik Kreinovich, Hung T. Nguyen, Songsak Sriboonchitta

Departmental Technical Reports (CS)

It is known that computational methods developed for solving equations of quantum physics can be successfully applied to solve economic problems; there is a whole related research area called quantum econometrics. Current quantum econometrics techniques are based on a purely mathematical similarity between the corresponding equations, without any attempt to relate the underlying ideas. We believe that the fact that quantum equations can be successfully applied in economics indicates that there is a deeper relation between these areas, beyond a mathematical similarity. In this paper, we show that there is indeed a deep relation between the main ideas of …

What If We Do Not Know Correlations?, Michael Beer, Zitong Gong, Ingo Neumann, Songsak Sriboonchitta, Vladik Kreinovich

Departmental Technical Reports (CS)

It is well know how to estimate the uncertainty of the result y of data processing if we know the correlations between all the inputs. Sometimes, however, we have no information about the correlations. In this case, instead of a single value σ of the standard deviation of the result, we get a range [σ] of possible values. In this paper, we show how to compute this range.

Quantitative Justification For The Gravity Model In Economics, Vladik Kreinovich, Songsak Sriboonchitta

Departmental Technical Reports (CS)

The gravity model in economics describes the trade flow between two countries as a function of their Gross Domestic Products (GDPs) and the distance between them. This model is motivated by the qualitative similarity between the desired dependence and the dependence of the gravity force (or potential energy) between the two bodies on their masses and on the distance between them. In this paper, we provide a quantitative justification for this economic formula.