PDF version of this document
The term entropy (commonly also referred to as Shannon's entropy [WWW-15]) is used to denote a general measure of uncertainty. It is not a very sophisticated idea, yet a very fundamental one which was first introduced in 1948. Uncertainty is associated with the required amount of data so it can also be thought of as an information measure or quantifier. The value that quantifies uncertainty originally related to random variables which take different probabilities amongst a set of states (reminiscent of Markov chain models). Shannon's entropy has become a very useful way of evaluating structures and pattern in some data. The lower the entropy value, the more data is already inherent in that data. In a sense, the entropy indicates how much can be learned from the data and what is still unknown.