Package rdkit :: Package ML :: Package InfoTheory :: Module entropy

Module entropy

Informational Entropy functions

The definitions used are the same as those in Tom Mitchell's
book "Machine Learning"

Functions

[hide private]

PyInfoEntropy(results)
Calculates the informational entropy of a set of results.

source code

PyInfoGain(varMat)
calculates the information gain for a variable

source code

InfoEntropy(results)
Calculates the informational entropy of a set of results.

source code

InfoGain(varMat)
calculates the information gain for a variable

source code

Variables

[hide private]

hascEntropy = 1

_log2 = math.log(2)

Imports: numpy, math, xrange, cEntropy

Function Details

[hide private]

PyInfoEntropy(results)

source code

Calculates the informational entropy of a set of results.

**Arguments**

  results is a 1D Numeric array containing the number of times a
  given set hits each possible result.
  For example, if a function has 3 possible results, and the
    variable in question hits them 5, 6 and 1 times each,
    results would be [5,6,1]

**Returns**

  the informational entropy

PyInfoGain(varMat)

source code

calculates the information gain for a variable

**Arguments**

  varMat is a Numeric array with the number of possible occurances
    of each result for reach possible value of the given variable.

  So, for a variable which adopts 4 possible values and a result which
    has 3 possible values, varMat would be 4x3

**Returns**

  The expected information gain

InfoEntropy(results)

source code

Calculates the informational entropy of a set of results.

**Arguments**

  results is a 1D Numeric array containing the number of times a
  given set hits each possible result.
  For example, if a function has 3 possible results, and the
    variable in question hits them 5, 6 and 1 times each,
    results would be [5,6,1]

**Returns**

  the informational entropy

InfoGain(varMat)

source code

calculates the information gain for a variable

**Arguments**

  varMat is a Numeric array with the number of possible occurances
    of each result for reach possible value of the given variable.

  So, for a variable which adopts 4 possible values and a result which
    has 3 possible values, varMat would be 4x3

**Returns**

  The expected information gain