In Python Small Floats Tending To Zero

I have a Bayesian Classifier programmed in Python, the problem is that when I multiply the features probabilities I get VERY small float values like 2.5e-320 or something like that

Solution 1:

What you describe is a standard problem with the naive Bayes classifier. You can search for underflow with that to find the answer. or see here.

The short answer is it is standard to express all that in terms of logarithms. So rather than multiplying probabilities, you sum their logarithms.

You might want to look at other algorithms as well for classification.

Solution 2:

Would it be possible to do your work in a logarithmic space? (For example, instead of storing 1e-320, just store -320, and use addition instead of multiplication)

Solution 3:

Floating point numbers don't have infinite precision, which is why you saw the numbers turn to 0. Could you multiply all the probabilities by a large scalar, so that your numbers stay in a higher range? If you're only worried about max and not magnitude, you don't even need to bother dividing through at the end. Alternatively you could use an infinite precision decimal, like ikanobori suggests.

Solution 4:

Take a look at Decimal from the stdlib.

fromdecimal import Decimal, getcontext

getcontext().prec =320Decimal(1) /Decimal(7)

I am not posting the results here as it is quite long.