10^(T*p)
(the log-likelihood is increased by T*p*log(10)
) where T
is the
number of observations and p
their dimension.
exp(-40^2/2)
will already be
rounded to zero (in double precision) so that these type of problems indeed
occur. The way the problem usually appears is that you get the message
``Warning: Divide by zero'' caused by the following line of hmm_fb
(or
the equivalent line in mix_post
):
alpha(i,:) = alpha(i,:) / scale(i);because all values of
alpha
for some time index i
are null. If this
occurs check your initialization (are the mean vectors centered on the data?)
and try increasing the variances (especially useful if you have outliers).
hmm_mint
does not correspond to an "optimal" initialization (if there
is one). It is just an heuristic commonly used in speech processing which
consist in chopping each parameter sequence into N
segments of equal
length where N
is the number of states. It will clearly not work if
you have many states, many allowed transitions (ie. many entries of the
transition matrix not initialized to zero) and few training sequences.
N
states and
mixtures of K
Gaussian densities as state conditional distributions is
equivalent to an HMM with N*K
states with some constraints on the
transition matrix. There is however two limitations in using H2M for that
purpose: (1) you will have to modify the EM re-estimation formulas to take
into account the constraints on the transition matrix (should not be too
difficult), (2) you will rapidly have to deal with huge transition matrices.
Olivier Cappé, Aug 24 2001