Please perform the following technical research:
Obtain the position of the character from the ABBYY library. This rectangle, has to be scanned pixel by pixel for the statistical color filling.
The first (mostly encountered) color would probably be the background. The second would be the font color. The rest is noise. In worst cases the font color and the background would be swapped for extremely bold fonts and thick digits.