It's difficult to have the AI "close in the for the kill" at just the right moment. Code designed to do this all too often ends up making the AI engage in suicidal attacks -- then people write "why does the AI engage in suicidal attacks?" We spent a while tweaking the aggressiveness of the AI to get it where it's at. If we were to work on improvements, I think a good mechanism would be to vary the aggressiveness of the AI between battles, so players can't as easily predict just how hard the AI will move at them. We are working on a FOF patch right now, and I may be able to insert a randomized aggression factor, or to vary the level of aggression during the course of a battle based on global parameters, without too much extension to our schedule.
For what it's worth, for COG:EE we have made many improvements to the AI along these lines -- both on offense and defense. If we do a full FOF expansion some day, we may be able to fit many of these improvements back into the FOF engine.
As for explaining how the AI works, the core movement routines for the detailed battle AI are almost 5,000 lines long, so it's not easy to explain the algorithm with anything like concision. There are several different modes of movement the AI can be in depending on the global situation (advancing, fighting, falling back, retreating, etc.) and there are some variant routines among these modes (different routines for advancing, for instance). The core of the fighting mode optimizes the best attack a unit can make relative to the best attacks that can be made on that unit from that position, multiplied by a scalar factor. There are other factors applied to this, such as the vicinity of friendly units, supply, terrain, and so forth. The way these factors work, the AI does preponderately try to keep its lines together and avoid taking large casualties at the expense of making a strong attack.