Reduced cohesion has two main effects:
1) It reduces the amount of purchase points the BG gains every turn.
2) It reduces the overall force morale for the BG in a battle (making your force morale 'break' easier).
The effects are not linear, however. If you keep a BG's cohesion 'in the green' (above about 70%) you won't see much of a penalty at all. As the cohesion drops below this the effects become more pronounced. If cohesion drops 'into the red' at the low end of the scale (below about 30-40%) the affect become magnified.
There is also an effective speed reduction to the BG if it drops below 'green' cohesion, making it act as if it were one speed rating slower, and an additional one level of speed reduction if it drops to 'red' cohesion. The speed rating is used to determine which BG gets to move first (and interrupt the other) when they are coming at each other head-on.
As Andrew pointed about above, the rating bar for cohesion shows in 10% increments, so it won't always show small changes in the current rating. And it is also a scaled system, so the lower the current cohesion, the more benefit you get from resting, and the higher the cohesion, the less the benefit.
All that being said, a good rule of thumb is that if you keep your cohesion bar 'in the green' (around 70-80% or above) you will rarely have to worry about the effects. Resting overnight is generally the best strategy, but even this can be ignored for a day or two if necessary. If it falls into the 'red' (30-40% or below) the BG will be fighting with magnified cohesion penalties.
Resting overnight will restore the most cohesion to a battle group. Resting for a day turn will restore about 20% of the full night's amount. Also, a BG that rests will usually get to bring in replacements to fill out soldiers in depleted units, though this can fail to happen if the BG is in the moderate to low range of cohesion, or is not fully supplied.
Any movement and/or combat reduces cohesion by a small amount. The big source of cohesion loss is losing a battle. Being forced to retreat is a very big hit. Just having your force morale break increases the cohesion loss for combat significantly, and there are additional penalties if you manage to lose very quickly (in the first few minutes) or in a very lopsided manner (your losses are very high and opponents losses are very low).
Hope that helps,