I think blaming the AI to the exclusion of all else is wrong. The game conditions are exploitable.
AI can always be improved. That's a given. I would rather see the exploits removed first.
I totally agree with jp on this. The main problem is that where there is an exploit, an intelligent human player will always find a way to turn it to his/her advantage.
One way to fix this, and thus make the game harder, is either to remove the exploit, or make the AI adapt to it.
I will detail a few concrete ways of doing this quite simply (by identifying the exploit and finding a way to even things for the AI):
1) When I first started playing this game and had everything on automated, the very first thing I felt compelled to do was take control of exploration. I quickly realised how exploring would give me an edge over the AI by finding strategic resources, key colonisable worlds, derelict battleships, colony ships, lost independent colonies, super-weapons, world destroyers etc...
the AI always under-estimates the importance of exploration. Look and see for yourself in the race policy folder. Nearly all races (except Zenox) are set at "Exploration Priority ;1,0"
Change that value to "2,0" (after saving all original policies in a separate folder) for all races and you will give the AI a very neat, cheat free, edge.
2) The second thing I focussed on after exploration was research (by the way, I never use the tech trading exploit which I personally find silly as it is). I don't think I need to explain just how important research is in any strategy game.
Again, if you look in race policies, only the Quameno and Kiadian are set "2,0" for "Research Priority". All other races are set at 1,0 or 1,5. Don't be afraid once again to give every race a solid "2,0". The game will not lose it's "unique race" flavour cause the Quameno have both Technocracy and gifted scientists, thus they will naturally remain ahead of.. say for ex: the Atuuk... in science.
3) Thirdly, you guessed it... ship design. And that does not mean simply adding more guns to the design. An intelligent player will always make a design that fits the need.
Unfortunately, it's very complicated to make the AI better at ship design without cheating. For example, if you armed all your ships with ubber latest design titan beams but no long range weapons, you'd be in trouble if the AI was "given" that info and decided to equip his ships with fast thrusters and long distance missile/torpedoes with orders to stand-off.
You'd also be in trouble if the AI realised that your ships had many shields but only a few layers of armour and decided to equip his ships mainly with railguns.
A human player can adapt. Unfortunately, an AI doesn't... unless *coughs* the programmer *cough* found a way to implement that somehow
I think I'll stop at those three points for now but trust me, there are many other ways of making this game interestingly harder (or more fun).