Expanding into a new area requires fleets to defend the initial structures, and it's those fleets that I need to have confidence that they will keep doing the task I set for them.
A couple things:
Firstly, there is usually a window of opportunity in which to set up stations, as pirates (and, really, all computer-controlled factions) will not typically revisit a system after initial exploration unless there was something of interest in that system or unless they become aware of activity in that system (e.g. through agent activity). As a result, you can usually set up at least one or two mines in uninhabited or recently-uninhabited systems where no pirate mines are present before the pirate factions become aware of the potential targets.
Secondly, mining stations are usually one of the last things I set up in the regions that I expand my empire into. Fleets go in first to reduce pirate activity to manageable levels, then colony and troop ships go in to grab territory, possibly accompanied or followed by more fleets to finish securing the region. Mining stations can wait until the region is secure or until I decide that the region needs to have greater resource production. There are really only two things for which local resource production is all that necessary - significant ship construction activity, and fuel. You don't need to and probably should not have significant ship construction activity going on in frontier regions, while the fuel supply situation can be covered by resupply ships (resupply ships have the added advantage of not attracting freighters looking to pick up cargo, which means less freighter traffic in a relatively dangerous area and less issues with the local fuel supply drying up because a freighter just hauled half the local stockpile off to the other side of the empire). There might be a rare strategic resource or two for which you need to build a mine in the area because you lack other sources, but other than that I wouldn't bother.
Do you think it's better to build bases and then destroy or retrofit into cheaper versions as the area becomes my quiet backfield--cheaper than simply changing the duty of the defensive fleet to the next hot zone?
The short answer is that I wouldn't do this, but then I also wouldn't normally defend mines or construction ships in the process of building mines in the first place.
Longer answer: When I suggested building defense bases, I thought that you wanted permanent defenses at the location, not temporary defenses. I expect that keeping track of where the temporary defensive bases are would be enough of a headache to not be worth any potential cost savings, though it's entirely possible that it would be cheaper (how much so, however, would be heavily dependent upon ship and station designs, on how many ships and how many stations you'd be using, on how long you keep the station/fleet at any given location, and on whether you're scrapping or retrofitting stations which are no longer needed); if you went for the retrofit option rather than the scrap and rebuild option, there's also the issue that keeping multiple current designs in a single role can be a bit of a headache, especially if you use automated or mass retrofits. Another issue is that if you aren't able to build a mining station before the construction ship gets chased off or the mine gets destroyed by pirates, I doubt that a defensive base would fare all that much better (armor starts working as soon as it's built, and shields and weapons can sometimes come online before the station is completed, so it's possible).
I also don't particularly care for fixed defenses. Space superiority wins the game, and the most stations can do for you under normal circumstances is help maintain local superiority in your own space. They're useful enough that I'd consider building them for sufficiently important locations, but sufficiently important locations are going to be places like Korrabia or the Fortress of Torak, not random mining stations under threat of pirate attack.
Some other thoughts:
In my opinion, it is for the best, especially in the early game, to concentrate mining activity into as few systems as possible, preferably as close to the primary points of use (typically, significant shipyards, and in the early game the only significant shipyard you probably have is your home spaceport). Hyperdrives, especially early-game hyperdrives, are slow relative to the size of a sector, so keeping the mines close to the point of use keeps travel times short, which in turn reduces the time it takes your freighters to respond to resource shortages (of course, if the shortage is caused by the mines not producing enough, this doesn't help that much). Concentrating the mining activity into as few systems as possible allows you to economize fleet deployments (assuming you are going to deploy the fleet to protect mines, which is something that I almost never bother doing). Mining targets go something like home system > nearby colonized systems > colonized systems > nearby high resource systems > high resource systems > everything else for me; colonized systems get higher priority simply because they're the systems where I'm most likely to deploy a defensive fleet, and I might as well have the fleets pull double duty defending mines and colonies.
When trying to clear pirates out of an area, scouting information is invaluable. Long-range scanners are great if you have them, but even a few scouts bouncing around local systems can work. Find the pirate spaceports and fuel mines and independent colonies, get a sense of how strong the pirate fleet presence is in the area, and decide whether you can make a direct assault against the pirate infrastructure. If you think you can, great; send fleets to assault the pirate spaceports and fuel mines and invade the independent colonies as the first step to clearing out the pirates from the region. If you think you cannot, or cannot while taking only acceptable losses, it's time to be more devious. You know how you were annoyed by your ships haring off after ships on the opposite side of the system? You can take advantage of that behavior to pull hostiles into ambushes or away from your real targets, because the computer's ships behave the same way. Jump a bait squadron to a point in the system relatively far from the colony or pirate station while a kill squadron sits just outside the system; once the pirates start moving against your bait squadron, move the kill squadron to the bait squadron or to the real target. For that matter, just jumping to a point in the system other than the colony or pirate station can make a big difference in how a battle plays out - any group of ships jumping to the same target coordinates will tend to emerge from hyperspace distributed in a ring around the target, which means that they're relatively dispersed compared to the ships at the target coordinates. Jumping to a point where the pirate ships are not gives your fleet time to concentrate itself and gets the pirate ships which had been concentrated in a big pile at the space station or colony to disperse a little and may not pull all of the pirate ships at once, and you can take further advantage of this dispersal by ordering your fleet to move or attack towards a certain section of the ring of pirate ships rather than letting the fight develop however it would go.
Agent missions, most notably Steal Operations Map and Deep Cover but also Steal Territory/Galaxy Map, can substitute to some degree for scouting; any of these should allow you to locate bases and mines, and Steal Operations Map and Deep Cover will additional help locate fleet elements (most importantly construction ships, which can otherwise be a pain to hunt down), and sabotage missions can be used to go after the pirates' fuel economy if you do not believe that the cost of attacking it with fleet elements is justified by the potential gain. I will however say that my feeling is that agents who get sent on missions against pirate factions are even more likely to die an early death than agents who get sent on missions against other empires.
PS--About those crazy pirates, I do actively hunt them when I'm capable of it, but in my current game, I've chased away no less than 3 different pirate factions from the same damn nebula--I spent the time cleaning them out only to have a new one mysteriously pop up and begin sending waves of ships at my background. This got to be such a waste of resources, that I made a cheap, placeholder base named "Denied!" simply to fill up that void. My hope is this will stop them from chain-spawning (worth the 1300 credits per year, if it works; yet another annoyance if it doesn't work, heh).
If you didn't do so for this game, I would suggest ensuring that the "Destroyed Pirates do not Respawn" tickbox is checked the next time you set up a game.
Assuming "Destroyed Pirates do not Respawn" was not checked: Standard pirate factions, as far as I know, only spawn over unoccupied fuel sources, though there might be potential for screwy behavior if no such locations exist. Building a mine at the location should prevent new pirate factions from popping up in that location, though existing pirate factions may still send construction ships or attack forces to the area (they will only send construction ships if they do not know that there is already a mine there).