My recollection is that a successful "incite revolt" mission is not guaranteed to cause the colony to break away and join your empire. If you want the colony for yourself, you may have to invade it even after successfully pulling off an "incite revolt" mission.
Also, if your agents get caught or detected, it'll damage your empire's reputation as well as its relations with the neighbor you're targeting. Poor reputation causes penalties to diplomatic relations; in combination with the direct relations penalty for conducting offensive covert operations against them, it may be enough to push you into the war you're nominally attempting to avoid by messing around with agents. It may also damage relations with other neighbors, causing them to terminate Free Trade Agreements and Mutual Defense Pacts or impose sanctions, or in more extreme cases wage declared or undeclared war on your empire. Additionally, my recollection is that there is a colony happiness penalty associated with a poor empire reputation for at least some species, though I could be mistaken.
I would additionally advise you that I don't think that the mission success rate indicated in the character screen is entirely accurate, especially for the longer missions.
As to the question of whether or not it's cost effective, my personal opinion is that taking individual colonies away from another empire is perhaps one of the least-useful things that an agent character can do with their time, all the more so in your case since it seems to me as though these are likely relatively young and not particularly valuable frontier colonies rather than older, more developed colonies closer to your neighbor's core. If you are a large and reasonably well-established empire, and a neighboring empire is strong enough that getting into a war with them is reasonably concerning, then that neighboring empire is probably strong enough that the loss of just one or two colonies - especially isolated frontier colonies, which are fairly likely to be relatively easily captured in the early stages of a war - is unlikely to weaken them to any significant degree, especially in the short term when whatever you did to take their colony away might provoke the war you were trying to avoid anyways. I'd much rather have my agents going after research information when at peace and maps (preferentially operations, though galaxy and territory maps can be at least somewhat useful) or, if possible, deep cover in preparation for or in support of a war, and maybe sabotage of a major shipyard or gas mining stations over fuel sources if I think it'll be useful enough to the war effort.
As to the question about the alien population, they'll have a happiness penalty for being at war with an empire of their own species, but it probably won't be a significant factor on any colony not predominantly populated by them, and my feeling is that it's unlikely to be that much of an issue on colonies predominantly populated by them unless they're already close to rebelling. Might perhaps want to drop taxes a notch or two to offset the happiness problems, and if colonies start looking really rebellious maybe have an invasion force or two on standby to redress their concerns and complaints at gunpoint, but probably not much of an issue overall.
As to the issue of insufficient ground forces for adequate invasion forces, while I do not know how many troops you have in total or how many troops are on your potential targets, your statement that you have a number of Robotics Facilities and Armor Factories suggests to me that you probably have significant garrison forces. You might want to consider pulling some of the garrison forces for service in the invasion forces. My personal feeling is that planetary garrisons are nothing more than speedbumps to an adequately-powerful assault force; if you cannot defeat the invasion in space, you're unlikely to beat them on the ground without reinforcements arriving from elsewhere, and unless you can contest control of the space around the colony at the same time it's probably not worth the risk to try and bring in reinforcements.