So, what I really want is a module that implements "hook_action_form_alter()" that can add my term filtering to any action config form, get notified when the action is invoked, and decide whether or not it should execute. hook_action_form_alter() should have a $op argument for: 'form', 'validate', 'pre-execute' (can return false to prevent execution), 'execute' (action just executed), maybe others too.
Seems to me that this is a specific instance of a conditional, with a proposed mechanism for how conditionals would be implemented. I agree that conditionals are eventually necessary, probably on an action set and individual action level. You can form_alter the actions configuration form currently by targeting form id 'actions_configure'.