additional HTML (that are not form elements) are typically added to the $form array using using #prefix and/or #suffix instead of #value.
After I posted I realized I can to a _theme function to go with the _form function to help lay things out nicely without having to add .tpl files or other theme overhead to the project. I think it's time to go make friends with drupal_render().