Yeah, I guess that's cool if a) they have an audience, and b) they return full autonomy and rights to you after a reasonable period of time.
humanodon is correct about publishing a book or chapbook; usually at the end there is a "notes" sort of section where the author can accredit the 1st publication. After publication, rights revert to the author. Unfortunately, "first publication rights" can be quite stringent; some magazines consider even posting the poem on a private Facebook account or blog (i.e., one you can't see without having a password or an invite of some sort) as "first publication." Of course, this does beg the question of "How would they be able to verify?" Some publications are a little more lenient and will say it can be published on Facebook, or if you have already put it up on a blog as long as you delete it you're okay. But it varies. Sometimes they are more specific and want, say "First North American publishing rights."
Yeah, I'm actually not as clear on the terms as I should be, but I'm pretty sure that it's just so that when a writer comes out with a book or chapbook, or the poem is otherwise reprinted, that the publication gets a mention like, "this poem originally appeared in the May 2014 issue of Juggs" (for example). I don't think they generally prevent a writer from choosing to reprint or republish a work in another publication or medium.