It sounds to me like you have two things you care about here: customisation, and community.

I don't know the lengths to which Tumblr is customisable, but for a one-person operation it sounds like it's fine. I'll point out that if you self-host Wordpress you can do pretty much whatever you want with it, and plugins make it extra-powerful. But that's beyond what it sounds like you want. It may pay to get a domain and have it point to your site, either way. This way people who follow your work have one place to go to find you, and that place won't change even if you change platforms.

The main problem I see with a Tumblr-powered blog is that Tumblr hosts it all, which means that if they go down/go out of fashion in 2-3 years you have to pull everything out and insert it into your new blog engine. Also, you have a bunch of followers outsider of Tumblr, a decent proportion of which you're going to lose when you switch, I'm guessing. (Incidentally, doesn't hurt to keep a big "I'VE MOVED HERE" sign on your wordpress blog for six months, in case people spot it in google or the like.)

Re: exposure. I'm guessing that involving yourself in writing communities (both local and on the internet) would help? The problem with the internet and its low barrier of entry is that there's so much noise out there of everyone spouting their thing. If you have a community (as you would do with Tumblr) people can find out about your stuff slowly, as your friends reblog you being clever etc.. I tend to feel that's a better way to gain followers than spamming everyone you know with links.

However! I am generally not good at promoting stuff on the internet: it's not something I've attempted much of nor studied. So take that all with a grain of salt.