@matt it will make things a bit slower for your users. I usually do 60 seconds for stuff that I expect to change regularly or when I'm planning a migration to a new server/IP, 5 minutes for stuff I don't have any plans to change.
Zero TTL would be, IMHO, for services that move around a lot via some dynamic mechanism. I can't think of any time I'd set TTL that low, or any benefit to doing so.
But, because Route 53 is very distributed with Anycast, it probably doesn't have a huge impact.