On any other SE site, I don't get it. When I click close this message, the next time I visit a question, I get the same message. Why is this, and can I turn it off?
This isn't a bug, this is by design:
Welcome back! If you found this question useful, don't forget to vote both the question and the answers up.
only appears if
- you haven't been seen on the target site for 48 hours
- you hold a valid user cookie on the target site
- your account has more than 15 rep on the target site
- you arrive on a question from a search engine
If you are a very infrequent visitor, yet tend to hit a lot of web search results... you might want to log out. But you really should be voting up all those search results if you found them, and they are helping you!