This is a good overview of what is at stake:

http://www.aaronkellylaw.com/sue-over-a-meme/

I think it is difficult, and most people are not at risk, but there still exists a risk. Defamation, publicizing private data/info, false light, invasion of privacy, copyright infringement and unauthorized us of property are all named in that link.

Like I said, the risk may be small, but always "think before you meme".