Don’t feel guilty about doing something the psychiatrist supports!
It’s easy to let paranoia squeeze in so it’s important to keep some perspective. They won’t kick you off disability just for having a lemonade session at your local. If they queried you then your doing what your doctor suggests. I’m sure they wouldn’t even bother even if someone complained!
Keep your chin up!