Good question! I understand your worries, I'd have them myself too. Sharing access to your inbox is a big thing. But without it, this couldn't exist. If you don't like to, that's okay, you can still use this site by listening to other people's inbox.
This site requests read-only access from Gmail to search your inbox for music links over a secure connection (called SSL). Those links are saved along with the subject of the mail and the recipients (to display this to you). The body or anything else from your messages is not saved. And only the mails with music links in them are visible. After 60 minutes, Gmail kills this site's access to your inbox!
It then displays all the music from your inbox that it could find, again over a secure connection.
To give you a personal guarantee of safety. Here's a link to my blog. I made this as part of my objective to launch 12 startups in 12 months.
Play My Inbox collects music links from your inbox and streams them to you
or see a demo