When a user sends you an invitation to a new game, you will have the possibility to reject the invitation if you do not want to play.
Classic new games will be shown in the “Games” tab. When you see the invitation that you have been sent, a pop-up where you can either reject or accept the game will be displayed.
Please note that rejecting a game before starting to play will not affect your game stats.