the mouse is supposed to be the vehicle of lord Ganesha.
there is a story to it . one day lord ganesha was mounting on this vehicle and had a fall, the legend says the moon laughed, ganesha was angry he cursed the moon that nobody will even look at him , the moon begged parden and asked for forgiveness. ganesha pardoned the moon, but not fully on the ganesha chaturthi day if anybody looks at the moon then he will bear the curse of ganesh. the curse was that he will be falsely implicated in a crime.this is what the legend says.