Pony Up! formed on New Year's Eve 2002 and, upon developing a relationship with singer Ben Lee, released their debut, self-titled EP, Pony Up! in 2005 on his friend Steve Aoki's label, Dim Mak.
My Little Pony: Friendship Is Magic | Pony Express | My Little Pony | Shetland pony | Trick Pony | Pony Canyon | Pony Club | pony | The Stone Pony | Coffin Bay Pony | Welsh Pony and Cob | The Red Pony | My Pretty Pony | Highland pony | Exmoor pony | Australian Pony | Zaniskari pony | Pony Up! | Pony Poindexter | Pony, Montana | Pony Express (roller coaster) | Pony Express Bridge | Pony Express (1860-1861) | pony express | Pony Da Look | Pony | My Little Pony: Friendship is Magic | Hucul pony | Galician Pony | Batak Pony |
The band began performing around Montreal on a regular basis, performing with bands such as The Dears, Bad Flirt, Rance Mulnix, Beaver, The Alex Chow Symphony Orchestra, Telefauna, Rocket v. Robot, Raw Madonna, Stars, The Diskettes, Pony Up!, Dead Bush, and many others.