Produced by Vile, Jeff Zeigler and bandmate Adam Granduciel, Childish Prodigy was Vile's first album to be released on Matador Records, and was preceded by the EP, The Hunchback (2009), and the single, "He's Alright".
Regarding the track, "Losing Momentum (for Jim Jarmusch)", which references filmmaker Jim Jarmusch in its title, Kurt Vile noted: "I have an early EP my buddy Richie put out, called The Hunchback EP, and there’s a song called “Losing It” on there. The record is 45 RPM, but it doesn’t say it on there, so people were playing it at 33, and tons of people were coming up to me saying, like, “Dude, you should listen to that at 33.”