He was simply an altered visual style of Meat Boy, and was returned back to Meat Boy after the day was over. Allegedly, Potato Boy was made as a "healthy" replacement for Meat Boy; he does not leave a trail of blood as he moves.
The only way to play as Potato Boy now is with a glitch called bandage duplication. An explanation to the glitch can be found here (read description), and is only possible on the PC version of the game. Dr. Fetus is also playable with this glitch.
Some pirated versions of the game have Potato Boy as a permanent replacement to Meat Boy. It is rumored that this was done by Team Meat to reduce piracy. It might also be that the pirated version is the same version as on April Fools 2011.