He's not from Caliban. He was never from Caliban. It presumably said this because in the movie novelization is refers to Mal meeting Jayne on Caliban. It never says that he's from there.
Jayne's homeworld is named explicitly as Sycorax in The Ghost Machine.
I doubt that Mal and Jayne met on Caliban truth be told. You know where Caliban is? It's Miranda's moon.
Fun fact: Miranda, Caliban and Sycorax are all named after characters from The Tempest.