That sounds a bit like an ovarian cyst. Those are less common on many estrogen-containing pills but not minipills or low-dose pills. If it is a cyst, those are quite common and rarely a big problem (though they can be painful if they rupture). Check with a doctor if you feel severe pain or a fever.