First of all, you should take care of your preferences. Since you like math, let's take an examination of graduate students majoring in math. After you get your master's degree, that is, three years later, your idea of taking an actuary exam may change. I graduated with a master's degree in mathematics, and now I stay in the university with my tutor to continue my research, because I really like mathematics.