Finance is related to mathematics, but the high number, linearity and probability you mentioned are too professional, and many things are useless or used in different ways in finance. Some concepts in finance are not found in mathematics. You are good at math. I can tell you that the two subjects are different sets, but they overlap. Don't confuse them.

If you want to teach yourself finance, I suggest you look at CFA (Chartered Financial Analyst). I'm taking the exam now, and it's good if I don't want to take the exam. It's also good to see some money banks, economics and corporate finance by myself.