This book presents exercises and problems in the mathematical methods of physics with the aim of offering undergraduate students an alternative way to explore and fully understand the mathematical notions on which modern physics is based. The exercises and problems are proposed not in a random order but rather in a sequence that maximizes their educational value. Each section and subsection starts with exercises based on first definitions, followed by groups of problems devoted to intermediate and, subsequently, more elaborate situations. Some of the problems are unavoidably "routine", but others bring to the fore nontrivial properties that are often omitted or barely mentioned in textbooks. There are also problems where the reader is guided to obtain important results that are usually stated in textbooks without complete proofs. In all, some 350 solved problems covering all mathematical notions useful to physics are included. While the book is intended primarily for undergraduate students of physics, students of mathematics, chemistry, and engineering, as well as their teachers, will also find it of value.