What are the best textbooks if I'm interested in an advanced field?

While it may be helpful to dive deeper on a particular interest, no textbook that is introduced here should be read primarily for the purpose of preparing for regular Olympiad exams with the exception of organic chemistry.

Miessler's Inorganic Chemistry is compact, concise, and overall provides a decent introduction to the field, although some of its explanations can sometimes be rather confusing and difficult to understand.
Housecroft's Inorganic Chemistry is long, detailed, and very dense. Most of its explanations are simple, but it contains a lot of specific knowledge that isn't required for Olympiads.

Klein's Organic Chemistry is well explained, simple, introductory, and focused on mechanisms. This is the only organic chemistry textbook that is truly required for Chemistry Olympiads, but might not be sufficient for competitions harder or equivalent to IChO.
Clayden's Organic Chemistry is also well explained, but more detailed, organized, and retains the focus on mechanisms. This is one of the best books in the field of Organic Chemistry but not necessary for Chemistry Olympiads.

Atkins' Elements of Physical Chemistry is a well explained, decent introduction to physical chemistry, and doesn't go into too advanced mathematics. If you are truly interested in this field, you should check out this book.

Harris' Quantiative Chemical Analysis is a great read with practice for advanced calculations and very good practice problems.

Lehninger Principles of Biochemistry is a well-written biochemistry textbook, although not relevant to Olympiad exams.