Antoinette L. Matlins, PG & A.C. Bonanno. FGA, PG, ASA. Jewelry & Gems, The Buying Guide, 3rd Edition.
Woodstock, Vermont: Gemstone Press, 1993. If you like to know what you're buying and can only afford one book, this is it. Highly recommended.
Antoinette L. Matlins, PG & A.C. Bonanno. FGA, PG, ASA. Gem Identification Made Easy, 2nd Edition.
Woodstock, Vermont: Gemstone Press, 1997. Then buy this one.
Christopher Cavey. Gems & Jewels.
Secaucus, New Jersey: Wellfleet Books, 1992. Lots of lovely pictures of pretty sparklies!
Jeffrey E. Post. The National Gem Collection.
New York, New York: Harry N. Abrams, Inc., 1997. Pictures from the gem collection at the Smithsonian.
Not as good as going yourself, but worthwhile until you can get there.
Arthur Court & Ian Campbell. Minerals, Nature's Fabulous Jewels.
New York, New York: Harry N. Abrams, Inc. Has some beautiful photographs of minerals.
Frank J. Anderson. Riches Of the Earth.
New York, New York: Windward, 1981. Excellent historical reference on how our ancestors used gems and minerals.
Alain Eid and Michel Viard. Minerals of the World.
Edison, New Jersey: Chartwell Book, 1997. Consists almost entirely of wonderful photographs of minerals.
Michael O'Donoghue. An Illustrated Guide to Rocks & Minerals.
New York, New York: Smithmark, 1991. Pretty much what the title says.