Blacksmiths work with "black" metals, especially iron. The black color comes from a layer of oxides that form on the surface of the metal during heating (called fire scale).
The term "smith" originates from the word "smite", which means to hit. Thus, a blacksmith is a person who smites the black metals.