This time of year is my absolute favorite! In addition to my Buddhist practice, I am also what is considered a Solitary. This means I practice my craft alone, and do not meet regularly with a coven.
Samhain is referred to "Halloween" because the Catholic church created All Saints Day (or All Hallows Day) on November 1st, in an attempt to turn people away from the original Pagan holiday.
The night before became known as "All Hallows Eve", which then got shortened up to Halloween.
If you are concerned about the Pagan nature of October 31st, then you should probably stop celebrating Christmas and Easter, too. Those holidays have been thoroughly adopted by the Christian religion & they originated as Pagan celebrations (Yule and Ostara). Halloween is the only holiday that has kept most of its Pagan meanings, without the Christian overtones.