I had to have all 4 of mine removed due to them becoming impacted. You can try stuff like Oragel but it didn't work for me. Clove oil supposedly also helps but it never did crap for me either.
About the only thing I could do was pace around in circles, sleep so that the side that was hurting was not the side I was laying on, and take 3 ibuprofen at a time until I could go into the dentist. Putting something cold on it also helped as it caused the inflammation to decrease.