First I would never buy a used starter especially from the yard, unless you don't plan on keeping the car very long, imo it's not worth taking that chance just pay the extra and get a refurb or new and your good with at least a 2 yr to lifetime warranty. my2cents
Anyways as far as teh oil leak I'm assuming you did the install yourself , did you make sure to look over the area thoroughly? I'm trying to remember if there's something oil related on the Starter side but it shouldn't be that hard to find just follow the oil. Is is dripping near the Timing cover area? Bell housing area etc?