Depends what you mean by "small". 3 inches is probably the point when worthwhile women start to see it as a meaningful deficit, as it then represents a hard functional problem. Pun intended.
Overall foreplay, setting, emotional connection, etc, is important regardless of penis size. I'm "only" 6 inches or so, at maximum, and can only see anything larger being a hindrance.
Either way, there are women that see larger dicks as a negative. Just like there is such a thing as hips or breasts that are simply too large for men.