I think breeding within families like marrying cousin or something is probably not good and maybe have bad effects over time.
However I think if you have a decent sized population and you avoid cousin fucking then there shouldn't really be much problems with inbreeding.
Though it would probably help also to have some movement like people from one side of town marrying people from other side of town, or in big city or high density areas this would happen naturally.
I am also skeptical of people who think that mixing races always produce stronger or better outcome.
I think it is complicated, maybe sometimes it work well but othertimes not so good.