Apparently there is/was major controversy between stevia and the Food & Drug Administration. Since stevia is an herb that is easy to grow and the sugar industry felt threatened by it, the FDA ruled that it could only be sold as a "dietary supplement" only.
As for xylitol, I read of no controversy.
I've tried neither, but I just ordered a pack of xylitol packets from www.xylitolstore.com.
So here are some links with information about each sweetener, for those who are interested: