Again, I think this has been put to you a few times now, but what happens when a person is honestly convicted that they know what God’s assertions about goodness are, and those assertions are harmful to others? This is not just a philosophical question for me, it seems to be an active cause of harm in certain Christian communities. I mean, didn’t Bob Jones University assert that interracial dating on their campus was not “good” up until like 2001, when the U.S. government put their tax-exempt status on the line? I remember those dudes, they published a lot of my homeschool textbooks. 
A practical example that has personal importance to me is how the SDA Church is dealing right now with an internal controversy over whether women should be permitted to be ordained as ministers. Most members I speak to have a similar moral view as you do, and so when approaching this issue their central question is always “what does God want?” or “what is God’s will?” The issue of whether these policies that prohibit women from entering the ministry or church leadership are actually harming anyone is beside the point. Harm to humans is not a part of this moral equation.
Perhaps worse still, honest and earnest Adventist believers who claim to base their views on the Bible and the leading of the Holy Spirit come to opposite conclusions about whether God wants women to be permitted in ministry. To me, this illustrates the additional epistemological problem here. How, exactly, can we fallible humans be certain of God’s will in each and every specific context? When I look at Christians, they certainly don’t seem to agree about many of these specific and relevant moral questions that affect our shared society.
Instead, I’d prefer a type of Christianity that asked “is this harmful?” or “is this helpful?” We will not ever have perfect answers to those questions, of course, but at least under a view centered on valuing human well-being the question becomes an empirical one.