He doesn’t say “the lower cultures” and “the higher cultures”, he says “the lower races” and “the higher races”. Obviously he thinks that the “higher races” have superior culture and technology, and he thinks this is the product of them being “higher races”. It’s difficult to avoid the sense of hierarchy raised by words like “higher” and “lower”. If we spoke like this today, people would rightly call us racist even if we tried to excuse it by saying “But I’m just saying their culture is lower than ours”.
The fact that he attributes the differences in the “races” to natural selection, and argues that “races” can advance or degrade through natural selection, also shows that when he speaks of “higher” and “lower” races, he doesn’t simply mean “races which for some reason we can’t understand, have higher culture” and “races which for some reason we can’t understand, have lower culture”.
Not curious given his scientific beliefs. He believed the best and brightest Englishmen had moved to the Americas, and that this provided an excellent population on which natural selection worked to produce a superior race to their origin, just as he identified the Western nations as superior to their “savage progenitors”.