One for the real philosophers. But I would say that the existence of mathematics as part of nature is problematic, because it presupposes universals (such as, most obviously, number), which have no reason to exist under materialism.
The best solution I can think of to that is to place mathematics within the human mind only, the universals being abstractions made by people.
But that in itself raises the regress problem of how a materialistic human mind could conceive such universals de novo anyway. But it also creates issues about the apparent universals of patterns within nature, which looks to follow regular immaterial mathematical laws apart from human cognition.
How does the material universe “know” that 2 + 2 of anything in it must equal 4 (an immaterial rule)? I’m not sure how one could go beyond “that’s just the way it is,” which of course could be an equally good answer to any similar question: “Why are there immaterial minds not reducible to matter?” “Well, that’s just the way it is.” Sure, but materialism with ghosts ceases to be materialism.