I think knowledge is a subset of belief. Some beliefs are held without evidence (beliefs in axioms, or things believed on faith), but beliefs held because of evidence are knowledge.
If the evidence is very weak, we might say we don’t really know it. I’d be fine with saying there is some cut-off where the evidence used to justify a belief is too weak to have the belief constitute knowledge.