We’ve complicated infinity as a concept, by definitively pointing out that it never ends. Of course it’s so hard to conceive of ‘it’ never ending, when ‘it’ is nothing.
Infinity is a description of a specific unending pattern. Infinity doesn’t have to amount to one unending pattern, combined of all unending patterns. That doesn’t make sense, because that couldn’t amount to a whole unending pattern. Such a situation would actually be finite because at some point, isolated unending patterns would cease interacting with other infinite unending patterns…and so the whole ends.