People joining my game without an invite or even being accepted

So for whatever reason I've been having people join my game randomly without an invite or anything else. I've had my game set to private except for when i'm trading with other player through poe trade but that's about it. That was through sending and accepting invites. I'm curious if i'm doing something wrong here. I thought the way the game worked was that you or the person joining sends an invite and the other party responds respectively? Did this change or am I doing something wrong?

Thanks for any help.

Are you advertising anything on the board?

If you join a public party and don’t leave it, the party is still public. While you’re in any party, all party members gain access to your open instances. So unless you make new instances after leaving the party, other people can get to them.