No, not every random person can join your event. Only those people who have your "Meeting ID" or "link to join meeting" can join your event.
Whenever, new guest books you event or you add new guest to your event, an email will be sent providing details of the event and "Link to Join" the event. This link to join the event will be unique and a guest won't be able to share it with any other person.
Once you selects "Click Here to Join" option, it will redirect you to the Zoom event.