You can make your server private by enabling the whitelist with the command /whitelist on, and then add anyone you want to be able to join with /whitelist add <username>. In addition to this you can hide your server with the server visibility option on the panel.