Whitelisting
First turn on the whitelist with the command /whitelist on. Then you can add all your friends by typing /whitelist add <username>. Once they're whitelisted, they can join the server.
Blacklisting
To "blacklist" or remove someone from your server, ban them by typing /ban <username>. Once you have banned them, they will not be able to join your server unless you revoke the ban (/pardon <username>).
Hope this helps!