Here is how whitelisting works in Minecraft. The command /whitelist does nothing on its own, you must type something after it.
List of /whitelist syntax:
/whitelist on/off: Turns the while list on or off
/whitelist add/remove: Adds or removes players from the whitelist
/whitelist list: Lists all players on the whitelist
Once the whitelist is on, players who are not on the whitelist will not be able to join. Keep in mind that if a player was already connected, they will not be kicked, just unable to join back once they leave/get kicked.