Well, yes and no. Sure, you can get hit with an algorithm change or penalty that destroys all your traffic. However, if you have good people who know what they are doing, this is not likely to happen, and if it does, it is easy (in most cases) to get your visits back. Panda and Penguin are another story, but if you get hit by those it is typically not accidental.
Facebook Ads and other social media ad platforms, for example, are pay-per-click platforms that do not fall under the SEM category. Instead of showing your ads to people who are searching for similar content like search ads do, social media sites introduce your product to people who happen to be just browsing through their feeds. These are two very, very different types of online advertising.
At our agency, we work with sites of varying sizes, from very large to quite small, and recently, we have noticed a trend at the enterprise level. These sites aren’t relying as much on Google for traffic any more. Not only are they not relying on Google traffic, but also, the sites are getting less than 10 percent (or slightly more) of their organic traffic from the search giant.