Cut out foods your sensitive to or not??

I love it how people say that you shouldn't completely cut out foods that you are sensitive to because you will increase your sensitivity to them even more. Our bodies are amazing at building up a tolerance to things that we poison ourselves with because it really has no choice when we keep slamming it with irritants. That doesn't mean we should keep consuming them. There are a few other items that share that reaction that I can think of... Alcohol and drugs are two. If you drink a bottle of wine every night you will no doubt be less sensitive to it's affects than someone who has a glass every few months. You may be less sensitive to it but your chances of chronic disease as a result of alcoholism are increased... You can build up a tolerance to cocaine as well, doesn't mean its good for you. Eating foods that you are sensitive to and that are causing damage to your digestive system as a result isn't very logical to me.