The problem is, in our modern world, chicken isn't always gluten free, hamburgers are sometimes made with pink slime, and vegetables can be contaminated with chemicals or toxins. It's really pretty upsetting once you start to realize how far food has moved away from its natural state.
As far as I've been able to conclude, no one really knows whether GMO products are completely fine, or contribute to an increased risk of health problems. BPA in bottles and cans probably contributes to an increased risk of cancer, but alternative plastics may not be much better. Chemical pesticides increase the output of crops, contributing to a more cost effective food supply, but also potentially contribute to increased risk of health problems. The only thing I know for sure is it's complicated.
As a philosophy, Paleo is intended to replicate the lifestyle of ancestral humans - people who lived and ate locally based on what they could hunt or forage in their nearby environment. The theory is that without the effects of overfarming or delayed ripening technology, fruits and vegetables contained more nutrients than our modern equivalents. It's also proven that animals who eat traditionally (i.e. cows that eat grass, chickens that hunt bugs, etc.) versus eating grains or other byproducts produce meat and products (milk, eggs) that are higher in beneficial fats (Omega 3's) and nutrients (like K2) than their conventional counterparts.
As you select your food sources, Paleo people suggest buying the highest quality food you can afford. I sort of agree with this, except this would imply that expensive food is more nutritious, and that's not always true (some really expensive stores sell primarily fancy Frankenfood). Rather than simply choosing the most expensive, choose the foods that are most likely to be raised in sustainable, nutritious way. Research the suppliers of your grocery store, or talk with the farmers at the farmers market. Try to eat mostly produce that is in season, and that was growing recently (versus picked months ago and ripened using Delayed Ripening Technology). Eat meats that are sustainably farmed or wild caught. Read labels, even on meat, to make sure that what you think is simple canned tomatoes actually is just tomatoes, versus a mix of tomatoes, chemicals, sugar, salt, etc.
In short, pay attention to what you're eating. Don't buy expensive food just because it's pricey, or buy the cheapest food thinking that it's all the same. Know why something costs more (or less), and understand whether you're paying for better nutrition, or for hype. Familiarize yourself with the sources of your foods. When it's necessary to buy something with a label, read that label, and strive to avoid the ingredients you don't understand (preservatives, coloring, etc.) It's all a continuum, and ultimately we can't (and probably wouldn't want to) actually live in the world of our ancestors, but we can make educated choices about food quality, so we might as well do so!