The biggest differences between the two lenses can be seen at 200mm and f/2.8, but that is where many of us use a 70-200 f/2.8 the most. If you ever plan on using a 2x extender with your 70-200 f/2.8, the difference between the two lenses at 400mm are remarkable. In the end, everyone has to decide for themselves what the relationship is between "close enough" and "good enough", and how much the difference, if there is any, is worth.
From Bryan Carthanan's review of the Tamron 70-200mm f/2.8 SP Di VC USD. The review is very comprehensive and includes a lot of comparisons between the two lenses mentioned in your question, as well as the current Sigma 70-200 f/2.8 offering.
You buy a 70-200 f/2.8 to use at f/2.8. Otherwise, buy a Canon EF 70-200mm f/4 L IS USM Lens. It will give you modestly better image quality in a smaller and lighter package – with a lower price tag. Thus, I feel that for most, the Tamron VC is a better choice than the Sigma OS from an image sharpness perspective.
The Canon 70-200 L IS II is perhaps the best zoom lens I've ever used – and represents a very formidable competitor to any lens in this class. Surprising is that, at its lower price point, the Tamron 70-200 VC delivers image quality very close to the Canon IS II until the long end of the focal length range where the Canon easily bests the Tamron. Before deciding that you are going to give up that Canon advantage for the lower price, remember that we tend to most-use the full extents of the focal length range in our zoom lenses. And 200mm is perhaps the most important focal length in a 70-200mm f/2.8 stabilized lens. The better 200mm image quality is also translated into better with-extender image quality at the 200mm focal length setting.
