There’s a difference between being data informed and being data driven. If you are data driven, you are being lazy by allowing the data to make decisions for you (bad). If you are data informed, you are using the data to help you validate your hypothesis and make informed decisions (good).

Simple example

Let’s say you’re gut says you should move a signup button from the right side of the page to the left. In testing the button on the left, the biggest thing you should be looking for is to ensure that having the button on the left doesn’t completely tank signups. Even if the data tells you that having the button on the right is slightly better, you should trust your gut.

Summing it up

Use your existing knowledge to make a hypothesis. Use A/B testing as a safety mechanism. As long as your change doesn’t sink the ship, trust your gut. Use data to help you make informed decisions, but don’t let data bully you around.