Straight Up vs Sexy: Testing Email Subject Lines

Here's what we found:

Right off the bat we noticed that the test had little to no effect on whether or the recipients decided to open the email. 

At first we couldn't believe it.

Seriously? A .5% discrepancy? Is this real life?

Once the initial shock wore off, we revisited the data and redirected our attention to click-to-open rate.

While 10.9% of the people who opened variation A clicked-through to the offer, only 7.8% of the people who opened variation B clicked. 

Aha! It appeared that we were on to something, but what?

While the mismatched subject lines had little influence on a recipient's decision to open the email, it did, in fact, impact their decision to click-through. 

But why?

Here's what you should know:

According to MailChimp, “the best subject lines tell what's inside, and the worst subject lines sell what's inside.”

We concur. 

Our test results revealed that people are more likely to engage with the content inside your email if you clearly set the expectation in the subject line. 

The recipients of variation A knew what they were getting into before they opened the email. The subject line was straight up. 

When they clicked into the email, The Beginner's Guide was there waiting for them, as expected. They clicked. 

The recipients of variation B received something a bit sexier. More actionable. More benefit-driven. Less explicit. 

When they clicked into the email, they were met with an offer that they may or may not have been expecting given the ambiguity of the subject line. They clicked less. 

This doesn't mean that you should flee to your email composer of choice and start blasting out definitive subject lines. Let's not get carried away. 

But it does serve as valuable data to call up when deciding between one subject line or another. 

Again, this is not the end all be all. 

Like any good A/B test, we'll repeat this experiment several times in an effort to replicate the results and ensure validity. 

We encourage you to do the same. The more data-backed email subject line best practices out there the better.

Happy testing!