I think this is going out today: A/B Testing for everyone who uses @telemetrydeck
A/B testing is live right now, but we’re labeling it beta — the UI is not as refined as we’d like it. Please give copious feedback 🩵
@daniel was wondering about the way it presents the improvement from A to B. For instance, we had a change that is a 20% improvement but it says it is "slightly better" and shows it as something like a 1.7% increase (i.e., 1.7% out of 100% possible conversion, even though it's 20% better than it's counterpart).
Follow
@thinktapwork oh I see. The difference is in absolute percent points right now, I might want to change that
@thinktapwork relative makes much more sense! I’ll add that to the list, thanks! 😊
@daniel I think so. I'm used to seeing it relative in other tools. But so far looks really nice and is much better than calculating manually.
Once you've got it all worked out would love to be able to add additional samples (if we're testing 4 scenarios, for instance).