A/B testing is live right now, but we’re labeling it beta — the UI is not as refined as we’d like it. Please give copious feedback
@daniel was wondering about the way it presents the improvement from A to B. For instance, we had a change that is a 20% improvement but it says it is "slightly better" and shows it as something like a 1.7% increase (i.e., 1.7% out of 100% possible conversion, even though it's 20% better than it's counterpart).
@thinktapwork oh I see. The difference is in absolute percent points right now, I might want to change that
@daniel I think so. I'm used to seeing it relative in other tools. But so far looks really nice and is much better than calculating manually.
Once you've got it all worked out would love to be able to add additional samples (if we're testing 4 scenarios, for instance).
Hi we're TelemetryDeck, a privacy-oriented company that makes app analytics with privacy built-in. We're exploring the fediverse!
@daniel Did you A/B test the A/B testing feature before rolling out? 🙃