ActiveCollab Experiments!
Mar 2, 2020

ActiveCollab Experiments!

Jelena Slatinac Project Management
Experiment [noun] - “A procedure carried out to support, refute, or validate a hypothesis. Experiments provide insight into cause-and-effect by demonstrating what outcome occurs when a particular factor is manipulated.”

Before you even start thinking about performing an experiment, its value must be determined. Why are you conducting it? What do you hope to get out of it?

There are three possible things to experiment:

  • Technology
  • Processes
  • Product

We’ve tried all three!

Yea or nay?

We went to a 2-day seminar, where we talked about using an innovative new tool for developing UI components. A couple of team members have worked on this, presented it, and introduced the rest of us to it. We agreed it could be useful for the development of ActiveCollab and decided to try it out for a few weeks, with no strings attached. 

The goal was to see if the development of features would speed up if our design would become more consistent, and the quality of our code better. In time, it turned out it was quite useful, therefore the experiment was successful, and we kept on using it!


As far as processes are concerned, we felt our teams were quite disconnected among themselves. Everyone was doing their part, but they lacked the clarity on what other teams were working on. So we decided to bring everything together in one place, visually. 

Our Scrum Masters got together and created a Timeline, with duct tape and sticky notes. It showed what every single team was working on, and deadlines by which some tasks needed to be completed, or certain decisions taken.

Finally, some walls were torn down. One could say this experiment was a success as well, but when looking back, we realized everyone could be even more connected. This is why we’re still working on it and trying to find new ways to tear down those walls to the ground.

Last, but not least, product. Since it’s something we already talked about in our previous posts, we’d like to get very particular here.

Prepping for an experiment

An experiment starts by asking a question. In our case, that would be: if we offer ActiveCollab in a user’s native language, will the number of our customers grow?

We assume that our visitors whose native language isn’t English won’t create a trial because they don’t wish to use software whose primary and only language is, in fact, English. Perhaps, if they were informed right from the start that other languages were available as well, the percentage of created trials vs. the number of visits would grow.

Then we do some research, and come up with a hypothesis: following the launch of the experiment, we’ll have an increase of a minimum of 10% in the number of trials created. The difference between the initial question and the hypothesis is: the latter is actionable.

In our case, this means that if the hypothesis proves to be correct, and the number of trials increases following the start of our experiment, we should invest more resources into translating the entire website. We should also consider communicating these options to our customers better, and enhance the localization of the app.

On the contrary, if the numbers tell us there’s no significant change in the number of trials, we’ll know there’s no need to keep investing time and resources in this feature.

Which door to choose?

3, 2, 1… The experiment can start! But what will it look like? We narrowed all the ideas down to three options.

Option n°1. The ActiveCollab website informs its users: “Did you know that you can use AC in more than 20 languages?”, and a button leads to a Help page that explains how to switch to a certain language. The experiment goes on for 7 days.

Option n°2. As part of a remarketing campaign, we display to certain people on certain markets the possibility to translate ActiveCollab. There’s a blog post that explains how, and the banner can lead to it.

Option n°3. We make an A/B test, where we communicate to one group of people ActiveCollab is available in 25 languages. We’ll need a budget, and we’ll decide each week whether to continue with the experiment or not, depending on the results it gives.

It’s important to choose the simplest possible way to test the hypothesis and discover relevant results. Find out in our next post which path we took to reach this goal!

Related Articles

Close