10 tips for better product analytics

Disclaimer: I am no expert in product analytics and definitely not claiming to be. I did spend the last 5 years playing with a lot of different analytical tools and made a lot of mistakes. With this post I hope other people can avoid some of those.

While most startup books and marketing series talk about being 'data driven', I've come to understand that it's easier said than done.

I've done my fair share of product analytics and I have found myself trying to convince people that the only way the project could be successful if we would measure and evaluate it. But that isn't always that simple.

In the last 10 years, I've built my own tracking stack, became a segment.com product expert and have used over 30 integrations from the segment.com catalog.

Here are some things I learned to get better at (product) analytics:

Let me know if you have any other important tips I can learn from.

Define what you want to report on

This is the mistake I made the most:

Look at all these cool/powerful reports we can make

Before you think about tracking or product analytics, try to define the thing you want to measure as simple as possible. KISS. I recommend to formulate it in a question format:

Bad: How much better is our new account activation wizard?
Good: How long does it take on average to go from step 1 to step 5 of the account activation wizard?

Bad: Can you create a dashboard of the features each customer uses?
Good: How many of our active customers (non-trial) have used feature X at least once in the last month?

Defining the question up front (one of the key concepts of Metabase) works well to keep it simple.

Use a tracking standard (analytics.js)

When we started out we had different integration code for Mixpanel, Customer.io and other platforms we wanted to send data to. Which meant that we had to maintain those tracking calls separately.

Customer.io Tracking call - Mixpanel Tracking call

Note that the more integrations you use (we currently use 6) the more code you need to maintain. Abstracting this all in one tracking language/standard was the main reason for switching to segment (using analytics.js) at that time.

The analytics library (available in open-source) will translate that payload into the format your integrations require.

Pick good event names

Install a consistent naming scheme for your events. We stick with the following rules:

Human Readable
Split words by spaces
[Entity] [Action]

Which leads to events like:

Recipient Added
Story Published
Story Removed
Contact Tag Applied

Don't use event names if you can use event traits

Regardless of which integrations you have enabled there is always a limit on the number of events you can use. Intercom has a default limit of 120 events so at some point you're going to hit limits.

That's why you'll find yourself balancing when to use a dedicated event vs adding more context in the tracking payload.

An example:

But that decision also depends on which integrations you have enabled. While Intercom only allows setting up automation based on the event name, other tools like Customer.io are much more powerful.

Mixpanel allows you to use that tracking payload to create Custom Events.

Don't count on page() calls

Most tracking libraries (as well as analytics.js) will start tracking data right away based on page paths or routing changes (SPA libs). That means that you immediately have interesting data to use.

Example Page() calls in Segment debugger

I learned that this data is oftentimes not that trustworthy or useful:

Route changes (changing urls) mess up historic reports
Often lacks context (tracking properties)
Variations of content (non-canonical pages) need extra thought

That's why today we're not tracking or using any page() calls in the app anymore but rather use our own events.

Choose tools that support group()

Most SaaS businesses have the concept of one account that is linked multiple user accounts. When we started using product analytics all events and tracking properties could only be linked to the user entity and thus also every report was tied to a user.

To work around this we found ourselves adding grouping tracking properties to the user analytics payload:

These properties allowed us then to create reports for all trials` or grouping certain reports by company...

Introducing group()

There is a better way to do it and that's using group() calls which is much more inline with most SaaS businesses` user/grouping model.

This way we're splitting up user and company properties in their own calls:

Identify() properties → Traits/properties that belong to a user
Group() properties → Tying the user to a group + traits/properties that belong to a company

These group() calls were first introduced in 2013 and while most integrations have caught up make sure to double check that your integration supports group() calls. It makes product analytics so much easier!

Mix server and client side calls

You can't fully count on client side tracking calls for important reports. That is because:

Users might have ad blocking browser/extensions
Something might go wrong with the tracking call (CORS, network issue, JS error...)

That's why we're using a mix of server side and client side calls. Important calls like 'Trial Account Activated' are triggered both on the server and the client to make sure that any automations also trigger for people that don't want tracking.

Document your tracking schema

I have yet to see a company with a clean tracking plan. As time goes by people add tracking properties, rename certain traits or create events that are not named properly. What about 'test' events?

That's why I'd suggest creating some kind of tracking document (Mixpanel has the tracking lexicon, segment has tracking plans and protocols) to keep track of your events and user/company traits.

Bonus points for keeping some kind of status to deprecate/clean up old events and getting everyone to use this!

Teach your team analytics

For any business to embrace data driven decision-making, your team needs to understand what's going on and be able to make better calls using data.

Here are some things I do to encourage people to use data:

Create a document explaining analytics at your company
- Include the tools (mixpanel, amplitude) they can use
- Highlight which data is where
- Link to the tracking schema
Give everyone full access to segment/analytics tools
Set up dashboards/reports for your team and teach them how to use it
Ask for success criteria when defining/starting projects
Argue using Data. Participate in conversations linking to a report/chart/data to back you up.

Even if you do all of the above, it's not a guarantee that people in the company will take data as serious as you want them to.

Integrate tracking in your stack

While most analytics platforms require you to load their javascript/sdk on your page I strongly recommend making the tracking components part of your build pipeline. This will bundle all analytics interactions as part of the main application and will be much more reliable. Analytics v2 has an npm package you can import in your project.

In combination with that, you can proxy your analytics calls. This will turn third-party tracking calls (to external domains) to a first-party XHR call similar to how most applications interact with their API making it practically impossible to block by browsers or blocking extensions.

Because we're doing an extensive feature audit I am going to write more about the different ways we use product analytics to make roadmap decisions. In absence of a subscribe box you can follow-me on twitter for updates.

Preview image by Isaac Smith on Unsplash