2. Web Analytics and Data Warehousing

In the world of product management, Web Analytics and Data Warehousing play a crucial role in understanding user behavior, measuring product performance, and making data-driven decisions. This document explores the concepts of web analytics and data warehousing, focusing on their practical application in product management, particularly in the context of GetNinjas.

Web Analytics for Granular Insights

Web analytics involves collecting and analyzing data from websites and mobile applications to gain insights into user behavior. At GetNinjas, the team uses web analytics to answer very specific, granular questions that go beyond high-level strategic metrics like Pirate Metrics. For instance, when analyzing a landing page for a technical service, the product team may ask questions like:

By asking these detailed questions, product teams can better understand the performance of each page and the various factors affecting conversion. This level of detail allows the team to identify patterns that may not be evident when looking at high-level averages.

The Importance of Segmentation

Segmentation is critical in web analytics. When managing thousands of combinations of categories and cities, as GetNinjas does, looking at average conversion rates can lead to misguided conclusions. For example, if the average conversion rate is 10%, this could mask significant differences between various segments, such as mobile versus desktop or specific cities versus others.

The ability to segment data by factors such as marketing channel, device type, page type, user behavior (new versus returning users), and load time is essential for identifying insights. For example, the conversion behavior of new users is often significantly different from that of returning users, and looking at the average behavior of all users would obscure these differences. By analyzing segmented data, product teams can identify opportunities to improve performance for specific user groups.

Moving Beyond Aggregated Data

A key principle in web analytics is that aggregated data is often misleading. As Avinash Kaushik famously said, "All aggregated data is crap." It fails to capture the nuances of user behavior and can prevent teams from identifying actionable insights. For instance, by separating new and returning users into distinct curves, teams can see clear differences in conversion behavior that would not be visible in aggregated data.

By analyzing the funnel (the steps users take before completing a desired action) and looking at how it changes over time, product teams can identify points of friction and track the effectiveness of product improvements. Additionally, focusing on the timeline of individual users can reveal patterns in their behavior, such as actions taken before downloading an app or purchasing a plan. These insights can inform product adjustments, communication strategies, and marketing efforts.

Limitations of Standard Tools

While tools like Google Analytics are widely used for web analytics, they have limitations when it comes to handling complex questions and custom segmentations. Some of the challenges include:

Data Warehousing with Snowplow

To overcome the limitations of standard web analytics tools, GetNinjas adopted Snowplow, an open-source platform for web and product analytics. Snowplow enables the team to:

  1. Track user behavior: Monitor what users are doing across the web and app environments, and define custom events that are relevant to the business.
  2. Store raw data in a data warehouse: Save every line of data in its raw form, making it accessible for detailed analysis.
  3. Analyze data in advanced tools: Use tools like Tableau, R, Python, and Looker to analyze and visualize the data.

Snowplow provides flexibility and control over the data, allowing the team to answer complex questions and build a holistic view of the customer journey across platforms.

Centralized Data Warehouse

A data warehouse serves as the central repository for all user data collected from various sources, including web and app analytics, server-side events, transactional data, and integrations with external tools like Salesforce and Intercom. At GetNinjas, the data warehouse is built on Amazon Redshift, and it processes over 500 million events per month.

This centralized system allows the team to analyze user behavior holistically, integrating data from multiple sources to gain a comprehensive understanding of each user’s interactions with the platform. Having all data in one place enables the creation of detailed user profiles and a full 360-degree view of their behavior.

Data Modeling for Business Logic

Each night, a data modeling process is run to apply business logic to the raw data in the warehouse. This process aggregates and transforms the data into more actionable insights, creating tables that analysts can use to track key metrics and identify opportunities. This approach ensures that the data reflects the specific needs and logic of the business, rather than relying on generic analytics models.

Tools for Data Analysis

Two main tools used for data analysis at GetNinjas are:

Conclusion

In product management, web analytics and data warehousing are essential for understanding user behavior and making data-driven decisions. By combining these tools with a centralized data warehouse, product teams can answer complex questions, track user journeys across devices, and apply business-specific logic to their analyses. With the right tools and processes in place, web analytics and data warehousing provide a foundation for deeper insights, more effective product development, and better decision-making.