Stack of code

Improving product analytics with Elasticsearch

It’s an exciting time to be a developer. Concepts and traditions that used to be firmly in the domain of software development, like Agile and the underlying principles of growth hacking, have made their way into operations and management workflows. In our little corner of the economy (the ‘agency space’), I’ve noticed what we offer has become equal parts product development and service delivery.

While it’s true we are providing creative and development services, the way we execute our deliverables and evaluate our performance is actually more reminiscent of a product development lifecycle. We collect requirements, strategize on possible solutions we can deliver, actualize these solutions, and document the internals in what is effectively a vehicle owner’s manual, but personalized for your own website/software solution. One need not worship at the altar of growth hacking to understand the value in systematically measuring the performance of a product’s features and, even more importantly, reacting to the evidence with new fixes, features, and offerings.

Gartner’s IT glossary contends that product analytics is:

a specialized application of business intelligence (BI) and analytical software that consumes service reports, product returns, warranties, customer feedback and data from embedded sensors to help manufacturers evaluate product defects, identify opportunities for product improvements, detect patterns in usage or capacity of products, and link all these factors to customers.

In this modern information economy, products are no longer just tangible, manufactured items; product analytics has evolved in practice, yet the spirit of the definition remains true.

One platform embodying this new age information economy is Elasticsearch by Elastic — an open-source search and analytics engine based on Apache’s Lucene that has taken the software world by storm.

Elastic Search

The unique value of Elasticsearch is the ability to handle both structured and unstructured data simultaneously. With this platform, documents are indexed according to a strategy defined by the user and the complementary tools in the ELK stack–like Logstash (parse and perform ETL from a variety of sources), Beats (lightweight data shippers sending system logs to Elasticsearch clusters), and Kibana (extensible data visualization dashboard)–provide an end-to-end solution for establishing an analytics pipeline.

Due to its versatility, the ELK stack has been leveraged for a wide variety of projects. Anything you can imagine, from server monitoring, to social media analysis of Game of Thrones tweets, is within the realm of possibility with this platform.

Server Log Breakdown

Server log monitoring to improve performance, preempt attacks, and prevent downtime (courtesy of Saavius)

 

Game of Thrones Analytics

Analysis of Game of Thrones tweets (courtesy of Dzone.com)

You may be thinking “so what?!,” but any product analytics team worth their salt knows the value in the following capabilities:

1. UX improvements: Is our UI and IA conducive to users performing our ‘critical event’ (“an action that users take within your product that aligns closely with your core value proposition” Source: Amplitude’s Product Analytics Playbook)?

2. Infrastructure analysis and auditing: Are we equipped to deliver the best possible experience to users? Why or why not? (Amazon Web Services originated from a tremendous capability to scale infrastructure during the holiday season, a result likely due to an investment in enhanced logging & analytics)

3. Engagement and retention: How effectively are we moving users through the conversion funnel? What holes exist in the funnel? How are our ads performing in concert with our other initiatives?

4. Operational: How can we prioritize improvements, endeavors, and requirements resulting from 1-3? Which changes are most critical?

With the ELK stack at your disposal, the possibilities are endless. Consider for a moment all the logs generated by your product: the Apache and MySQL logs on your server, the system.log files on all your organization’s machines, the WiFi & Bluetooth logs on your IoT devices, even the commerce.log file chronicling all the App/iTunes Store interactions taking place on your MacBook. With a little creative engineering, these logs can paint a comprehensive picture of the state-of-affairs at any level of your organization, network, or product.

The ELK stack empowers organizations to filter, segment, and aggregate information to their hearts content. Truly forward-thinking enterprises, like Netflix, eBay, Facebook and Verizon, have already adopted the open-source platform. We believe your organization (regardless of size; after all, it’s a FREE software!) would be wise to follow suit. Let’s talk about how analytics can help you move the needle.

Joe Haaga, Former Full Stack Developer at Grafik
Joseph Haaga

Related Articles

How to go cookieless in 2024: What marketers need to know
Beginning the first week of 2024, Google is set to being phasing out third-party cookies. With cookied inventory accounting for 78% or more of programmatic ad buys, the post-cookie era will bring about a seismic…
Read More
How brands are using AI and more to deck out social feeds for the holidays
As the holiday season approaches, businesses are gearing up for the annual online shopping frenzy. With consumers increasingly turning to the digital realm for their holiday purchases, social media advertising has become a crucial tool…
Read More
INBOUND 2023: The brave new world of the AI boom
Earlier this fall, several members of the Grafik marketing team attended INBOUND, a cutting-edge marketing and technology conference hosted by Hubspot. The event was jam-packed with insights, innovation, and inspiration, with two dominant topics that,…
Read More
Why website accessibility should be part of your DE&I agenda
In the previous segment of our Web Accessibility Standards series, we delved into the significance of website accessibility and its profound impact on user experience. In this installment, we’ll dig deeper and explore a set…
Read More