Planning and analyzing my concert attendance with Splunk

This past year I added some additional datasets to the Splunk environment I use to analyze my music: information about tickets that I’ve purchased, and information about upcoming concerts.

Ticket purchase analysis

I started keeping track of the tickets that I’ve purchased over the years, which gave me good insights about ticket fees associated with specific ticket sites and concert promoters.  

Table showing a ticket cost analysis by site, with LiveNation having the largest percentage of fees at 25.7%, eTix second with 24.69%, Ticketfly third with 23.87% (and second-highest volume of ticket purchases, with over $700 in ticket purchases), AXS Guest Services fourth with 22.57%, Eventbrite fifth with 22.39%, and Ticketmaster 7th with 21.42% fees, but highest volume of ticket purchases at over $1616. AXS (probably should have been combined) had 19.6% fees, Ticketweb with 19.11%, Bottom of the Hill with 18%, Schubas with 16.67%, and the physical box office never charges fees!

Based on the data that I’ve accumulated so far, Ticketmaster doesn’t have the highest fees for concert tickets. Instead, Live Nation does. This distinction is relatively meaningless when you realize they’ve been the same company since 2010.

However, the ticket site isn’t the strongest indicator of fees, so I decided to split the data further by promoter to identify if specific promoters had higher fees than others.

Table showing ticket cost analysis by promoter. Relevant data is duplicated in surrounding text. The Independent and Goldenvoice were the most frequently-purchased=from promoters, charging 22% and 20.6% fees respectively.

Based on that data you can see that the one show I went to promoted by AT&T had fee percentages of nearly 37%, and that shows promoted by Live Nation (through their evolution and purchase by Ticketmaster) also had fees around 26%. Shows promoted by independent venues have somewhat higher fees than others, hovering around 25% for 1015 Folsom and Mezzanine, but shows promoted by organizations whose only purpose is promotion tend to have slightly lower fees, such as select entertainment with 18%, Popscene with 16.67%, and KC Turner Presents with 15.57%.

I realized I might want to refine this, so I recalculated this data, limiting it to promoters from which I’ve bought at least two tickets.

Table showing ticket cost analysis by promoter with 2+ shows. Relevant data duplicated in surrounding text.

It’s a much more even spread in this case, ranging from 25% to 11% in fees. However, you can see that the same patterns exist— for the shows I’ve bought tickets to, the independent venues average 22-25% in fees, while dedicated independent promoters are 16% or less in added fees, with corporate promoters like Another Planet, JAM, and Goldenvoice filling the middle of the data ranging from 18% to 22%.

I also attempted to determine how I’m discovering concerts. This data is entirely reliant on my memory, with no other data to back it up, but it’s pretty fascinating to track.

Table showing concert discovery methods. Songkick is first, with 46 concerts discovered via songkick, followed by unknown methods for 32 concerts, a friend for 19 concerts, emails for 14 concerts, concert raptors facebook group for 10 concerts, family for 2, instagram for 2, and searching for 1 show.

It’s clear that Songkick has become a vital service in my concert-going planning, helping me discover 46 shows, and friends and email newsletters from venues helping me stay in the know as well for 19 and 14 shows respectively. Social media contributes as well, with a Facebook community (raptors) and Instagram making appearances with 10 and 2 discoveries respectively.

Concert data from Songkick

Because Songkick is so vital to my concert discovery, I wanted to amplify the information I get from the service. In addition to tracking artists on the site, I wanted to proactively gather information about artists coming to the SF Bay Area and compare that with my listening habits. To do this, I wrote a Songkick alert action in Python to run in Splunk.

Songkick does an excellent job for the artists that I’m already tracking, but there are some artists that I might have just recently discovered but am not yet tracking. To reduce the likelihood of missing fast-approaching concerts for these newly-discovered artists, I set up an alert to look for concerts for artists that I’ve discovered this year and have listened to at least 5 times.

To make sure I’m also catching other artists I care about, I use another alert to call the Songkick API for every artist that is above a calculated threshold. That threshold is based on the average listens for all artists that I’ve seen live, so this search helps me catch approaching concerts for my historical favorite artists.

Also to be honest, I also did this largely so that I could learn how to write an alert action in Splunk software. Alert actions are essentially bits of custom python code that you can dispatch with the results of a search in Splunk. The two alert examples I gave are both saved searches that run every day and update an index. I built a dashboard to visualize the results.

Dashboard showing the results of the songkick alert action in Splunk. There’s a table on the left showing “Upcoming concerts for Artists” and a Songkick logo on the right. Some upcomingi concerts are Elder Island at Cafe du Nord and Gorgon City at the Regency Ballroom.

I wanted to use log data to confirm which artists were being sent to Songkick with my API request, even if no events were returned. To do this I added a logging statement in my Python code for the alert action, and then visualized the log statements (with the help of a lookup to match the artist_mbid with the artist name) to display the artists that had no upcoming concerts at all, or had no SF concerts.

Dashboard showing side-by-side tables, the table on the left showing a list of artists I like but which aren’t on tour, and the one on the right showing artists on tour but with no SF concerts.

For those artists without concerts in the San Francisco Bay Area, I wanted to know where they were going instead, so that I could identify possible travel locations for the future.

Table showing a list of locations for specific artists, to identify where they are on tour. The first artist, George Fitzgerald, is on tour in Europe, visiting Amsterdam, Dour, Istanbul, London, Manchester, Marrakesh, Nijmegen, Nottingham, Oxford, Waterford, and Zadar.

It seems like Paris is the place to be for several of these artists—there might be a festival that LAUER, Max Cooper, George Fitzgerald, and Gerald Toto are all playing at, or they just happen to all be visiting that city on their tours.

I’m planning to publish a more detailed blog post about the alert action code in the future on the Splunk blogs site, but until then I’ll be off looking up concert tickets to these upcoming shows….