My 2018 Year in Music: Data Analysis and Insights

This past year has been pretty eventful in music for me. I’ve attended a couple new festivals, seen shows while traveling, and discovered plenty of new bands. I want to examine the data available to me and contrast it with my memories of the past year.

I’ve been using Splunk to analyze my music data for the past couple years. You can learn more about what I’ve learned from that in the past in my other posts, see Reflecting on a Decade of Quantified Music Listening and Best of 2017: Newly-Discovered Music. I also wrote a blog post for the Splunk blog (I work there) about this too: 10 Years of Listens: Analyzing My Music Data with Splunk.

Comparing Spotify’s Data with Mine

Spotify released its #2018wrapped campaign recently, sharing highlights from the year of my listening data with me (and in an ad campaign, aggregate data from all the users). As someone that uses Spotify but not as my exclusive source of music listening, I was curious to compare the results with my holistic dataset that I’ve compiled in Splunk. 

Top Artists are Poolside, The Blaze, Justice, Born Ruffians, and Bob Moses. Top Songs are Beautiful Rain, For the Birds, Miss You, Faces, and Heaven. I listened for 30.473 minutes, and my top genre was Indie.

Spotify’s top artists for me were somewhat different from the results that I found from the data I gather from Last.fm and analyze with Splunk software.  Spotify and my holistic listening data agree that I listened to Poolside more than anyone else, and was also a big fan of Born Ruffians, but beyond that they differ. This is probably due to the fact that I bought music and when I’m mobile I switch my primary listening out of Spotify to song files stored on my phone. 

Table showing my top artists and their listens, Poolside with 162 listens, The Vaccines with 136, Young Fathers with 124, Born Ruffians with 102 and Mumford and Sons with 99 listens.

In addition, my top 5 songs of the year were completely different from those listed in Spotify. My holistic top 5 songs of the year were all songs that I purchased. I don’t listen to music exclusively in Spotify, and my favorites go beyond what the service can recognize.

Table showing top songs and the corresponding artist and listen count for the song. Border Girl by Young Fathers with 35 was first, followed by Era by Hubert Kirchner with 32, Naive by the xx with 29, Sun (Viceroy Remix) by Two Door Cinema Club with 27 and There Will Be Time by Mumford & Sons with Baaba Maal also with 27 listens.

Spotify identified that I’ve listened to 30,473 minutes of music, but I can’t make a similarly reliable calculation with my existing data because I don’t have track length data for all the music that I’ve listened to. I can calculate the number of track listens so far this year, and based on that, make an approximation based on the track length data that I do have from my iTunes library. The minute calculation I can make indicates that I’ve so far spent 21,577 minutes listening to 3,878 of the 10,301 total listens I’ve accumulated so far this year (Numbers to change literally as this post is being written).

Screen capture showing total listens of 10,301 and total minutes listened to itunes library songs as 21,577 minutes.

I’m similarly lacking data allowing me to determine my top genre of the year, but Indie is a pretty reliable genre for my taste. 

Other Insights from 2018

I was able to calculate my Top 10 artists, songs, and albums of the year, and drill down on the top 10 artists to see additional data about them (if it existed) in my iTunes library, like other tracks, the date it was added, as well as the kind of file (helping me identify if it was purchased or not), and the length of the track.

Screen capture displaying top 10 artists, top 10 songs, top 10 albums of the year, with the artist Hubert Kirchner selected in the top 10 song list, with additional metadata about songs by Hubert Kirchner listed in a table below the top 10 lists, showing 3 songs by Hubert Kirchner along with the album, genre, rating, date_added, Kind, and track_length for the songs. Other highlights described in text.

There are quite a few common threads across the top 10 artists, songs, and albums, with Poolside, Young Fathers, Gilligan Moss, The Vaccines, and Justice making consistent appearances. The top 10 songs display obsessions with particular songs that outweigh an aggregate popularity for the entire album, leading other songs to be the top albums of the year.

Interestingly, the Polo & Pan album makes my top 10 albums while they don’t make it to my top 10 artist or song lists. This is also true for the album Dancehall by The Blaze. I’m not much of an album listener usually, but I know I listened to those albums several times.

The top 10 song list is more dominated by specific songs that caught my attention, and the top 10 artists neatly reflect both lists. The artists that have a bit more of a back catalog also reveal themselves, given that Born Ruffians managed to crack the top 10 despite not having any songs or albums make the top 10 lists, and Hey Rosetta! makes the top artist and album lists, despite having no top songs.

Screen capture that says Songs Purchased in 2018. 285 songs.

I purchased 285 songs this year, an increase of 157 compared to the year before. I think I just bought songs more quickly after first hearing them this year, and there are even some songs missing from this list that I bought on Beatport or Bandcamp because they weren’t available in the iTunes Store. While I caved in to Spotify premium this year, I still kept up an old promise to myself to buy music (rather than acquire it without paying for it, from a library or questionable download mechanisms) now that I can afford it. 

A Year of Concerts

Screen capture of 4 single value data points, followed by 2 bar charts. Single value data points are total spent on concerts attended in 2018 ($1835.04), total concerts in 2018 (48), artists seen in concert in 2018 (116 artists), and total spent on concert tickets in 2018 ($2109). The first bar chart shows the number of concerts attended per month, 2 in January, 3 in February, 2 in March, 6 in April, 4 in May, 2 in June, 3 in July, 8 in August, 4 in September, 6 in October, 5 in November, and 3 so far in December. The last bar chart is the number of artists seen by month: 5 in Jan, 10 in Feb, 3 in March, 14 in April, 8 in May, 3 in June, 8 in July, 18 in August, 9 in Sep, 22 in Oct, 10 in Nov, 6 in December.

I’ve been to a lot of concerts so far this year. 48, to be exact. I spent a lot of money on concert tickets, both for the shows I attended this year and for shows that went on sale during 2018 (but at this point, might be happening in 2019). I often will buy tickets for multiple people, so this number isn’t very precise for my own personal ticket usage.

I managed to go to at least 2 concerts every month. By the time the year is over, I’m on track to go to 51 different shows. Based on the statistics, there are some months where I went to many more than 1 show per week, and others where I didn’t. Especially apparent are the months with festivals—February, August, and October all included festivals that I attended. 

Many of those festivals brought me to new-to-me locations, with the Noise Pop Block Party and Golden Gate Park giving me new perspectives on familiar places, and Lollapalooza after shows bringing me out to Schubas Tavern for the first time in Chicago.  

Screen capture listing venues visited for the first time in 2018, with venue, city, state, and date listed. Notable ones mentioned in text, full list of venue names: Audio, The New Parish, San Francisco Belle, Schubas Tavern, Golden Gate Park, August Hall, Noise Pop Block Party, Bergerac, Great American Music Hall, Cafe du Nord, Swedish American Hall.

If you’re reading this wondering what San Francisco Belle is, it’s a boat. That’s one of several new venues that electronic music brought me to—DJ sets on that boat as part of Goldroom and Gigamesh’s tour, plus a day party in Bergerac and a nighttime set at Audio other times throughout the year.

Some of those new venue locations brought newly-discovered music to me as well.

Screen capture showing top 20 artists discovered in 2018, sorted by count of listens, featuring a sparkline to show how frequently I listened to the artist throughout the year, and a first_discovered date. List: Gilligan Moss, The Blaze, Polo & Pan, Hubert Kirchner, Keita Sano, Jude Woodhead, Ben Böhmer, Karizma, Luxxury, SuperParka, Chris Malinchak, Mumford & Sons and Baaba Maal, Jon Hopkins, Yon Yonson,  Brandyn Burnette and dwilly, Asgeir, The Heritage Orchestra Jules Buckley and Pete Tong, Confidence Man, Bomba Estereo, and Jenn Champion.

The 20th-most-popular artist I discovered this year was Jenn Champion, who opened for We Were Promised Jetpacks at their show at the Great American Music Hall. I started writing this assuming that I hadn’t heard Jenn Champion before that night, but apparently I first discovered them on July 9, but the show wasn’t until October 9. 

As it turns out, I listened to what is now my favorite song by Jenn Champion that day in July, likely as part of a Spotify algorithm-driven playlist (judging by the listening neighbors around the same time) but it didn’t stick until I saw them play live months later. The vagaries of playlists that refresh once a week can mean fleeting discoveries that you don’t really absorb.

Screen capture showing Splunk search results of artist, track_name, and time from July 9th. Songs near Jenn Champion's song in time include Mcbaise - Le Paradis Du Cuir, Wolf Alice - Don't Delete the Kisses (Tourist Remix) and Champyons - Roaming in Paris.
Other songs I listened to that day in July

Because of how I can search for things in Splunk, I was also curious to see what others songs I heard when I first discovered Hubert Kirchner, a great house artist.

Songs listened to around the same time as I first heard Hubert Kirchner's song Era.... I listened to Dion's song Dream Lover, Deradoorian's song You Carry the Dead (Hidden Cat Remix) followed by Hubert Kirchner, then listened to Miguel's song Sure Thing, How to Dress Well with What You Wanted, then listen to Rihanna, Love on the Brain, Selena Gomez with Bad Liar, and Descendents with I'm the One. I have no idea how I got into this mix of songs.

I have really no idea what playlist I was listening to that might have led to me making jumps from Sofi Tukker, to Tanlines, to Dion, to Deradoorian, then to Hubert Kirchner, Miguel, How to Dress Well, Rihanna, Selena Gomez, and Descendents. Given that August 24th was a Friday, my best guess is perhaps that it was a Release Radar playlist, or perhaps an epic shuffle session. 

Repeat of earlier screen capture showing top 20 artists discovered in 2018. Sorted by count of listens, featuring a sparkline to show how frequently I listened to the artist throughout the year, and a first_discovered date. List: Gilligan Moss, The Blaze, Polo & Pan, Hubert Kirchner, Keita Sano, Jude Woodhead, Ben Böhmer, Karizma, Luxxury, SuperParka, Chris Malinchak, Mumford & Sons and Baaba Maal, Jon Hopkins, Yon Yonson,  Brandyn Burnette and dwilly, Asgeir, The Heritage Orchestra Jules Buckley and Pete Tong, Confidence Man, Bomba Estereo, and Jenn Champion

For the top 20 bands I discovered in 2018, many of them I started listening to on Spotify, but not necessarily because of Spotify. Gilligan Moss was a discovery from a collaborative playlist shared with those that are also in a Facebook group about concert-going. I later saw them at one of the festivals I went to this year, and it even turned out that a friend knew one of the band members! Their status as my most-listened-to discovery of this year is very accurate.

 Polo & Pan was a discovery from a friend, fully brought to life with a playlist built by Polo & Pan themselves and shared on Spotify. Spent some quality time sitting in a park listening to that playlist and just enjoying life. They were at the same festival as Gilligan Moss, playing the same day, making that day a standout of my concerts this year.

Karizma was a discovery from Jamie xx’s set at Outside Lands. I tracked down the song from the set with the help of several other people on the internet (not necessarily anyone I knew) and then the song that was from the set itself wasn’t even on Spotify itself (Spotify, however, did help me discover more of the artist’s back catalog, like my other favorite song ‘Nuffin Else) Apparently I was far behind the curve hearing the song from the set, since it came out in 2017 and was featured in a Chromebook ad, but Work It Out still made me lose my mind at that set. (For the record, so did Take Me Higher, a song I did not manage to track down at all, and have so much thanks for the person that messaged me on Facebook ages later to send me the link!)

Similarly, Luxxury was a DJ I first spotted on a cruise that I went on because it featured other DJs I had heard of from college, Goldroom and Gigamesh, whom I’d discovered through remixes of songs I downloaded from mp3 blogs like The Burning Ear.

~ Finding Meaning in the Platforms ~

Many of these discoveries were deepened by Spotify, or had Spotify as a vector—through a collaborative playlist, algorithmically-generated one, or the quick back-catalog access for a new artist—but don’t rely on Spotify as a platform. I prefer to keep my music listening habits platform-adjacent. 

Spotify, SoundCloud, iTunes, Beatport and other music platforms I use help make my music experiences possible. But the artists making the music, performing live in venues that I have the privilege to live near and afford to visit, they are creating what keep my mind alive and energized.

The social platforms too, mediate the music-related experiences I’ve had, whether it’s with the people I share music and concert experiences with in a Facebook group, the people I exchange tracks and banter with in Slack channels, or those of you reading this on yet another platform. 

I like to listen to music that moves me, physically, or that arrests my mind and takes me somewhere. More now than ever I realize that musical enjoyment for me is an intense instantiation of the continuous tension-and-release pattern that exists in so many human art forms. The waves of neatness that clash and collide in a house music track, or the soaring crescendos of harmonies. 

It’s become clear to me over the years that I can’t separate my enjoyment of music from the platforms that bring me closer to it. Perhaps supporting the platforms in addition to the musical artists, performers, and venues, is just another element of contributing to a thriving music scene.

Politeness in Virtual Assistant Design

The wave of chatbots and virtual assistants like Cortana, Siri, and Alexa means that we’re engaging in conversations with non-humans more than ever before. Problem is, those non-human conversations can turn inhuman when it comes to social norms.

Interactions with virtual assistants aren’t totally devoid of human interaction. Indeed, they often disguise a true human interaction. Many chatbots aren’t fully automated and rely on humans to pick up the slack from the code. More fully-constructed virtual assistants like you find in Amazon’s Echo or your Apple iPhone are carefully programmed by humans. The programming choices they make also define your interactions with the personalities—and these interactions can redefine how you treat people.

A clear indication that someone is truly polite and kind is treating service people with respect, patience, and kindness. The rise of chatbots and virtual assistants, however, means that you’re never quite sure whether you’re speaking to a human. You might think that people can easily tell the difference between when they’re interacting with humans and when they’re interacting with a voice inside a smart box, but as the technology behind virtual assistants like Google Assistant, Amazon Alexa, or used by call centers evolves, that will get harder to evaluate. (Even when you’re calling a call center, it can be hard to tell whether you’ve reached a well-programmed intake bot or a real person who’s fully in the groove of their phone voice).

I find it fascinating (and saddening) that the programmers of Google Assistant’s Duplex chose to program in “umms” and “mmhmms” and did not program in any kindness indicators. Instead the voices come across as impatient and slightly condescending. I listened to the sample clips linked by Ethan Marcotte in his post Kumiho, about Google Duplex. If virtual assistants don’t include programmed kindness, the emotional labor performed by service workers will continue to be too high. 

Programming to add kindness from virtual assistants is important, but so too is programming virtual assistants to expect kindness. We’re starting to be conditioned to treat chatbots as recipients for code-like commands, requiring a specific set of inputs, and those inputs do not acknowledge politeness.

It may seem overly-prescriptive, but in the same way that parents withhold items from their children until they “ask for it nicely”, it might be practical to include a “politeness mode” in virtual assistants. Hunter Walk wrote about how Amazon Alexa interactions are affecting his child, and Ben Hammersley blogged about the fact that there is no reward for politeness when he interacts with Amazon Alexa:

But there’s the rub. Alexa doesn’t acknowledge my thanks. There’s no banter, no trill of mutual appreciation, no silly little, “it is you who must be thanked” line. She just sits there sullenly, silently, ignoring my pleasantries.

And this is starting to feel weird, and makes me wonder if there’s an uncanny valley for politeness. Not one based on listening comprehension, or natural language parsing, but one based on the little rituals of social interaction. If I ask a person, say, what the weather is going to be, and they answer, I thank them, and they reply back to that thanks, and we part happy. If I ask Alexa what the weather is, and thank her, she ignores my thanks. I feel, insanely but even so, snubbed. Or worse, that I’ve snubbed her.”

“It’s the computing equivilent of being rude to waitresses. We shouldn’t allow it, and certainly not by lack of design. Worries about toddler screen time are nothing, compared to future worries about not inadvertently teaching your child to be rude to robots.

As virtual assistants become more common in day-to-day interactions, if they do not account for politeness, we might become a less kind society. Not only that, but impolite virtual assistants will add to the emotional labor performed by the service workers that don’t find their jobs replaced by technology.

Rediscovering Me and Moving Forward

After a breakup, how do you rediscover the activities that you enjoy and make you you?

if someone
does not want me
it is not the end of the world.
but
if i do not want me
the world is nothing but endings.
nayyirah waheed

For myself, I spent several years in a relationship where I slowly let my own needs, wants, and desires be subsumed by those of my partner’s, and what I anticipated to be his needs, wants, and desires of me. Explicitly and implicitly, I lost myself in becoming who (I thought) he wanted me to be. After we broke up I was left with a profoundly distant sense of self. The last time I’d felt truly myself I was living at home (and that wasn’t a strong confident self). I was nothing like the person I became… or was I?

What followed has been an attempt to rediscover a sense of self and a sense of strength. I retried things I’d enjoyed with my partner in different contexts, and with different people (alone or with new friends), to derive new meaning. I needed to know if I truly enjoyed these activities or if I was only doing them because of him.

Something simple like making a bucket list helped me make real what I care about. Why would I want to go to one place instead of another? What sorts of things do I want to put on my list, activity and location-wise? How do I prioritize myself enough to get to go to those places and do those things? This also helps me tap into the sense of freedom and unpredictability in life, but in an ordered way (because that’s how I roll) that helps me discover my “true self”.

A bucket list also helped me think through shared goals, hopes, or dreams. How can I let go of a dream, or hold onto it, knowing that they might still hold that dream too? How can I travel to certain places without being reminded of them and a future I thought we’d share? How can I separate my dreams from those that we shared, created and dreamed together? Maybe I can’t. But that doesn’t mean I have to give them up. I can assess them, and see if I want to keep those goals, hopes, and dreams in my new life.

I worked to find comfort and strength in art and poetry. I asked a friend of mine for some poems about “living your best life”. I wanted some spiritual salve to learn how to remake myself after the relationship ended. She sent me poems like “My Dead Friends” by Marie Howe, “What the Living Do” by Marie Howe, and “The Journey” by Mary Oliver. I went to art museums, lingering amidst the modern art from Germany, a longtime favorite.

I also revisited things from before we started dating. I had to test things that I once cared about (to see if they still mattered to me). I’d neglected them or moved on from them or never gave myself the chance to fully commit to them. For me that was things like climbing, and going to concerts, or out dancing. I’ll attend my 100th show next month, and I recently got back from the Flash Foxy women’s climbing festival. I found myself again in the familiar experiences of going to shows, and in the community of climbing.

The crux of this process has been learning to feel like myself and like I know myself again.

 

Moving Forward

With a stronger sense of self, I’ve started dating again. This is hard. (All things involving people are hard). This has led me to think a lot about what makes people compatible, and what qualities are important and which ones cannot be compromised on.

I saw author Kim Culbertson speak at a panel at the Bay Area Book Festival, and she said “Lots of people are uncomfortable when they hold themselves up to another person and the edges don’t match.”

That’s a lot of what dating feels like (that’s a lot of what talking to other humans feels like, honestly). One inclination to ease that discomfort is to disengage—this person is different from me, so I won’t talk to them (or share much of myself with them), or befriend or date them. Another way to ease that discomfort is to soften my own edges so that the mismatched edges are less apparent. And that’s where this essay comes in.

I had to re-sharpen the edges that make me me. Now I’m working to remind myself not to soften my own edges, but instead work to find a way to appreciate mismatched edges. I don’t need to find a person that perfectly interlocks with the edges of myself to find joy, happiness, intrigue, and personal growth. I do need to find someone that appreciates my edges (and whose edges I can appreciate).

With that in mind, what does it mean to be compatible with someone? Is it the mutual appreciation of edges, or something else? I think there are various levels of it.

  • The surface level compatibility that provides the initial intrigue—you find each other attractive, there is some chemistry, you started talking about a shared interest.
  • A deeper level compatibility when you share interests or passions. It’s easy when someone shares my music taste, or shares my appreciation for music. It’s harder to appreciate the edges of someone who doesn’t like music as much as I do.
  • The more fundamental, deep levels of compatibility reveal themselves as you get to know someone. You start to learn whether or not your communication styles complement each other or conflict with each other. Maybe you each communicate feelings differently, or miss each other’s “love language” signals. Maybe you want to discuss deep, introspective, existential questions over lunch, and your partner just wants to eat.

The edges of the people I meet and date won’t match up with me perfectly, but part of knowing where my edges are is knowing which edges of mine need to line up with those of someone else, and which ones can be different. (I’m still learning this, and I probably always will be).

That knowledge can help me keep my edges intact as I get to know someone. There is a distinct difference between learning to appreciate or respect the interests and passions of someone else and adopting those interests and passions wholesale for myself. I’m trying them out to see what they’re like. As I experience the interests and passions of others, I might be adding new facets to my edges.

But I’m also learning that it’s okay not to share the same interests and passions as the person I’m dating. It’s enough to appreciate that they have those interests and passions. As a perfectionist, I often try to not just to be perfect, but to be perfect for someone. So I have to take a step back (often) and remind myself that other people are flawed, that I’m also flawed, and that not everyone will appreciate raisins in baked goods, or disco music, or staying up late. And that’s okay.

We’re all human, we all have edges. Keep yours sharp, and admire those of others.

This is the new year

Thinking lately

  • How do you decide to make a big change in life?
  • How do you rediscover what’s important to you?
  • How many concerts in a week is too many?

I’m struggling with the first one, working on the second, and am pretty sure the answer to the third one is “three”.

Reading lately

The American Top 40 chart includes more dance songs, more songs performed by DJs, and significantly more white artists than its counterpart, the Billboard charts.

Shit’s racist. I used to listen to the Ryan Seacrest Top 40 driving between Chicago and Michigan because it was one of the few things that I could listen to consistently along that entire drive on just a few radio stations. It wasn’t exactly quality radio, but it kept me awake.

The Secret Lives of Playlists

The business meets somewhere at the crossroads of public relations and payola—a tradition as old as the music industry itself, historically used to define the illegal practice of record companies paying for commercial radio airtime. (Under U.S. law and FCC regulations, Payola is illegal on radio, but those laws do not apply to digital streaming platforms.) According to a 2015 Billboard article, a major-label marketing executive confirmed that pay-for-play is (or was) definitely happening.“According to a source, the price can range from $2,000 for a playlist with tens of thousands of fans to $10,000 for the more well-followed playlists.” And many are already calling the platform’s new “Sponsored Songs” endeavor a 2017 incarnation of payola.

I keep thinking I’ll get sick of Spotify thinkpieces but I’m not there yet. This one covers (in part) how Spotify structures their service to prioritize playlists over albums or other artist-created works, instead effectively reinstating payola and creating pay-to-playlists that then earn top billing all throughout the service. Me, I make my own playlists most of the time.

Can anyone turn streaming music into a real business?

Everyone wants streaming music to be cheap or free for listeners, offer every song ever recorded, be made available on every device, be consistently lucrative for the industry, and give new and established artists robust support for new music. We all want snow that isn’t cold or wet. In principle, everyone is willing to pay, and everyone is willing to compromise, but no one is willing to compromise enough.

Womp womp. This is why for all of my use and support of services like Spotify and SoundCloud, now that I can afford it, I’m trying to buy the music that matters to me when possible. Less likely to disappear that way.

Within The Context Of All Contexts: The Rewiring Of Our Relationship To Music​

Old music, reframed or brought into new circulation, can be as dynamic and unpredictable as new music.

How relying on ~ the algorithms ~ has changed how we encounter music and what that means.

I Used to Insist I Didn’t Get Angry. Not Anymore.

Confronting my own aversion to anger asked me to shift from seeing it simply as an emotion to be felt, and toward understanding it as a tool to be used: part of a well-stocked arsenal.

Leslie Jamison is one of my favorite essayists, and this is no exception.

Writing Lately

I wrote two posts about analyzing my personal music data corpus. Reflecting on a decade of (quantified) music listening fits in with the rest of my blog posts about music, taking the personal tack to the quantified side of things. I also wrote up how I did all the analysis for my company blog, 10 Years of Listens: Analyzing My Music Data with Splunk. I’ve done some more analyses since these posts, like building something that lets me review the listening patterns for a specific artist compared with the dates that I’ve seen them in concert, and I’m working on analyzing if there is an average listen threshold before I see a band in concert (or not).

I also wrote about the importance that climbing has had in my life over the last year and a half in Finding Myself on the Wall. Grateful to get back on the wall tomorrow.

I took the time last year to start converting a dormant side project into a blogging series to share the links I’d collected. Calling it Borders on the Web, I post reminders of the borders that do exist on the web, as much as the techno-utopians in the world might like to pretend that they’re going away.

Listening Lately

The trend in the last year or so toward more disco vibes has been… unexpectedly awesome. Going to see at least three of these artists live in the next few months… hoping to see more music from Thunder Jackson and Disco Despair soon too.

Some great DJ sets / mixtapes on here too. Seeing the xx live last year was a highlight, almost entirely because of Jamie xx. Realized that’s a show I’d pay more than I’d like to admit to go see if it were just him DJing. Haven’t managed to see Alex Cruz yet, though he’s been in the city a couple times since I’ve been here.

Happy 2018, everyone. Feel free to follow me on Twitter if you don’t mind the occasional youtube artifact retweet.

Unexpectedly ccTLDs

Some countries have trendy ccTLDs, and startups buy in to their domain space. Vox Media has more details:

Even very small countries get ccTLDs. Here’s a close-up of the area around Australia and the many small island nations that have their own domain names. Some of these countries realized that they could make a lot of money if they opened their domains to foreigners. The result: popular websites like last.fm (.fm is the domain of the Federated States of Micronesia) and twitch.tv (.tv is the domain for the island nation of Tuvalu). The .io domain, assigned to the British Indian Ocean Territory, has become popular among programmers. They associate the domain with the technical term input/output and use it to create “artisinal websites.”

Best of 2017: Newly-Discovered Music

I used my music data to look up my favorite artists that I discovered in 2017. These are the ones that are the memorable favorites, beyond the statistical favorites.

Pional

This one is a surprise but a good reminder that small obsessions can make a big difference in overall statistics. I have The Burning Ear to thank for this discovery, and Spotify for entertaining it.

Song recommendation:

R.Lum.R

I discovered this artist because they’re touring as the headliner with Gibbz, who I was already familiar with. The groovy vibe of this artist took those tickets from a probable insta-purchase to an actual insta-purchase.

Song recommendation:

Jason Gaffner

A discovery thanks to The Burning Ear, I discovered Jason Gaffner’s nu-disco grooves around the same time that I got obsessed with some songs by Gibbz (who I must’ve discovered in 2016). I bought this song soon after and am keeping an eye out for new releases.

Song recommendation:

Alex Cruz

I heard Alex Cruz for the first time when I was in Greece, listening to a set that my friend started playing. It took me three tries to figure out who she was talking about, and then I discovered a few of his sets that he puts out as the Deep and Sexy Podcast.

Song recommendation:

Perfume Genius

I can’t remember if I started listening to Perfume Genius because of Discover Weekly or the Song Exploder podcast, but damn they’re good. My only regret is that I discovered them too late to get tickets to their sold out show.

Song recommendation:

Super Duper

I don’t remember how I discovered this artist. I think it was an autoplay on SoundCloud after listening to some tracks The Burning Ear had posted? Either way, I fell in love with this remix.

Song recommendation:

Shallou

I came across this band on The Burning Ear too. I think they’ll be around for Noise Pop next year so I’ll have to decide if I want to go see them. I’m mostly in love with this song.

Song recommendation:

Sampha

He opened for the xx, so I checked out his Spotify page after I found out he was opening for them. Sweet, sweet grooves.

Song recommendation:

James Barrett

This guy showed up in my Discover Weekly playlist. I really like this song, but didn’t get as into the rest of his songs. Still a damn good song tho.

Song recommendation:

Ella Vos

I enjoyed her song Little Brother so much that I got tickets to see her next year. I’ll be keeping an eye out for new releases from her as well.

Song recommendation:

Less notable discoveries:

Jane

I came across this band on SoundCloud through The Burning Ear again. This song was an easy purchase because it’s so catchy.

Song recommendation:

Bjéar

This artist showed up on my Discover Weekly playlist. Great for fans of Bon Iver.

Song recommendation:

Imad Royal

This was another The Burning Ear discovery, and an easy purchase!

Song recommendation:

The Full List

The full list of 35 artists that had more than 10 listens each, first listened to in 2017:

Artist Listens Tracks
Pional 42
A New Dawn
As Time Was Passing By
Casualty
In Another Room
Invisible / Amenaza
It’s All Over
It’s All Over – John Talabot’s Stripped Refix
Of My Mind
The Way That You Like
Alex Vargas 41
7 Sins
Ashes
Follow You
Giving Up The Ghost
Higher Love
Inclosure
Indivisible
Oh Love, How You Break Me Up
Renegade
Shackled Up
Solid Ground
Sweet Abandon
Warnings
Wear Your Demons Out
Jason Gaffner 34
Feel Something
Feel Something (Garruda Remix)
Losing My Mind
Losing My Mind (3 Monkeyzz Remix)
Murder In The First Degree
Murder In The First Degree (Aristo G Remix)
Phantom
Phantom (Keljet Remix)
When The Sun Goes Down
Sampha 30
(No One Knows Me) Like The Piano
Beneath The Tree
Blood On Me
Happens
Incomplete Kisses
Kora Sings
Plastic 100°C
Reverse Faults
Take Me Inside
Timmy’s Prayer
Too Much
Under
What Shouldn’t I Be?
Kyko 28
Animals
Dive In
Drive
Headlights
Hideaway
Horizon
Mexico
Native
Nature
Pull Me Up
R.Lum.R 25
Be Honest
Be Honest (Attom Remix)
Bleed Into The Water
Close Enough
Frustrated
Frustrated – Russ Macklin Remix
Learn
Love Less
Nothing New
Show Me
Suddenly
Tell Me
Utah 25
02:12
Hail the Underdog
In Slow Motion
Lights Out
Mirrors
No Coast
On the Mountain by the Sea
One Million
People of the Future
SFSG
Still Good
Watercolor
When People Come Together
Young Summer 25
Alright
Alright (Karl Kling Remix)
Blood Love
Echo
Fallout
Old Chunk of Coal
Sons Of Lightning (Super Duper Remix)
Taken
Waves That Rolled You Under (backstroke. Remix)
Ralph 23
Busy Man
Cold to the Touch
Cold to the Touch – Nicolaas Remix
Screenplay
Something More
Tease
This Is Funky
Alex Cruz 21
Haunting – Original Mix
Haunting – Radio Edit
Haunting – Sebastien Radio Edit
Haunting – Sebastien Remix
Haunting [ANR063] – Sebastien Remix
Rubberband – Radio Edit
Shoreline – Extended Mix
Sweet Child
Sweet Child – Club Mix
Sweet Child – Extended
Sweet Child – Original Mix
National Parks 21
Backwards Centaur
Five Hour Winnipeg
Julia
Long Winter
The Plural of Moose Is Moose
Bien 20
Confetti
Crowd Goes Wild
Electric Dream
Flashback
Last Man Standing
Must Be Dreaming
Spinning on Blue
Stars Across the Sky
The Best Part
Perfume Genius 20
Body’s In Trouble – Recorded at Spotify Studios NYC
Choir
Die 4 You
Every Night
Go Ahead
Just Like Love
Otherside
Sides
Slip Away
Slip Away – Recorded at Spotify Studios NYC
Valley
Wreath
Wreath (Kaitlyn Aurelia Smith Remix)
Super Duper 20
Angela
Angela [Thissongissick.com Premiere]
Don’t Worry
Finale (feat. Ruelle)
Finale Ft. Ruelle
Hollow (feat. Quinn Lewis)
Innocence (feat. REMMI)
Innocence (feat. REMMI) (LUCA LUSH Remix) [NEST HQ Premiere]
Innocence (feat. REMMI) (Madeaux Remix) [NEST HQ Premiere]
Innocence Ft. Remmi
Makes The Wind Ft. Remmi & Jung Youth
Makes the Wind (feat. REMMI & Yung Youth)
Never Gets Old (feat. Remmi)
Revival
Second Chances (feat. Louis Johnson)
Undercover Ft. Patrick Baker
Emerson Jay 18
Fake It Slow
Feel Like Gold
LZY Me
Light Out
Move
Perspective
Secret City
Smok
Take Take Take
Tru
War
When It’s Night
Ruby Empress 17
Danseuse De Delphes
Deluca
Escapism Deluxe
Kimono House
Lovelight (JV-30)
Strung Out
The Empress
Ella Vos 16
00000 Million – Recorded at Spotify Studios NYC
Little Brother
White Noise
Majik 16
27
Closer
High
How It Is
It’s Alright
Paralysed
Real – Skeleton Mix
Save Me
Talk to Me
à la mer 16
Abroad ~ Say That You Want It
Abroad ~ Time
Imad Royal 15
Bad 4 U
Bad 4 U – Light House Remix
Down For Whatever (feat. Pell)
Losing It All
Smile
Troubles
Mr Sanka 15
Be Easy
Flight Mode
Flight Mode (Jengi Beats Remix)
Flight Mode (Lauer Remix)
Forever and a Day
Gallon
Gallon (Cassian Remix)
Midnight Air
Midnight Air (JAQ Remix)
Midnight Air – JAQ Remix
Crooked Colours 14
Another Way
Capricious (Benson Remix)
Capricious (Paces Remix)
Come Down
Come Down [Alison Wonderland Remix]
Flow
Flow – Extended Re-Rub
In Your Bones
In Your Bones (Chiefs Remix)
Step
Rex Orange County 14
A Song About Being Sad
BEST FRIEND
Corduroy Dreams
Edition
Green Eyes, Pt. II
Loving Is Easy
Paradise
Uno
Shallou 14
. . . Love
Begin (feat. Wales)
Begin – Recorded at Spotify Studios NYC
Fictions
Friends – Recorded at Spotify Studios NYC
Heights
Heights – Extended Mix
Motion Picture Soundtrack
Slow
You and Me
James Barrett 13
College
Marrow
Rodger
The Metamorphosis
You Used to Remind Me of the Sky
Klyne 13
Break Away (FaltyDL Remix)
Closer
Don’t Stop
Don’t Stop – Boston Bun Remix
Entropy
Lend Me Another Name
Sure Thing – Lxury Remix
Waiting
Wit U
Liv Dawson 13
Hush
Last Time – Live At RAK
Open Your Eyes
Painkiller
Painkiller – Acoustic
Reflection
Searching
Still
Tapestry
bjéar 13
Big Sky
Cold
Firefall
Firefall – Radio Edit
Going to the Sun
Hymn
Nell
Nevada
Tuolumne
Jane 12
Sister
We Don’t Wanna Dance
Sean McVerry 12
Kerosene
Marcy and the Apparition
Motion Picture Films
Natalie
Strangers
Tiger Lily
Charles Fauna 11
Abandon
Hypnosis
Hypnosis – Brothertiger Remix
Liaison
Myth
Restless Child
Ed Tullett 11
Faux
In Cure
Kadabre
Malignant
Posturer
Silver Dive
Maggie Rogers 11
Alaska
Alaska – Sofi Tukker Remix
Alaska – Toby Green Remix
Dog Years
On + Off
Polish Club 11
Able
Beeping
Did Somebody Tell Me
Don’t Fuck Me Over
My House
Shy Girls 11
Arrest Me (Noah Breakfast Remix) [feat. Tei Shi]
Out of Touch (feat. Rome Fortune)
Say You Will
Time After Time
Trivial Motion
Watercolor Dreams
Why I Love

Best of 2017: Live Shows

My favorite shows of 2017. Here’s to more great ones in 2018!

October 27, 2017: DJ Aaron Axelson, Lewis Ofman, Yelle

Rickshaw Stop, San Francisco CA

Popscene became my favorite concert sponsor this year, in no large part because of the skills of their DJs. This show surpassed my low expectations to be a great time of dancing and grooving and new music discoveries.

February 23, 2017: Rad Dad, Gibbz

The Hotel Utah Saloon, San Francisco CA

A local band opened for an undersung nu-disco artist, Gibbz. A great way to open p Noise Pop week 2017, and unexpectedly great sound quality for such a small space. Excited to see Gibbz play again next year.

September 19, 2017: NVDES, RAC

The Independent, San Francisco CA

RAC has put on a spectactularly dance-able show every time I’ve seen them. This most recent adventure did not disappoint.

April 16, 2017: Sampha, The XX

Bill Graham Civic Auditorium, San Francisco CA

I would pay Jamie XX to DJ my life, but I can’t afford it. I could afford this show, though. It was incredible. Sampha was great too. Highlight: a mirror that appeared partway through the set that gave the audience a view of Jamie XX’s DJing and his dorky dance moves.

September 13, 2017: The Dirty Nil, Bleached, Against Me!

Regency Ballroom, San Francisco CA

Just as good as they were 10 years ago when I saw them in Chicago, if not better. A restorative and energetic show.

February 4 2017: Wheatus, Mike Doughty

The Independent, San Francisco CA

Wheatus played old hits and new jams, and Mike Doughty pulled them out to back him as he played a bunch of Soul Coughing songs. I was there more for his solo songs, but the artistry and adventure of his live conducting of the band behind him made for an incredible show that was supremely groove-able.

Best of 2017: Books

The best books I read this year, loosely categorized.

Favorite Book

Uproot: Travels in 21st-Century Music and Digital Culture

Fantastic. Jace Clayton has an unnervingly well-placed finger on the pulse of modern music culture, in a way that makes you feel out-of-touch no matter how much music you listen to. I feel like I understand the music industry, global commerce, music-making, and people around the world better after reading this book. It blends together all those aspects and manages to be writing about music without making you miss the music (but the website for the book has playlists, just in case you do). A personal non-fiction book, a style I turn out to like quite a bit (Word by Word has a similar style).

Beyond Historical Fiction

The Atlas of Forgotten Places

A book picked up at the library on a whim turned out to be one of my favorite books of the year. Tying history with stories of personal struggle and tragedy, this doesn’t tie up neatly and doesn’t come across as try-hard either. A reminder of reality in a novel.

The Nightingale

This story follows two sisters in World War II through their wartime decisions and the present-day. Not quite as brilliant as “All The Light We Cannot See” but just as moving.

Manhattan Beach

Egan’s research shows in the vividness of the storytelling and the mental imagery constructed. You can feel the weight of the decisions made by the characters and their physical burdens in the novel.

The Three-Body Problem

I didn’t manage to finish the trilogy, but this novel stunningly takes the prospect of alien contact and puts it in context of Communist China, with some perspective from competing American, and Russian global interests too. Reading it the same year as Arrival (Stories of Your Life and Others) leads to echoes of similar themes, but the approach is so vastly different that I only thought of the comparison in writing this, not in reading the novel. This book is solidly sci-fi, but the role of history seemed so relevant to the story that I’m categorizing it here.

The Japanese Lover

The first I’ve read by Isabel Allende, and a love story hidden inside a story about the Japanese internment during World War II and the havoc it wreaked on families, alongside the present-day immigrant experience in the United States.

Pleasant Feel-Good Discoveries

The Hating Game

A brand new author on the romance novel scene wrote this and it is delightful. Doesn’t rely over-much on existing romance novel tropes, and manages to be well-written even while you’re rolling your eyes occasionally. Depicts the internal struggle that prevents many of us from believing something is real all too well.

The Royal We

This may be loosely Kate and William fanfic, but I. Am. Here. For. This. It made the rounds at book swap this year and I maintain that it took the classic trope of “ordinary person meets royal but doesn’t know they’re royal” and makes it unexpected and a delight.

Morgan Matson novels

The Sarah Dessen novels of a new age, with less tragic character backstories. Enjoyable discoveries for this year, and I’m looking forward to her next one due out next year.

Eligible

Curtis Sittenfeld is a delight. I didn’t realize this was a Pride and Prejudice rewrite until the end, and that made me like it more. An enjoyable read that helps you realize just how much of modern romance fiction is based on the tropes (first?) established in Pride and Prejudice.

Proper Literature or Vague Classics

The Unwomanly Face of War

This was devastating. A vivid window into the reality and the legacy of women who fought for or worked for the Soviet Union in World War II, her work manages to be both a record of history and an critical eye cast toward the Soviet government. Just as unrelenting as Voices of Chernobyl. I am inclined to seek out all of her work.

Stories of Your Life and Others

The first eight stories were great. Skip the rest of the collection. Perfect for the overly-analytical people that try to analyze rather than experience their emotions. The film Arrival was based on one of these stories.

On Immunity

A beautiful book of personal essays interwoven with research. Brings the human back to science and medicine. Also swapping this book at book swap led to my first encounter with “the first page” and my friends’ desire to have me read the first page of books aloud for a podcast.

Snow Crash

Finally read this novel and it has stayed vividly with me over the past few months since reading it. A clear precursor to so many novels that followed it, and a great reminder that what is online is never truly only online.

Fantasy

Graceling Realm series

Court of Thorns and Roses series

Six of Crows duology

I grouped these three series together because they handled in varying degrees:

  • Mind control and/or a race of superpowered/magical people
  • Romance (from hints at beginnings of love, to explicit seduction)
  • Warring states and the steps that those embroiled among them must take to win power
  • Redemption of the self in the face of personal insecurities

The Graceling series was the best of these three, I’d wager. I read the third and the second books in the wrong order on accident, and might prefer that order instead of the intended order. That could be because I’m a less attentive reader than some.

For a focus on heist and revenge adventures, read the Six of Crows duology. Not much of a romance thread through these books, it focuses more on coming of age and learning what matters.

For the most romance, make it through the near-insufferable first book of the Court of Thorns and Roses series and follow it through to the end of the third book (then reconsider rereading the first book). The next few books that aren’t out yet are spinoffs, so if you, like me, have a rule about not starting series before they end, never fear. This series has the most similar tropes to the Graceling series, so consider reading them far apart.

Reflecting on a decade of (quantified) music listening

I recently crossed the 10 year mark of using Last.fm to track what I listen to.

From the first tape I owned (Train’s Drops of Jupiter) to the first CD (Cat Stevens Classics) to the first album I discovered by roaming the stacks at the public library (The Most Serene Republic Underwater Cinematographer) to the college radio station that shaped my adolescent music taste (WONC) to the college radio station that shaped my college experience (WESN), to the shift from tapes, to CDs, (and a radio walkman all the while), to the radio in my car, to SoundCloud and MP3 music blogs, to Grooveshark and later Spotify, with Windows Media Player and later an iTunes music library keeping me company throughout…. It’s been quite a journey.

Some, but not all, of that journey has been captured while using the service Last.fm for the last 10 years. Last.fm “scrobbles” what you listen to as you listen to it, keeping a record of your listening habits and behaviors. I decided to add all this data to Splunk, along with my iTunes library and a list of concerts I’ve attended over the years, to quantify my music listening, acquisition, and attendance habits. Let’s go.

What am I doing?

Before I get any data in, I have to know what questions I’m trying to answer, otherwise I won’t get the right data into Splunk (my data analysis system of choice, because I work there). Even if I get the right data into Splunk, I have to make sure that the right fields are there to do the analysis that I wanted. This helped me prioritize certain scripts over others to retrieve and clean my data (because I can’t code well enough to write my own).

I also made a list of the questions that I wanted to answer with my data, and coded the questions according to the types of data that I would need to answer the questions. Things like:

  • What percentage of the songs in iTunes have I listened to?
  • What is my artist distribution over time? Do I listen to more artists now? Different ones overall?
  • What is my listen count over time?
  • What genres are my favorite?
  • How have my top 10 artists shifted year over year?
  • How do my listening habits shift around a concert? Do I listen to that artist more, or not at all?
  • What songs did I listen to a lot a few years ago, but not since?
  • What personal one hit wonders do I have, where I listen to one song by an artist way more than any other of their songs?
  • What songs do I listen to that are in Spotify but not in iTunes (that I should buy, perhaps)?
  • How many listens does each service have? Do I have a service bias?
  • How many songs are in multiple services, implying that I’ve probably bought them?
  • What’s the lag between the date a song or album was released and my first listen?
  • What geographic locations are my favorite artists from?

As the list goes on, the questions get more complex and require an increasing number of data sources. So I prioritized what was simplest to start, and started getting data in.

 

Getting data in…

I knew I wanted as much music data as I could get into the system. However, SoundCloud isn’t providing developer API keys at the moment, and Spotify requires authentication, which is a little bit beyond my skills at the moment. MusicBrainz also has a lot of great data, but has intense rate-limiting so I knew I’d want a strategy to approach that metadata-gathering data source. I was left with three initial data sources: my iTunes library, my own list of concerts I’ve gone to, and my Last.fm account data.

Last.fm provides an endpoint that allows you to get the recent tracks played by a user, which was exactly what I wanted to analyze. I started by building an add-on for Last.fm with the Splunk Add-on Builder to call this REST endpoint. It was hard. When I first tried to do this a year and a half ago, the add-on builder didn’t yet support checkpointing, so I could only pull in data if I was actively listening and Splunk was on. Because I had installed Splunk on a laptop rather than a server in ~ the cloud ~, I was pretty limited in the data I could pull in. I pretty much abandoned the process until checkpointing was supported.

After the add-on builder started supporting checkpointing, I set it up again, but ran into issues. Everything from forgetting to specify the from date in my REST call to JSON path decision-making that meant I was limited in the number of results I could pull back at a time. I deleted the data from the add-on sourcetype many times, triple-checking the results each time before continuing.

I used a python script (thanks Reddit) to pull my historical data from Last.fm to add to Splunk, and to fill the gap between this initial backfill and the time it took me to get the add-on working, I used an NPM module. When you don’t know how to code, you’re at the mercy of the tools other people have developed. Adding the backfill data to Splunk also meant I had to adjust the max_days_ago default in props.conf, because Splunk doesn’t necessarily expect data from 10+ years ago by default. 2 scripts in 2 languages and 1 add-on builder later, I had a working solution and my Last.fm data in Splunk.

To get the iTunes data in, I used an iTunes to CSV script on Github (thanks StackExchange) to convert the library.xml file into CSV. This worked great, but again, it was in a language I don’t know (Ruby) and so I was at the mercy of a kind developer posting scripts on Github again. I was limited to whatever fields their script supported. This again only did backfill.

I’m still trying to sort out the regex and determine if it’s possible to parse the iTunes Library.xml file in its entirety and add it to Splunk without too much of a headache, and/or get it set up so that I can ad-hoc add new songs added to the library to Splunk without converting the entries some other way. Work in progress, but I’m pretty close to getting that working thanks to help from some regex gurus in the Splunk community.

For the concert data, I added the data I had into the Lookup File Editor app and was up and running. Because of some column header choices I made for how to organize my data, and the fact that I chose to maintain a lookup rather than add the information as events, I was up for some more adventures in search, but this data format made it easy to add new concerts as I attend them.

Answer these questions…with data!

I built a lot of dashboard panels. I wanted to answer the questions I mentioned earlier, along with some others. I was spurred on by my brother recommending a song to me to listen to. I was pretty sure I’d heard the song before, and decided to use data to verify it.

Screen image of a chart showing the earliest listens of tracks by the band VHS collection.

I’d first heard the song he recommended to me, Waiting on the Summer, in March. Hipster credibility: intact. Having this dashboard panel now lets me answer the questions “when was the first time I listened to an artist, and which songs did I hear first?”. I added a second panel later, to compare the earliest listens with the play counts of songs by the artist. Maybe the first song I’d heard by an artist was the most listened song, but often not.

Another question I wanted to answer was “how many concerts have I been to, and what’s the distribution in my concert attendance?”

Screen image showing concerts attended over time, with peaks in 2010 and 2017.

It’s pretty fun to look at this chart. I went to a few concerts while I was in high school, but never more than one a month and rarely more than a few per year. The pace picked up while I was in college, especially while I was dating someone that liked going to concerts. A slowdown as I studied abroad and finished college, then it picks up for a year as I get settled in a new town. But after I get settled in a long-term relationship, my concert attendance drops off, to where I’m going to fewer shows than I did in high school. As soon as I’m single again, that shifts dramatically and now I’m going to 1 or more show a month. The personal stories and patterns revealed by the data are the fun part for me.

I answered some more questions, especially those that could be answered by fun graphs, such as what states have my concentrated music listens?

Screen image of a map of the contiguous united states, with Illinois highlighted in dark blue, indicating 40+ concerts attended in that state, California highlighted in a paler blue indicating 20ish shows attended there, followed by Michigan in paler blue, and finally Ohio, Wisconsin, and Missouri in very pale blue. The rest of the states are white, indicating no shows attended in those states.

It’s easy to tell where I’ve spent most of my life living so far, but again the personal details tell a bigger story. I spent more time in Michigan than I have lived in California so far, but I’ve spent more time single in California so far, thus attending more concerts.

Speaking of California, I also wanted to see what my most-listened-to songs were since moving to California. I used a trellis visualization to split the songs by artist, allowing me to identify artists that were more popular with me than others.

Screen image showing a "trellis" visualization of top songs since moving to California. Notable songs are Carly Rae Jepsen "Run Away With Me" and Ariana Grande "Into You" and CHVRCHES with their songs High Enough to Carry You Over and Clearest Blue and Leave a Trace.

I really liked the CHVRCHES album Every Open Eye, so I have three songs from that album. I also spent some time with a four song playlist featuring Adele’s song Send My Love (To Your New Lover), Ariana Grande’s Into You, Carly Rae Jepsen’s Run Away With Me, and Ingrid Michaelson’s song Hell No. Somehow two breakup songs and two love songs were the perfect juxtaposition for a great playlist. I liked it enough to where all four songs are in this list (though only half of it is visible in this screenshot). That’s another secret behind the data.

I also wanted to do some more analytics on my concert data, and decided to figure out what my favorite venues were. I had some guesses, but wanted to see what the data said.

Screen image of most visited concert venues, with The Metro in Chicago taking the top spot with 6 visits, followed by First Midwest Bank Ampitheatre (5 visits), Fox Theater, Mezzanine, Regency Ballroom, The Greek Theatre, and The Independent with 3 visits each.

The Metro is my favorite venue in Chicago, so it’s no surprise that it came in first in the rankings (I also later corrected the data to make it its proper name, “Metro” so that I could drill down from the panel to a Google Maps search for the venue). First Midwest Bank Ampitheatre hosted Warped Tour, which I attended (apparently) 5 times over the years. Since moving to California it seems like I don’t have a favorite venue based on visits alone, but it’s really The Independent, followed by Bill Graham Civic Auditorium, which doesn’t even make this list. Number of visits doesn’t automatically equate to favorite.

But what does it MEAN?

I could do data analysis like that all day. But what else do I learn by just looking at the data itself?

I can tell that Last.fm didn’t handle the shift to mobile and portable devices very well. It thrives when all of your listening happens on your laptop, and it can grab the scrobbles from your iPod or other device when you plug it into your computer. But as soon as internet-connected devices got popular (and I started using them), listens scrobbled overall dropped. In addition to devices, the rise of streaming music on sites like Grooveshark and SoundCloud to replace the shift from MediaFire-hosted and MegaUpload-hosted free music shared on music blogs also meant trouble for my data integrity. Last.fm didn’t handle listens on the web then, and only handles them through a fragile extension now.

Two graphs depicting distinct song listens and distinct artist listens, respectively, with a peak and steady listens through 2008-2012, then it drops down to a trough in 2014 before coming up to half the amount of 2010 and rising slightly.

Distinct songs and artists listened to in Last.fm data.But that’s not the whole story. I also got a job and started working in an environment where I couldn’t listen to music at work, so wasn’t listening to music there, and also wasn’t listening to music at home much either due to other circumstances. Given that the count plummets to near-zero, it’s possible there were also data issues at play.  It’s imperfect, but still fascinating.

What else did I learn?

Screen image showing 5 dashboard panels. Clockwise, the upper left shows a trending indicator of concerts attended per month, displaying 1 for the month of December and a net decrease of 4 from the previous month. The next shows the overall number of concerts attended, 87 shows. The next shows the number of iTunes library songs with no listens: 4272. The second to last shows a pie chart showing that nearly 30% of the songs have 0 listens, 23% have 1 listen, and the rest are a variety of listen counts. The last indicator shows the total number of songs in my iTunes library, or 16202.

I have a lot of songs in my iTunes library. I haven’t listened to nearly 30% of them. I’ve listened to nearly 25% of them only once. That’s the majority of my music library. If I split that by rating, however, it would get a lot more interesting. Soon.

You can’t see the fallout from my own personal Music-ocalypse in this data, because the Library.xml file doesn’t know which songs don’t point to actual files, or at least my version of it doesn’t. I’ll need more high-fidelity data to determine the “actual” size of my library, and perform more analyses.

I need more data in general, and more patience, to perform the analyses to answer the more complex questions I want to answer, like my listening habits of particular artists around a concert. As it is, this is a really exciting start.

If you want more details about the actual Splunking I did to do these analyses, I’ll be posting a blog on the official Splunk blog. That got posted on January 4th! Here it is: 10 Years of Listens: Analyzing My Music Data with Splunk.

Yoga Beta for Climbers

As a companion to Finding Yourself on the Wall, sometimes what you need while climbing isn’t real beta or advice of what to do, but mental reinforcement. This beta can sound kind of like the mantras that someone might give you in the midst of a yoga class—yoga beta.

  • Do what feels right
  • Don’t forget to breathe
  • Don’t look, just feel
  • You are stronger than you think
  • Just let go