This is the new year

Thinking lately

  • How do you decide to make a big change in life?
  • How do you rediscover what’s important to you?
  • How many concerts in a week is too many?

I’m struggling with the first one, working on the second, and am pretty sure the answer to the third one is “three”.

Reading lately

The American Top 40 chart includes more dance songs, more songs performed by DJs, and significantly more white artists than its counterpart, the Billboard charts.

Shit’s racist. I used to listen to the Ryan Seacrest Top 40 driving between Chicago and Michigan because it was one of the few things that I could listen to consistently along that entire drive on just a few radio stations. It wasn’t exactly quality radio, but it kept me awake.

The Secret Lives of Playlists

The business meets somewhere at the crossroads of public relations and payola—a tradition as old as the music industry itself, historically used to define the illegal practice of record companies paying for commercial radio airtime. (Under U.S. law and FCC regulations, Payola is illegal on radio, but those laws do not apply to digital streaming platforms.) According to a 2015 Billboard article, a major-label marketing executive confirmed that pay-for-play is (or was) definitely happening.“According to a source, the price can range from $2,000 for a playlist with tens of thousands of fans to $10,000 for the more well-followed playlists.” And many are already calling the platform’s new “Sponsored Songs” endeavor a 2017 incarnation of payola.

I keep thinking I’ll get sick of Spotify thinkpieces but I’m not there yet. This one covers (in part) how Spotify structures their service to prioritize playlists over albums or other artist-created works, instead effectively reinstating payola and creating pay-to-playlists that then earn top billing all throughout the service. Me, I make my own playlists most of the time.

Can anyone turn streaming music into a real business?

Everyone wants streaming music to be cheap or free for listeners, offer every song ever recorded, be made available on every device, be consistently lucrative for the industry, and give new and established artists robust support for new music. We all want snow that isn’t cold or wet. In principle, everyone is willing to pay, and everyone is willing to compromise, but no one is willing to compromise enough.

Womp womp. This is why for all of my use and support of services like Spotify and SoundCloud, now that I can afford it, I’m trying to buy the music that matters to me when possible. Less likely to disappear that way.

Within The Context Of All Contexts: The Rewiring Of Our Relationship To Music​

Old music, reframed or brought into new circulation, can be as dynamic and unpredictable as new music.

How relying on ~ the algorithms ~ has changed how we encounter music and what that means.

I Used to Insist I Didn’t Get Angry. Not Anymore.

Confronting my own aversion to anger asked me to shift from seeing it simply as an emotion to be felt, and toward understanding it as a tool to be used: part of a well-stocked arsenal.

Leslie Jamison is one of my favorite essayists, and this is no exception.

Writing Lately

I wrote two posts about analyzing my personal music data corpus. Reflecting on a decade of (quantified) music listening fits in with the rest of my blog posts about music, taking the personal tack to the quantified side of things. I also wrote up how I did all the analysis for my company blog, 10 Years of Listens: Analyzing My Music Data with Splunk. I’ve done some more analyses since these posts, like building something that lets me review the listening patterns for a specific artist compared with the dates that I’ve seen them in concert, and I’m working on analyzing if there is an average listen threshold before I see a band in concert (or not).

I also wrote about the importance that climbing has had in my life over the last year and a half in Finding Myself on the Wall. Grateful to get back on the wall tomorrow.

I took the time last year to start converting a dormant side project into a blogging series to share the links I’d collected. Calling it Borders on the Web, I post reminders of the borders that do exist on the web, as much as the techno-utopians in the world might like to pretend that they’re going away.

Listening Lately

The trend in the last year or so toward more disco vibes has been… unexpectedly awesome. Going to see at least three of these artists live in the next few months… hoping to see more music from Thunder Jackson and Disco Despair soon too.

Some great DJ sets / mixtapes on here too. Seeing the xx live last year was a highlight, almost entirely because of Jamie xx. Realized that’s a show I’d pay more than I’d like to admit to go see if it were just him DJing. Haven’t managed to see Alex Cruz yet, though he’s been in the city a couple times since I’ve been here.

Happy 2018, everyone. Feel free to follow me on Twitter if you don’t mind the occasional youtube artifact retweet.

Best of 2017: Newly-Discovered Music

I used my music data to look up my favorite artists that I discovered in 2017. These are the ones that are the memorable favorites, beyond the statistical favorites.

Pional

This one is a surprise but a good reminder that small obsessions can make a big difference in overall statistics. I have The Burning Ear to thank for this discovery, and Spotify for entertaining it.

Song recommendation:

R.Lum.R

I discovered this artist because they’re touring as the headliner with Gibbz, who I was already familiar with. The groovy vibe of this artist took those tickets from a probable insta-purchase to an actual insta-purchase.

Song recommendation:

Jason Gaffner

A discovery thanks to The Burning Ear, I discovered Jason Gaffner’s nu-disco grooves around the same time that I got obsessed with some songs by Gibbz (who I must’ve discovered in 2016). I bought this song soon after and am keeping an eye out for new releases.

Song recommendation:

Alex Cruz

I heard Alex Cruz for the first time when I was in Greece, listening to a set that my friend started playing. It took me three tries to figure out who she was talking about, and then I discovered a few of his sets that he puts out as the Deep and Sexy Podcast.

Song recommendation:

Perfume Genius

I can’t remember if I started listening to Perfume Genius because of Discover Weekly or the Song Exploder podcast, but damn they’re good. My only regret is that I discovered them too late to get tickets to their sold out show.

Song recommendation:

Super Duper

I don’t remember how I discovered this artist. I think it was an autoplay on SoundCloud after listening to some tracks The Burning Ear had posted? Either way, I fell in love with this remix.

Song recommendation:

Shallou

I came across this band on The Burning Ear too. I think they’ll be around for Noise Pop next year so I’ll have to decide if I want to go see them. I’m mostly in love with this song.

Song recommendation:

Sampha

He opened for the xx, so I checked out his Spotify page after I found out he was opening for them. Sweet, sweet grooves.

Song recommendation:

James Barrett

This guy showed up in my Discover Weekly playlist. I really like this song, but didn’t get as into the rest of his songs. Still a damn good song tho.

Song recommendation:

Ella Vos

I enjoyed her song Little Brother so much that I got tickets to see her next year. I’ll be keeping an eye out for new releases from her as well.

Song recommendation:

Less notable discoveries:

Jane

I came across this band on SoundCloud through The Burning Ear again. This song was an easy purchase because it’s so catchy.

Song recommendation:

Bjéar

This artist showed up on my Discover Weekly playlist. Great for fans of Bon Iver.

Song recommendation:

Imad Royal

This was another The Burning Ear discovery, and an easy purchase!

Song recommendation:

The Full List

The full list of 35 artists that had more than 10 listens each, first listened to in 2017:

Artist Listens Tracks
Pional 42
A New Dawn
As Time Was Passing By
Casualty
In Another Room
Invisible / Amenaza
It’s All Over
It’s All Over – John Talabot’s Stripped Refix
Of My Mind
The Way That You Like
Alex Vargas 41
7 Sins
Ashes
Follow You
Giving Up The Ghost
Higher Love
Inclosure
Indivisible
Oh Love, How You Break Me Up
Renegade
Shackled Up
Solid Ground
Sweet Abandon
Warnings
Wear Your Demons Out
Jason Gaffner 34
Feel Something
Feel Something (Garruda Remix)
Losing My Mind
Losing My Mind (3 Monkeyzz Remix)
Murder In The First Degree
Murder In The First Degree (Aristo G Remix)
Phantom
Phantom (Keljet Remix)
When The Sun Goes Down
Sampha 30
(No One Knows Me) Like The Piano
Beneath The Tree
Blood On Me
Happens
Incomplete Kisses
Kora Sings
Plastic 100°C
Reverse Faults
Take Me Inside
Timmy’s Prayer
Too Much
Under
What Shouldn’t I Be?
Kyko 28
Animals
Dive In
Drive
Headlights
Hideaway
Horizon
Mexico
Native
Nature
Pull Me Up
R.Lum.R 25
Be Honest
Be Honest (Attom Remix)
Bleed Into The Water
Close Enough
Frustrated
Frustrated – Russ Macklin Remix
Learn
Love Less
Nothing New
Show Me
Suddenly
Tell Me
Utah 25
02:12
Hail the Underdog
In Slow Motion
Lights Out
Mirrors
No Coast
On the Mountain by the Sea
One Million
People of the Future
SFSG
Still Good
Watercolor
When People Come Together
Young Summer 25
Alright
Alright (Karl Kling Remix)
Blood Love
Echo
Fallout
Old Chunk of Coal
Sons Of Lightning (Super Duper Remix)
Taken
Waves That Rolled You Under (backstroke. Remix)
Ralph 23
Busy Man
Cold to the Touch
Cold to the Touch – Nicolaas Remix
Screenplay
Something More
Tease
This Is Funky
Alex Cruz 21
Haunting – Original Mix
Haunting – Radio Edit
Haunting – Sebastien Radio Edit
Haunting – Sebastien Remix
Haunting [ANR063] – Sebastien Remix
Rubberband – Radio Edit
Shoreline – Extended Mix
Sweet Child
Sweet Child – Club Mix
Sweet Child – Extended
Sweet Child – Original Mix
National Parks 21
Backwards Centaur
Five Hour Winnipeg
Julia
Long Winter
The Plural of Moose Is Moose
Bien 20
Confetti
Crowd Goes Wild
Electric Dream
Flashback
Last Man Standing
Must Be Dreaming
Spinning on Blue
Stars Across the Sky
The Best Part
Perfume Genius 20
Body’s In Trouble – Recorded at Spotify Studios NYC
Choir
Die 4 You
Every Night
Go Ahead
Just Like Love
Otherside
Sides
Slip Away
Slip Away – Recorded at Spotify Studios NYC
Valley
Wreath
Wreath (Kaitlyn Aurelia Smith Remix)
Super Duper 20
Angela
Angela [Thissongissick.com Premiere]
Don’t Worry
Finale (feat. Ruelle)
Finale Ft. Ruelle
Hollow (feat. Quinn Lewis)
Innocence (feat. REMMI)
Innocence (feat. REMMI) (LUCA LUSH Remix) [NEST HQ Premiere]
Innocence (feat. REMMI) (Madeaux Remix) [NEST HQ Premiere]
Innocence Ft. Remmi
Makes The Wind Ft. Remmi & Jung Youth
Makes the Wind (feat. REMMI & Yung Youth)
Never Gets Old (feat. Remmi)
Revival
Second Chances (feat. Louis Johnson)
Undercover Ft. Patrick Baker
Emerson Jay 18
Fake It Slow
Feel Like Gold
LZY Me
Light Out
Move
Perspective
Secret City
Smok
Take Take Take
Tru
War
When It’s Night
Ruby Empress 17
Danseuse De Delphes
Deluca
Escapism Deluxe
Kimono House
Lovelight (JV-30)
Strung Out
The Empress
Ella Vos 16
00000 Million – Recorded at Spotify Studios NYC
Little Brother
White Noise
Majik 16
27
Closer
High
How It Is
It’s Alright
Paralysed
Real – Skeleton Mix
Save Me
Talk to Me
à la mer 16
Abroad ~ Say That You Want It
Abroad ~ Time
Imad Royal 15
Bad 4 U
Bad 4 U – Light House Remix
Down For Whatever (feat. Pell)
Losing It All
Smile
Troubles
Mr Sanka 15
Be Easy
Flight Mode
Flight Mode (Jengi Beats Remix)
Flight Mode (Lauer Remix)
Forever and a Day
Gallon
Gallon (Cassian Remix)
Midnight Air
Midnight Air (JAQ Remix)
Midnight Air – JAQ Remix
Crooked Colours 14
Another Way
Capricious (Benson Remix)
Capricious (Paces Remix)
Come Down
Come Down [Alison Wonderland Remix]
Flow
Flow – Extended Re-Rub
In Your Bones
In Your Bones (Chiefs Remix)
Step
Rex Orange County 14
A Song About Being Sad
BEST FRIEND
Corduroy Dreams
Edition
Green Eyes, Pt. II
Loving Is Easy
Paradise
Uno
Shallou 14
. . . Love
Begin (feat. Wales)
Begin – Recorded at Spotify Studios NYC
Fictions
Friends – Recorded at Spotify Studios NYC
Heights
Heights – Extended Mix
Motion Picture Soundtrack
Slow
You and Me
James Barrett 13
College
Marrow
Rodger
The Metamorphosis
You Used to Remind Me of the Sky
Klyne 13
Break Away (FaltyDL Remix)
Closer
Don’t Stop
Don’t Stop – Boston Bun Remix
Entropy
Lend Me Another Name
Sure Thing – Lxury Remix
Waiting
Wit U
Liv Dawson 13
Hush
Last Time – Live At RAK
Open Your Eyes
Painkiller
Painkiller – Acoustic
Reflection
Searching
Still
Tapestry
bjéar 13
Big Sky
Cold
Firefall
Firefall – Radio Edit
Going to the Sun
Hymn
Nell
Nevada
Tuolumne
Jane 12
Sister
We Don’t Wanna Dance
Sean McVerry 12
Kerosene
Marcy and the Apparition
Motion Picture Films
Natalie
Strangers
Tiger Lily
Charles Fauna 11
Abandon
Hypnosis
Hypnosis – Brothertiger Remix
Liaison
Myth
Restless Child
Ed Tullett 11
Faux
In Cure
Kadabre
Malignant
Posturer
Silver Dive
Maggie Rogers 11
Alaska
Alaska – Sofi Tukker Remix
Alaska – Toby Green Remix
Dog Years
On + Off
Polish Club 11
Able
Beeping
Did Somebody Tell Me
Don’t Fuck Me Over
My House
Shy Girls 11
Arrest Me (Noah Breakfast Remix) [feat. Tei Shi]
Out of Touch (feat. Rome Fortune)
Say You Will
Time After Time
Trivial Motion
Watercolor Dreams
Why I Love

Best of 2017: Live Shows

My favorite shows of 2017. Here’s to more great ones in 2018!

October 27, 2017: DJ Aaron Axelson, Lewis Ofman, Yelle

Rickshaw Stop, San Francisco CA

Popscene became my favorite concert sponsor this year, in no large part because of the skills of their DJs. This show surpassed my low expectations to be a great time of dancing and grooving and new music discoveries.

February 23, 2017: Rad Dad, Gibbz

The Hotel Utah Saloon, San Francisco CA

A local band opened for an undersung nu-disco artist, Gibbz. A great way to open p Noise Pop week 2017, and unexpectedly great sound quality for such a small space. Excited to see Gibbz play again next year.

September 19, 2017: NVDES, RAC

The Independent, San Francisco CA

RAC has put on a spectactularly dance-able show every time I’ve seen them. This most recent adventure did not disappoint.

April 16, 2017: Sampha, The XX

Bill Graham Civic Auditorium, San Francisco CA

I would pay Jamie XX to DJ my life, but I can’t afford it. I could afford this show, though. It was incredible. Sampha was great too. Highlight: a mirror that appeared partway through the set that gave the audience a view of Jamie XX’s DJing and his dorky dance moves.

September 13, 2017: The Dirty Nil, Bleached, Against Me!

Regency Ballroom, San Francisco CA

Just as good as they were 10 years ago when I saw them in Chicago, if not better. A restorative and energetic show.

February 4 2017: Wheatus, Mike Doughty

The Independent, San Francisco CA

Wheatus played old hits and new jams, and Mike Doughty pulled them out to back him as he played a bunch of Soul Coughing songs. I was there more for his solo songs, but the artistry and adventure of his live conducting of the band behind him made for an incredible show that was supremely groove-able.

Reflecting on a decade of (quantified) music listening

I recently crossed the 10 year mark of using Last.fm to track what I listen to.

From the first tape I owned (Train’s Drops of Jupiter) to the first CD (Cat Stevens Classics) to the first album I discovered by roaming the stacks at the public library (The Most Serene Republic Underwater Cinematographer) to the college radio station that shaped my adolescent music taste (WONC) to the college radio station that shaped my college experience (WESN), to the shift from tapes, to CDs, (and a radio walkman all the while), to the radio in my car, to SoundCloud and MP3 music blogs, to Grooveshark and later Spotify, with Windows Media Player and later an iTunes music library keeping me company throughout…. It’s been quite a journey.

Some, but not all, of that journey has been captured while using the service Last.fm for the last 10 years. Last.fm “scrobbles” what you listen to as you listen to it, keeping a record of your listening habits and behaviors. I decided to add all this data to Splunk, along with my iTunes library and a list of concerts I’ve attended over the years, to quantify my music listening, acquisition, and attendance habits. Let’s go.

What am I doing?

Before I get any data in, I have to know what questions I’m trying to answer, otherwise I won’t get the right data into Splunk (my data analysis system of choice, because I work there). Even if I get the right data into Splunk, I have to make sure that the right fields are there to do the analysis that I wanted. This helped me prioritize certain scripts over others to retrieve and clean my data (because I can’t code well enough to write my own).

I also made a list of the questions that I wanted to answer with my data, and coded the questions according to the types of data that I would need to answer the questions. Things like:

  • What percentage of the songs in iTunes have I listened to?
  • What is my artist distribution over time? Do I listen to more artists now? Different ones overall?
  • What is my listen count over time?
  • What genres are my favorite?
  • How have my top 10 artists shifted year over year?
  • How do my listening habits shift around a concert? Do I listen to that artist more, or not at all?
  • What songs did I listen to a lot a few years ago, but not since?
  • What personal one hit wonders do I have, where I listen to one song by an artist way more than any other of their songs?
  • What songs do I listen to that are in Spotify but not in iTunes (that I should buy, perhaps)?
  • How many listens does each service have? Do I have a service bias?
  • How many songs are in multiple services, implying that I’ve probably bought them?
  • What’s the lag between the date a song or album was released and my first listen?
  • What geographic locations are my favorite artists from?

As the list goes on, the questions get more complex and require an increasing number of data sources. So I prioritized what was simplest to start, and started getting data in.

 

Getting data in…

I knew I wanted as much music data as I could get into the system. However, SoundCloud isn’t providing developer API keys at the moment, and Spotify requires authentication, which is a little bit beyond my skills at the moment. MusicBrainz also has a lot of great data, but has intense rate-limiting so I knew I’d want a strategy to approach that metadata-gathering data source. I was left with three initial data sources: my iTunes library, my own list of concerts I’ve gone to, and my Last.fm account data.

Last.fm provides an endpoint that allows you to get the recent tracks played by a user, which was exactly what I wanted to analyze. I started by building an add-on for Last.fm with the Splunk Add-on Builder to call this REST endpoint. It was hard. When I first tried to do this a year and a half ago, the add-on builder didn’t yet support checkpointing, so I could only pull in data if I was actively listening and Splunk was on. Because I had installed Splunk on a laptop rather than a server in ~ the cloud ~, I was pretty limited in the data I could pull in. I pretty much abandoned the process until checkpointing was supported.

After the add-on builder started supporting checkpointing, I set it up again, but ran into issues. Everything from forgetting to specify the from date in my REST call to JSON path decision-making that meant I was limited in the number of results I could pull back at a time. I deleted the data from the add-on sourcetype many times, triple-checking the results each time before continuing.

I used a python script (thanks Reddit) to pull my historical data from Last.fm to add to Splunk, and to fill the gap between this initial backfill and the time it took me to get the add-on working, I used an NPM module. When you don’t know how to code, you’re at the mercy of the tools other people have developed. Adding the backfill data to Splunk also meant I had to adjust the max_days_ago default in props.conf, because Splunk doesn’t necessarily expect data from 10+ years ago by default. 2 scripts in 2 languages and 1 add-on builder later, I had a working solution and my Last.fm data in Splunk.

To get the iTunes data in, I used an iTunes to CSV script on Github (thanks StackExchange) to convert the library.xml file into CSV. This worked great, but again, it was in a language I don’t know (Ruby) and so I was at the mercy of a kind developer posting scripts on Github again. I was limited to whatever fields their script supported. This again only did backfill.

I’m still trying to sort out the regex and determine if it’s possible to parse the iTunes Library.xml file in its entirety and add it to Splunk without too much of a headache, and/or get it set up so that I can ad-hoc add new songs added to the library to Splunk without converting the entries some other way. Work in progress, but I’m pretty close to getting that working thanks to help from some regex gurus in the Splunk community.

For the concert data, I added the data I had into the Lookup File Editor app and was up and running. Because of some column header choices I made for how to organize my data, and the fact that I chose to maintain a lookup rather than add the information as events, I was up for some more adventures in search, but this data format made it easy to add new concerts as I attend them.

Answer these questions…with data!

I built a lot of dashboard panels. I wanted to answer the questions I mentioned earlier, along with some others. I was spurred on by my brother recommending a song to me to listen to. I was pretty sure I’d heard the song before, and decided to use data to verify it.

Screen image of a chart showing the earliest listens of tracks by the band VHS collection.

I’d first heard the song he recommended to me, Waiting on the Summer, in March. Hipster credibility: intact. Having this dashboard panel now lets me answer the questions “when was the first time I listened to an artist, and which songs did I hear first?”. I added a second panel later, to compare the earliest listens with the play counts of songs by the artist. Maybe the first song I’d heard by an artist was the most listened song, but often not.

Another question I wanted to answer was “how many concerts have I been to, and what’s the distribution in my concert attendance?”

Screen image showing concerts attended over time, with peaks in 2010 and 2017.

It’s pretty fun to look at this chart. I went to a few concerts while I was in high school, but never more than one a month and rarely more than a few per year. The pace picked up while I was in college, especially while I was dating someone that liked going to concerts. A slowdown as I studied abroad and finished college, then it picks up for a year as I get settled in a new town. But after I get settled in a long-term relationship, my concert attendance drops off, to where I’m going to fewer shows than I did in high school. As soon as I’m single again, that shifts dramatically and now I’m going to 1 or more show a month. The personal stories and patterns revealed by the data are the fun part for me.

I answered some more questions, especially those that could be answered by fun graphs, such as what states have my concentrated music listens?

Screen image of a map of the contiguous united states, with Illinois highlighted in dark blue, indicating 40+ concerts attended in that state, California highlighted in a paler blue indicating 20ish shows attended there, followed by Michigan in paler blue, and finally Ohio, Wisconsin, and Missouri in very pale blue. The rest of the states are white, indicating no shows attended in those states.

It’s easy to tell where I’ve spent most of my life living so far, but again the personal details tell a bigger story. I spent more time in Michigan than I have lived in California so far, but I’ve spent more time single in California so far, thus attending more concerts.

Speaking of California, I also wanted to see what my most-listened-to songs were since moving to California. I used a trellis visualization to split the songs by artist, allowing me to identify artists that were more popular with me than others.

Screen image showing a "trellis" visualization of top songs since moving to California. Notable songs are Carly Rae Jepsen "Run Away With Me" and Ariana Grande "Into You" and CHVRCHES with their songs High Enough to Carry You Over and Clearest Blue and Leave a Trace.

I really liked the CHVRCHES album Every Open Eye, so I have three songs from that album. I also spent some time with a four song playlist featuring Adele’s song Send My Love (To Your New Lover), Ariana Grande’s Into You, Carly Rae Jepsen’s Run Away With Me, and Ingrid Michaelson’s song Hell No. Somehow two breakup songs and two love songs were the perfect juxtaposition for a great playlist. I liked it enough to where all four songs are in this list (though only half of it is visible in this screenshot). That’s another secret behind the data.

I also wanted to do some more analytics on my concert data, and decided to figure out what my favorite venues were. I had some guesses, but wanted to see what the data said.

Screen image of most visited concert venues, with The Metro in Chicago taking the top spot with 6 visits, followed by First Midwest Bank Ampitheatre (5 visits), Fox Theater, Mezzanine, Regency Ballroom, The Greek Theatre, and The Independent with 3 visits each.

The Metro is my favorite venue in Chicago, so it’s no surprise that it came in first in the rankings (I also later corrected the data to make it its proper name, “Metro” so that I could drill down from the panel to a Google Maps search for the venue). First Midwest Bank Ampitheatre hosted Warped Tour, which I attended (apparently) 5 times over the years. Since moving to California it seems like I don’t have a favorite venue based on visits alone, but it’s really The Independent, followed by Bill Graham Civic Auditorium, which doesn’t even make this list. Number of visits doesn’t automatically equate to favorite.

But what does it MEAN?

I could do data analysis like that all day. But what else do I learn by just looking at the data itself?

I can tell that Last.fm didn’t handle the shift to mobile and portable devices very well. It thrives when all of your listening happens on your laptop, and it can grab the scrobbles from your iPod or other device when you plug it into your computer. But as soon as internet-connected devices got popular (and I started using them), listens scrobbled overall dropped. In addition to devices, the rise of streaming music on sites like Grooveshark and SoundCloud to replace the shift from MediaFire-hosted and MegaUpload-hosted free music shared on music blogs also meant trouble for my data integrity. Last.fm didn’t handle listens on the web then, and only handles them through a fragile extension now.

Two graphs depicting distinct song listens and distinct artist listens, respectively, with a peak and steady listens through 2008-2012, then it drops down to a trough in 2014 before coming up to half the amount of 2010 and rising slightly.

Distinct songs and artists listened to in Last.fm data.But that’s not the whole story. I also got a job and started working in an environment where I couldn’t listen to music at work, so wasn’t listening to music there, and also wasn’t listening to music at home much either due to other circumstances. Given that the count plummets to near-zero, it’s possible there were also data issues at play.  It’s imperfect, but still fascinating.

What else did I learn?

Screen image showing 5 dashboard panels. Clockwise, the upper left shows a trending indicator of concerts attended per month, displaying 1 for the month of December and a net decrease of 4 from the previous month. The next shows the overall number of concerts attended, 87 shows. The next shows the number of iTunes library songs with no listens: 4272. The second to last shows a pie chart showing that nearly 30% of the songs have 0 listens, 23% have 1 listen, and the rest are a variety of listen counts. The last indicator shows the total number of songs in my iTunes library, or 16202.

I have a lot of songs in my iTunes library. I haven’t listened to nearly 30% of them. I’ve listened to nearly 25% of them only once. That’s the majority of my music library. If I split that by rating, however, it would get a lot more interesting. Soon.

You can’t see the fallout from my own personal Music-ocalypse in this data, because the Library.xml file doesn’t know which songs don’t point to actual files, or at least my version of it doesn’t. I’ll need more high-fidelity data to determine the “actual” size of my library, and perform more analyses.

I need more data in general, and more patience, to perform the analyses to answer the more complex questions I want to answer, like my listening habits of particular artists around a concert. As it is, this is a really exciting start.

If you want more details about the actual Splunking I did to do these analyses, I’ll be posting a blog on the official Splunk blog. That got posted on January 4th! Here it is: 10 Years of Listens: Analyzing My Music Data with Splunk.

Kill Legacy Apple Software


Benedict Evans pointed out in a recent newsletter, “there’s a story to be written about Apple feeling its way from a piecemeal legacy technology stack for services, evolved bit by bit from the old iPod music store of a decade ago, to an actual new unified platform, something that it is apparently building.”

I’d argue for a focused set of decoupled applications, rather than a new unified platform. iTunes has bloated beyond practicality. The App store doesn’t work well for users or developers. Here’s where I think the future of these applications lies.

Continue reading

Libraries, Digital Advertising, and the Machine Zone

Librarians are an underused, underpaid, and underestimated legion. And one librarian in particular is frustrated by e-book lending. Not just the fact that libraries have to maintain waitlists for access to a digital file, but also that the barriers to checking out an ebook are unnecessarily high. As she puts it,

“Teaching people about having technology serve them includes helping them learn to assess and evaluate risk for themselves.”

In her view,

“Information workers need to be willing to step up and be more honest about how technology really works and not silently carry water for bad systems. People trust us to tell them the truth.”

That seems like the least that can be expected by library patrons.

Continue reading

Torture, Ownership, and Privacy

The Senate Intelligence Committee released hundreds of pages (soon available as a book) detailing acts of torture committed by the CIA.

Continue reading

Taylor Swift and Being Between Stars

Taylor Swift has been blowing up the music industry lately, first by surprising everyone with the beauty of her latest album. SNL dubbed it a result of Swiftamine, and I can certainly say I’m under the spell.

Then, pre-release, she removed her entire discography from Spotify. The Atlantic reflects on this decision by pointing out, “Owning music outright, instead of renting it through a streaming service, would be better for listeners and artists in the long run. Indeed, it would be better for just about everyone except Spotify.”

Continue reading

Reading, Drones, and Georgie Washington

Americans are still reading books, Internet and all! Younger Americans are actually reading more than older generations, which could be partially due to the fact that with the rise of texting and social media, so much of our communication is text-based, so everyone is doing a lot more reading (and writing) in order to communicate with their friends. The original study is linked in that article and in this graph:

What are some other ways to get people to read books?

Well it helps a lot if your college library not only tells you the call numbers of the book, but it gives you precise directions to the location of the book, which is pretty awesome. Much more useful when navigating a giant library, like I have access to at the university I work at, as opposed to the smaller library at the university I actually attended.

Continue reading