Tutorial: Getting Music Data with the Last.fm API using Python

APIs allow us to make requests from servers to retrieve data. APIs are useful for many things, but one is to be able to create a unique dataset for a data science project. In this tutorial, we’re going to learn some advanced techniques for working with the Last.fm API.

In our beginner Python API tutorial, we worked with a simple API that was ideal for teaching the basics:

• It had a few, easy to understand end points.
• Because it didn’t require authentication, we didn’t have to worry about how to tell the API that we had perlesson to use it.
• The data that each end point responded with was small and had an easy-to-understand structure.

In reality, most APIs are more complex than this, and so to work with them you need to understand some more advanced concepts. Specifically, we’re going to learn:

• How to authenticate yourself with an API key.
• How to use rate limiting and other techniques to work within the guidelines of an API.
• How to use pagination to work with large responses.

This tutorial assumes you understand the basics of working with an API in Python. If you don’t, we recommend our beginner API tutorial to start. We also assume you have an intermediate knowledge of Python and pandas. If you don’t, you can start learning for free with our Python Fundamentals course.

Working with the Last.fm API

We’ll be working with the Last.fm API. Last.fm is a music service that builds personal profiles by connecting to music streaming apps like iTunes, Spotify and others like it and keeping track of the music you listen to.

They provide free access to their API so that music services can send them data, but also provide endpoints that summarize all the data that Last.fm has on various artists, songs, and genres. We’ll be building a dataset of popular artists using their API.

Following API Guidelines

When working with APIs, it’s important to follow their guidelines. If you don’t, you can get yourself banned from using the API. Beyond that, especially when a company provides an API for free, it’s good to respect their limitations and guidelines since they’re providing something for nothing.

Looking at the Introduction Page in the API documentation, we can notice a few important guidelines:

Please use an identifiable User-Agent header on all requests. This helps our logging and reduces the risk of you getting banned.

When you make a request to the last.fm API, you can identify yourself using headers. Last.fm wants us to specify a user-agent in the header so they know who we are. We’ll learn how to do that when we make our first request in a moment.

Use common sense when deciding how many calls to make. For example, if you’re making a web application, try not to hit the API on page load. Your account may be suspended if your application is continuously making several calls per second.

In order to build our data set, we’re going to need to make thousands of requests to the Last.fm API. While they don’t provide a specific limit in their documentation, they do advise that we shouldn’t be continuously making many calls per second. In this tutorial we’re going to learn a few strategies for rate limiting, or making sure we don’t hit their API too much, so we can avoid getting banned.

Before we make our first request, we need learn how to authenticate with the Last.fm API

Authenticating with API Keys

The majority of APIs require you to authenticate yourself so they know you have perlesson to use them. One of the most common forms of authentication is to use an API Key, which is like a password for using their API. If you don’t provide an API key when making a request, you will get an error.

The process for using an API key works like this:

1. You create an account with the provider of the API.
2. You request an API key, which is usually a long string like 54686973206973206d7920415049204b6579 .
3. You record your API key somewhere safe, like a password keeper. If someone gets your API key, they can use the API pretending to be you.
4. Every time you make a request, you provide the API key to authenticate yourself.

To get an API key for Last.fm, start by creating an account. After you create your account, you should be taken to this form:

Fill out each field with information about how you plan to use the API. You can leave the ‘Callback URL’ field blank, as this is only used if you are building a web application to authenticate as a specific Last.fm user.

After you submit the form, you will get details of your API key and shared secret:

Note these down somewhere safe – we won’t need to use the shared secret for this tutorial, but it’s good to have it noted down in case you want to do something that requires you to authenticate as a specific user.

Making our first API request

In order to create a dataset of popular artists, we’ll be working with the chart.getTopArtists endpoint.

Looking at the Last.fm API documentation, we can observe a few things:

• It looks like there is only one real endpoint, and each “endpoint” is actually specified by using the method parameter.
• The documentation says This service does not require authentication. While this might seem slightly confusing at first, what it’s telling us is that we don’t need to authenticate as a specific Last.fm user. If you look above this, you can see that we do need to provide our API key.
• The API can return results in multiple formats – we’ll specify JSON so we can leverage what we already know about working with APIs in Python

Before we start, remember that we need to provide a user-agent header to identify ourselves when we make a request. With the Python requests library, we specify headers using the headers parameter with a dictionary of headers like so:

We’ll start by defining our API key and a user-agent (the API key shown in this tutorial is not a real API key!)

Next, we’ll import the requests library, create a dictionary for our headers and parameters, and make our first request!