Django and HTMX Live Scores Project

In this post, we will build an application using Django and HTMX that simulates live sports scores streaming into a webpage. The application will use HTMX's polling capabilities for this, and will dynamically update a frontend page with new scores and full-time results as they come in.

We are going to use football data in this example, and will build out models for Tournaments, Teams and Fixtures.

The associated video for this post can be found below:


Objectives

In this post, we will:

  • Design and build a live sports scores webpage that updates with new scores as they come in
  • Understand how to use interval-based polling in HTMX
  • Build a small script that simulates scores updating periodically
  • Understand some pitfalls and considerations when using polling

Project Setup

This project builds off a starter repository that can be found here. Clone this repository to your local filesystem, and install the requirements with the pip install -r requirements.txt command.

This project has a single app called scores and already has a few basic templates set up. There is a base.html template, which loads in HTMX, jQuery, Bootstrap and Bootstrap Icons for use within the app.

Within the settings.py file, we have added a few Django packages to our installed apps:

  • django_extensions - this allows us to use a handy shell_plus command that loads all models and other useful utilities into the Django shell automatically.
  • django_htmx - an HTMX-specific extension that adds an htmx attribute to the Django request object, allowing us to detect when requests originate from HTMX attributes.

After cloning this repository and installing the dependencies, we are now ready to start building out the application. We'll start with the models!

Creating Django Models

There will be three simple models in this application.

  • Tournament - this will store the name of tournaments that are added to our application
  • Team - this stores the name of the teams in our application
  • Fixture - the central class. This contains foreign keys to home teams, away teams and the tournament to which the fixture belongs. We also have fields for the number of goals the home team has scored, the number of goals the away team has scored, and a whether or not the fixture is completed.

Within the models.py file, add the following code to create these models.

from django.db import models

# Create your models here.
class Tournament(models.Model):
    name = models.CharField(max_length=128)

    def __str__(self):
        return self.name


class Team(models.Model):
    name = models.CharField(max_length=128, unique=True)

    def __str__(self):
        return self.name


class Fixture(models.Model):
    tournament = models.ForeignKey(Tournament, on_delete=models.CASCADE, related_name='fixtures')
    home_team = models.ForeignKey(Team, on_delete=models.CASCADE, related_name='home_games')
    away_team = models.ForeignKey(Team, on_delete=models.CASCADE, related_name='away_games')
    home_goals = models.PositiveSmallIntegerField(default=0)
    away_goals = models.PositiveSmallIntegerField(default=0)
    game_finished = models.BooleanField(default=False)

    def __str__(self):
        return f"{self.home_team} vs {self.away_team}"

The Fixture model is the one we will work with, and contains foreign keys to our other models. We use a PositiveSmallIntegerField to store the goals scored for each team (it'll be a small, positive number, so this makes sense). We also use a BooleanField to determine whether each fixture is completed, or not.

After creating these models, we need to reflect our changes to the database. Run the following Django management commands:

python manage.py makemigrations
python manage.py migrate

This should add the tables to the database.

The next step is to populate the database with some real data. We'll use a custom management command for this.

Populating the Database

In the starter code from Github, there's a scores\management\commands\load_teams.py command that we are going to fill out. This command will populate our database with a tournament, some teams from the English Premier League, and some fixtures.

Within that file, add the following code.

from django.core.management.base import BaseCommand
from scores.models import Team, Tournament, Fixture

TEAMS = [
    'Chelsea', 'Man City', 'Liverpool', 'West Ham', 'Arsenal', 'Wolves', 'Tottenham',
    'Man Utd', 'Brighton', 'Crystal Palace', 'Everton', 'Leicester', 'Southampton',
    'Brentford', 'Aston Villa', 'Watford', 'Leeds', 'Burnley', 'Norwich', 'Newcastle'
]

class Command(BaseCommand):
    help = 'Load EPL teams and fixtures'

    def handle(self, *args, **kwargs):
        # add the tournament
        tournament = Tournament.objects.get_or_create(name="Premier League")[0]

        # add the teams
        if Team.objects.count() == 0:
            team_objs = [Team(name=team_name) for team_name in TEAMS]
            teams = Team.objects.bulk_create(team_objs)
        else:
            teams = Team.objects.all()

        # Next step: create a set of fixtures from the teams list

        fixtures = []
        for i in range(0, len(teams), 2):
            fixtures.append(
                Fixture(home_team=teams[i], away_team=teams[i+1], tournament=tournament)
            )

        # bulk create the fixtures
        if Fixture.objects.count() == 0:
            fixtures = Fixture.objects.bulk_create(fixtures)

We have all teams in a global TEAMS variable, and these are created in the database on lines 18-20 - but only if there are no teams in the database already. If the team count is greater than zero, we simply load all teams (lines 21-22). We also use the get_or_create() method to fetch (or create) a Tournament in the database (line 15).

On lines 26-30, we populate a list of Fixture instances, using the range() function and indexes to select two sequential teams from the TEAMS list, and create a fixture for those teams.

The range(0, len(teams), 2) statement uses the third parameter of range() - the step size - to skip two elements forward at each iteration. This allows us to use indexing to select the team at the current iteration, and the next team (i+1), without overlapping any teams.

On lines 33-34, we save the created fixtures, if there are none already in the database.

We now need to run this custom management command to populate our database with these instances. Run the command: python manage.py load_teams to do so.

Let's now create a page in Django to show our newly created fixtures.

Creating Fixtures Page

The first thing to do is create a URL and a View to fetch the fixtures. After that, we'll create the template to display them on the webpage.

Let's start with the URL. Within scores\urls.py, add the following to the urlpatterns.

path('fixtures/', views.fixtures, name='fixtures')

Let's now create the fixture view. In scores\views.py add the following function.

def fixtures(request):
    fixtures = Fixture.objects.all()
    context = {'fixtures': fixtures}
    return render(request, 'fixtures.html', context)

Finally, let's create our fixtures.html template, in the templates directory. This will render out the fixtures we've added to our context in the above function. Add this code:

{% extends 'base.html' %} 

{% block content %}
    {{ fixtures }}
{% endblock %} 

This will now show a list of fixtures on our fixtures page. You might also want to fill in the link on the navbar to the fixtures page - you can do this by changing the relevant line in the scores\templates\partials\navbar.html file to:

<a class="nav-link" href="{% url 'fixtures' %}">Scores</a>

This uses the url template tag to link the anchor to the URL we've just created. You should now be able to navigate to the fixtures page.

You should see something similar to the following:

This works, but it's not particularly nice! Let's style this properly in the next section.

Enhancing Templates

To give us some UI direction, we are going to try and replicate the styles found on the BBC results/fixtures pages, which you can view here. We will build out the header, the search bar, and the results list.

Within our new fixtures.html template, add the following code:

{% extends 'base.html' %} 

{% block content %}
<main>
    <h2>{{ fixtures.first.tournament.name }} Scores & Fixtures</h2>

    <form class="input-group my-4">
        <input type="text"
               id="search-input"
               name="search" 
               class="form-control" 
               placeholder="Enter team" />

        <span class="input-group-text border-0 bg-transparent" 
              id="search-addon" 
              style="cursor: pointer;">
            <i class="bi bi-search"></i>
        </span>
    </form>

    <div id="fixturelist-container">
        {% include 'partials/fixturelist.html' %} 
    </div>
</main>
{% endblock %}

On line 5, we create the header, which references the tournament name.

On lines 7-19, we create the search form UI element. Currently this doesn't do anything, but later we'll use HTMX to drive a dynamic search of our fixture list!

On lines 21-23, we include a template partial that we will create below. This partial will be responsible for rendering out each fixture within a Django template for-loop. It's important to separate this into its own fragment, because HTMX specific requests will later return this fragment and swap it into the DOM, allowing our fixture list to update dynamically.

Let's now create the scores\templates\partials\fixturelist.html template, and add the following code:

<div id="fixture-list">
{% for fixture in fixtures %}
    <div class="fixture row">
        <div class="home col-6 text-right p-0">
            <span class="font-weight-bold">{{ fixture.home_team }}</span>
            <span class="border px-2 py-1 bg-warning">{{ fixture.home_goals }}</span>
        </div>
        
        <div class="away col-6 text-left p-0">
            <span class="border px-2 py-1 bg-warning">{{ fixture.away_goals }}</span>
            <span class="font-weight-bold">{{ fixture.away_team }}</span>
        </div>
    </div>
    {% if fixture.game_finished %}
        <div class="text-center mt-1 text-success">FT</div>
    {% endif %}
    <hr/>
{% endfor %}
</div>

We use a template for-loop to iterate over the fixtures in our context. We use Bootstrap's row and col-6 classes to divide the page into two columns, with the left-hand side containing the home team and their goals, and the right-hand side containing the away team and their goals.

We use additional Boostrap classes to style the UI elements with borders, padding, and background colours.

On lines 13-15, we add centered text underneath each fixture's row that will display FT (full-time) if the fixture is finished.

The effect of these changes is shown below:

We've done a decent job of replicating the styles used by the BBC fixtures page, using Bootstrap.

Now, let's get some HTMX into the mix!

HTMX Polling for Changes

At a high-level, we want to use HTMX to send periodic requests (every 3 seconds) to our backend view, and return an updated fragment containing all of our fixtures. This should update the above view with new scores, as they come in.

We are going to add some HTMX attributes to the following <div> element in the scores\templates\fixtures.html template:

<div id="fixturelist-container">
     {% include 'partials/fixturelist.html' %} 
</div>

This <div> encloses the template fragment that displays all the fixtures in our context. We want to replace the innerHTML of this <div> when we receive a new response with updated fixtures. Let's add the relevant attributes to the element:

<div id="fixturelist-container" hx-get="{% url 'fixtures' %}" hx-trigger="every 3s" hx-swap="innerHTML">
     {% include 'partials/fixturelist.html' %} 
</div>

We've added 3 HTMX attributes here. Firstly, we use hx-get to wire up an HTTP GET request to our fixtures URL. Secondly, we use hx-trigger="every 3s" to specify that the HTMX request should be triggered periodically, with the interval being set to three seconds. Finally, the hx-swap="innerhtml" attribute instructs HTMX to swap out the included fixturelist.html fragment with the newly returned HTML from the backend, each time the response is received.

The every trigger is used by HTMX to poll our backend with a specified interval.

We now need to amend our Django view to handle the HTMX polling. Change the code to the following:

def fixtures(request):
    fixtures = Fixture.objects.all()
    context = {'fixtures': fixtures}
    if request.htmx:
        return render(request, 'partials/fixturelist.html', context)
    else:
        return render(request, 'fixtures.html', context)

We now return a different template, depending on whether we are dealing with an HTMX request or not. The request.htmx property is added by the django-htmx library that was added to our starter code.

When we're polling the backend, we only need to replace the list of fixtures each time we get a response. This is why we've separated the fixturelist into a separate template - to allow us to easily replace this list when we're polling with HTMX.

When the request comes into this view every 3 seconds, we re-fetch all our fixtures from the database. Any changes to the fixures that have occurred since the previous request will be reflected in the new fixtures that are passed to the context, and therefore the rendered HTML will show our new scores.

If you now run the Django development server and navigate to the fixtures page, you'll see the same image as earlier - but we are now polling for updates. The problem is, there are no updates! So let's create a custom management command to simulate our scores being updated in real-time.

Result Generator Command

We're going to simulate live score updates by creating the following file: scores\management\commands\result_generator.py

Within this file, add the following code.

import time
import random
from django.core.management.base import BaseCommand
from scores.models import Team, Fixture

class Command(BaseCommand):
    help = 'Load EPL teams and fixtures'

    def handle(self, *args, **kwargs):
        ITERATIONS = 10

        for i in range(ITERATIONS):
            time.sleep(random.randint(1,6))

            # select how many fixtures we're going to update
            update_count = random.randint(1,6)

            # order_by("?") gets random order of rows
            fixtures = Fixture.objects.filter(game_finished=False).order_by("?")

            # get the fixtures we're going to update
            fixtures = fixtures[:update_count]

            self.update_fixtures(fixtures)

            self.is_game_finished(fixtures)

    def update_fixtures(self, fixtures):
        """ Add 1 or 2 goals to each team in the fixture """
        for fixture in fixtures:
            home_goal = random.randint(1,2)
            away_goal = random.randint(1,2)
            fixture.home_goals += home_goal
            fixture.away_goals += away_goal

        Fixture.objects.bulk_update(fixtures, ['home_goals', 'away_goals'])

    def is_game_finished(self, fixtures):
        """ With probability 0.3, mark the game as finished """
        for fixture in fixtures:
            # Generate uniform value between 0 and 1
            # If this is < 0.3, then mark game as finished
            P = random.uniform(0,1)
            if P < 0.3:
                fixture.game_finished = True
                fixture.save()

This script uses the time module to periodically sleep, then updates some scores when the sleep call ends, and then sleeps again for another randomly-generated number of seconds. You can configure the ITERATIONS constant to determine how many updates to provide.

The code: Model.objects.order_by("?") will return a random order of the models in the queryset.

You can run the above management script with the command: python manage.py result_generator.

If you run this command while observing the fixtures page on the browser, you should now see results interactively updating, similar to below!

We're now seeing the scores dynamically update, without any page refresh. The script also selects some fixtures to move to full-time, which is also reflected in our updated fragment. Cool!

There's one final optimization to make in this post. We want to prevent polling the backend if all the fixtures in our list are marked as completed. We'll do this in the next section.

Removing Polling if Fixtures are Completed

If all fixtures are completed, then we should stop the polling from occurring. This will help relieve the load on our server(s), by preventing unnecessary AJAX requests - fixtures that are completed cannot change, so there is nothing to gain from continued polling!

How do we achieve this? We can first change our view function to check if all the fixtures are completed. Amend the fixtures view as below:

def fixtures(request):
    fixtures = Fixture.objects.all()

    # check if all games are finished
    all_completed = all(f.game_finished for f in fixtures)

    context = {
        'fixtures': fixtures,
        'all_completed': all_completed
    }

    if request.htmx:
        return render(request, 'partials/fixturelist.html', context)
    else:
        return render(request, 'fixtures.html', context)

We check on line 5 whether all games are finished, using the built-in all() function. This returns a boolean, which we add to our context. We're now going to amend the fixtures.html template to only include the HTMX polling attributes if all_completed = False. To this template, amend this code:

<div id="fixturelist-container" hx-get="{% url 'fixtures' %}" hx-trigger="every 3s" hx-swap="innerHTML">
     {% include 'partials/fixturelist.html' %} 
</div>

To the following:

{% if all_completed %}
    <div id="fixturelist-container">
{% else %}
    <div id="fixturelist-container" 
        hx-get="{% url 'fixtures' %}" 
        hx-trigger="every 3s" 
        hx-swap="innerHTML">
{% endif %}
    {% include 'partials/fixturelist.html' %} 
</div>

We only attach the HTMX attributes if all fixtures are not completed. Otherwise, we use a plain <div> element, with no HTMX magic! This will prevent further polling.

This prevents polling when the fixtures page is first loaded, if all fixtures are completed at the time when the page is loaded. However, if all fixtures complete while the user is already on the page, then the polling will not stop - this is because the code above is not in the template fragment returned in HTMX requests. The HTMX template fragment only replaces the list of fixtures, so if all games are finished while the user is already on the page, nothing will happen and the polling will occur indefinitely.

We need a mechanism to reload the parent fixtures.html template that is responsible for including our fixturelist.html fragment. If we can reload the parent, our all_completed boolean context variable will be correctly set to True, and will prevent the HTMX attributes being added. Let's look at one way to achieve this.

HX-Refresh Response Header

We can trigger a page refresh by adding an HTMX-specific HTTP header to our response - the HX-Refresh header, which we must set to "true". For more details, check out HTMX's documentation on the different response headers that can trigger client-side behaviour.

A page refresh would force the entire page to be reloaded, and would correctly prevent the polling. This refresh should only occur when all fixtures are completed, and within an HTMX request - let's change our view to add this HTTP header below.

def fixtures(request):
    fixtures = Fixture.objects.all()

    all_completed = all(f.game_finished for f in fixtures)

    context = {
        'fixtures': fixtures,
        'all_completed': all_completed
    }

    if request.htmx:
        if all_completed:
            response = render(request, 'partials/fixturelist.html', context)
            response['HX-Refresh'] = "true"
            return response
        return render(request, 'partials/fixturelist.html', context)
    else:
        return render(request, 'fixtures.html', context)

The new code is on lines 12-15. If all games are completed during the HTMX polling request, then we'll render the response into a variable, and we can set the HX-Refresh header on that response, and return it. Otherwise, if not all games are completed, we return the same template as before with no changes (line 16).

So, if a user is on the page, and the final fixture is completed, then we will enter the conditional in lines 13-15. By adding this HTTP Header, we trigger a client-side refresh, which will result in the parent fixtures.html template being loaded on the next request (line 18). This will have the all_completed boolean set to True, which prevents the HTMX attributes being added within that template.

Polling Pitfalls & Considerations

Polling is a great tool for a task such as this one, where you want to periodically reflect updates to the frontend. This technique applies in any scenario with data that is regularly updated, such as sports scores, cryptocurrency and stock market data, and social media data.

However, be aware that polling should be done sensibly, because it generates a lot of load on your servers. Imagine thousands of users, all polling the server every 1 second. You will need a robust architecture to handle this scenario. For example, storing updates in a cache rather than having thousands of database queries every second would be beneficial.

You can use common sense to determine a good polling interval. For a sports/scores application, you might be fine polling every 30 seconds or so, as scores do not update that often. On the other hand, if you are monitoring a nuclear power station, it might be more important to get updates more frequently. Stock market data changes very regularly, too.

Summary

We have built an app that displays sports scores on a page, and uses HTMX's polling capabilities to interactively update that page whenever the scores are changed on the backend. We use a custom Django management command to simulate updates, which are subsequently reflected on the frontend.

We haven't written a single line of JavaScript to achieve this. Amazing!

In the next post, we're going to extend this example to give users feedback when polling occurs, using the hx-indicator attribute. We're also going to build out the search functionality using HTMX, and integrate this with Django querysets.

If you enjoyed this post, please subscribe to our YouTube channel and follow us on Twitter to keep up with our new content!

Please also consider buying us a coffee, to encourage us to create more posts and videos!

;