Custom Django Model Fields

By: @shaib_il and @daonb

Background

Django's ORM main objects are:

  • Manager & QuerySets - operations related to more than one object
  • Model - operations on a single object
  • Field - a single data item

Field classes include the code needed to translate a datum between

  • Python ⇄ Database
  • Python ⇄ String

The Model instances fields include just the values. The field instances are stored in model._meta

Field classes can be found in:

  • django.db.models.fields
  • django.contrib.gis.db.models.fields
  • django.contrib.postgres.fields

If they are not enough, you can write your own

Matific's use case:

Storing Users' achievments

The Specs:

  • Millions of users
  • Every user plays very few episodes
  • By playing an episode the user gets a score
  • We need to store last and high score per episode
The Model

...
from slatemath.commons.fields import CompressedListField
...
class UserCredits(models.Model):
    account = models.ForeignKey(User)
    last_score = CompressedListField()
    high_score = CompressedListField()
    ...

More Model

...
    def update(self, score, episode_seq_id):
        ''' update score and points of a specific episode '''
        if (score is not None
            and score != self.last_score[episode_seq_id]):
            self.last_score[episode_seq_id] = score
            if (self.high_score[episode_seq_id] ==
                        self.high_score.EMPTY
                or self.high_score[episode_seq_id] < score):
                self.high_score[episode_seq_id] = score
            updated = True
            ...
        return updated

CompressedListField

Design

  • Field's value is an auto-growing-list holding integers
  • Field is based on BinaryField
  • Field is compressed using bz2

Python ⇄ Database


    def get_prep_value(self, value):
        if value:
            value = bz2.compress(array.array('l', list(value)))
        else:
            value = b''
        return super(CompressedListField, self).get_prep_value(value)

    def from_db_value(self, value, expression, connection, context):
        if value:
            compressed = super(CompressedListField, self).to_python(value)
            data = bz2.decompress(compressed)
            return AutoGrowingList(array.array(self.type_code, data))
        else:
            return self.get_default()

Python ⇄ String


    def to_python(self, value):
        if isinstance(value, AutoGrowingList):
            return value
        if value:
            compressed = super(CompressedListField, self).to_python(value)
            data = bz2.decompress(compressed)
            return AutoGrowingList(array.array(self.type_code, data))
        else:
            return self.get_default()

    def value_to_string(self, obj):
        value = self.value_from_object(obj)
        return self.get_prep_value(value)

More methods:

  • deconstruct() - for migrations
  • get_default()- for initialization

Where to get it:

Thanks You!

amtific logo

We're Hiring