Field Indexes

Default

If you do not specify any indexes, the gcloudc backend will index all django model fields by default.

This is a pit of success design, because Datastore/Firestore requires an index to perform any query.

For example:


class MyModel(models.Model):
    field1 = models.IntegerField(null=True)
    field2 = models.CharField(max_length=32)

In the above example field1 and field2 will be indexes because there is no explicit definition of the index configuration, and we default to indexing all fields.

Meta Indexes

gcloudc supports granular control over indexes through the opt-in use of Meta.indexes.


class MyModel(models.Model):
    field1 = models.IntegerField(null=True)
    field2 = models.CharField(max_length=32)
    field3 = models.CharField(max_length=32)

    class Meta:
        indexes = [
            models.Index(fields=["field1"]),
            models.Index(fields=["field2"])
        ]

In the above example field1 and field2 will be indexed. field3 will be excluded because it is NOT referenced in Meta.indexes.

This configurability is useful because excluding fields can provide cost and performance improvements.

Runtime Warnings

To help identify queries which are executed through the ORM which depend on an index which is missing, we raise a DatabaseError at runtime.

Limitations

  • We only support this on the Datastore backend. For Firestore single field exceptions can be added from the console, see the documentation
  • We only support single property indexes on the model, i.e. composite indexes are not supported yet (you need an index.yaml or similar for these).
  • db_index is not supported at field level (the django docs have a future deprecation warning). You must use Meta.indexes.
  • TextField and BinaryField will not be indexed, even if defined in the Meta. This is because these fields exceed the maximum size limits for their respective indexed properties.
  • Queries requiring composite indexes need all properties to be indexed - see the unindexed properties section in the docs, i.e. "if a property appears in a composite index, then excluding the property will disable it in the composite index."
  • Projection queries require that the specified fields are indexed - see the projection queries section in the docs, i.e. "Projection queries require the specified properties to be indexed."
  • If you add fields to the meta indexes, these indexes won’t be applied to existing entities in the datastore but will require a migration.
  • Setting the indexes in the meta to an empty list (i.e. indexes = []) will not exclude all fields from being indexed; instead, all fields will remain indexed. If you wish to exclude all fields that can be indexed, you can pass the primary key field to the meta, i.e. indexes = [models.Index(fields=["id"])] in the above example, as the primary key field will always be indexed, whether defined in the meta or not.