Schevo Database Usage Reference

Overview

A Schevo database is the combination of a schema definition, stored data, and an engine that exposes the two. This document describes in detail the public API exposed by the Schevo engine.

Schema definitions are pure-Python modules that make use of Schevo’s syntax extensions. Schema definitions can be directly imported anywhere, but typically the Schevo database engine manages the import process in order to tie a schema to an open database.

Stored data is managed by the SchevoDurus package, based on Durus. The data is kept in a general structure used by all Schevo databases, described in the internal database structures reference.

The database engine is defined in the schevo.database module, and provides read/write access to stored data using the higher-level structures defined by the database’s schema definition.

Doctest Setup

This document also functions as a doctest:

.. sourcecode:: pycon

  >>> from schevo.test import DocTest

sys Namespaces

sys (short for system) is a namespace that contains the public methods and properties of a Schevo object that are not fields, query methods, transaction methods, or view methods. These are kept in a separate namespace in order to reduce the amount of reserved words.

sys namespaces are accessed by getting the sys attribute from a Schevo object.

The following types of Schevo objects have a sys namespace:

  • Entity instances
  • Find query instances
  • Param query instances
  • Transaction instances
  • View instances

Database instances

Database labels

By default, a database instance is labeled “Schevo Database”:

.. sourcecode:: pycon

  >>> from schevo.label import label, relabel
  >>> t = DocTest("""
  ...     class Foo(E.Entity):
  ...         pass
  ...     """)
  >>> print label(t.db)
  Schevo Database

Override the label by using relabel:

.. sourcecode:: pycon

  >>> relabel(t.db, 'My Database')
  >>> print label(t.db)
  My Database

Extent instances

Relaxing and enforcing key restrictions

Schevo has the unique ability to temporarily relax key restrictions while still ensuring that the database remains in a consistent state.

Key restrictions may only be relaxed within the execution of a transaction.

Given an extent db.Foo that has a key specification of _key(field1, field2), relax that key restriction by calling relax_index:

.. sourcecode:: pycon

  db.Foo.relax_index('field1', 'field2')

Key restrictions that are relaxed within a transaction are relaxed for the entire execution of the transaction, or until the key is enforced within that transaction, or until that transaction ends, whichever comes first.

When a key restriction is relaxed by a transaction, it remains relaxed for all sub-transactions, regardless of any request by those sub-transactions to enforce the key restriction.

To re-enforce the key restriction before the end of a transaction’s execution, call enforce_index:

.. sourcecode:: pycon

  db.Foo.enforce_index('field1', 'field2')

See the tests/test_relax_index.py test case for code examples.

Extent labels

By default, an extent has a singular and plural label based on the entity class name associated with the extent:

.. sourcecode:: pycon

  >>> from schevo.label import label, plural
  >>> t = DocTest("""
  ...     class GreatPerson(E.Entity):
  ...
  ...         name = f.string()
  ...     """)
  >>> print label(t.db.GreatPerson)
  Great Person
  >>> print plural(t.db.GreatPerson)
  Great Persons

To override the singular and/or plural labels of an extent, assign values to the _label and/or _plural attributes of the associated entity class:

.. sourcecode:: pycon

  >>> from schevo.label import label, plural
  >>> t = DocTest("""
  ...     class GreatPerson(E.Entity):
  ...
  ...         name = f.string()
  ...
  ...         _label = 'Great PERSON'
  ...         _plural = 'Great PEOPLE'
  ...     """)
  >>> print label(t.db.GreatPerson)
  Great PERSON
  >>> print plural(t.db.GreatPerson)
  Great PEOPLE

Entity instances

Entity sys namespace

Entity labels

The label of an entity instance is the same as the unicode representation of the entity instance.

By default, the label is determined by the default key defined in the entity class:

.. sourcecode:: pycon

  >>> from schevo.label import label
  >>> t = DocTest("""
  ...     class State(E.Entity):
  ...
  ...         name = f.string()
  ...         abbreviation = f.string()
  ...
  ...         _key(name)
  ...         _key(abbreviation)
  ...
  ...         _initial = [
  ...             ('Missouri', 'MO'),
  ...             ]
  ...     """)
  >>> missouri = t.db.State.findone(name='Missouri')
  >>> print label(missouri)
  Missouri

If the key has more than one field, the values are separated by two colons:

.. sourcecode:: pycon

  >>> t = DocTest("""
  ...     class City(E.Entity):
  ...
  ...         name = f.string()
  ...         state = f.entity('State')
  ...
  ...         _key(name, state)
  ...
  ...         _initial = [
  ...             ('Monett', ('Missouri',)),
  ...             ]
  ...
  ...     class State(E.Entity):
  ...
  ...         name = f.string()
  ...         abbreviation = f.string()
  ...
  ...         _key(name)
  ...         _key(abbreviation)
  ...
  ...         _initial = [
  ...             ('Missouri', 'MO'),
  ...             ]
  ...     """)
  >>> missouri = t.db.State.findone(name='Missouri')
  >>> monett = t.db.City.findone(name='Monett', state=missouri)
  >>> print label(monett)
  Monett :: Missouri

Override the __unicode__ method of an entity class to customize the label:

.. sourcecode:: pycon

  >>> t = DocTest("""
  ...     class City(E.Entity):
  ...
  ...         name = f.string()
  ...         state = f.entity('State')
  ...
  ...         _key(name, state)
  ...
  ...         def __unicode__(self):
  ...             return u'%s, %s' % (self.name, self.state.abbreviation)
  ...
  ...         _initial = [
  ...             ('Monett', ('Missouri',)),
  ...             ]
  ...
  ...     class State(E.Entity):
  ...
  ...         name = f.string()
  ...         abbreviation = f.string()
  ...
  ...         _key(name)
  ...         _key(abbreviation)
  ...
  ...         _initial = [
  ...             ('Missouri', 'MO'),
  ...             ]
  ...     """)
  >>> missouri = t.db.State.findone(name='Missouri')
  >>> monett = t.db.City.findone(name='Monett', state=missouri)
  >>> print label(monett)
  Monett, MO

Entity placeholders

schevo.placeholder.Placeholder instances store information about an entity in a database. Schevo databases keep Placeholder instances internally, and work with them instead of actual Entity instances since the latter cannot be directly persisted.

Do not worry about using Placeholder instances directly, but be aware of their existence. At times, you may run across an Exception whose message includes the repr of a Placeholder instance, such as a schevo.error.KeyCollision error.

Field classes

Field instances

Query methods and objects

At their highest level, schevo.query.Query objects use criteria you specify to create sequences of results.

There are several types of queries, and some queries’ criteria consist of subqueries, from which it obtains results during its execution.

For the purposes of example, let us assume that we are working with the following schema for this section:

>>> t = DocTest("""
... from schevo.schema import *
... schevo.schema.prep(locals())
...
... class Person(E.Entity):
...
...     name = f.string()
...     birthdate = f.date()
...
...     _key(name)
...
...     _plural = u'People'
...
...     def __unicode__(self):
...         return u'%s, born on %s' % (self.name, self.birthdate)
...
...     _sample_unittest = [
...         ('Robert Jones', '1965-02-03'),
...         ('Roberto Gonzales', '1976-04-05'),
...         ('Roberta Smith', '1987-06-07'),
...         ('Rance Thomas', '1954-12-01'),
...         ]
...
... class Location(E.Entity):
...
...     name = f.string()
...
...     _key(name)
...
...     _sample_unittest = [
...         ('Bank',),
...         ('Grocery Store',),
...         ('Library',),
...         ]
...
... class Sighting(E.Entity):
...
...     person = f.entity('Person')
...     location = f.entity('Location')
...     when = f.date()
...
...     def __unicode__(self):
...         return u'%s at %s on %s' % (
...             self.person.name,
...             self.location,
...             self.when,
...             )
...
...     _sample_unittest = [
...         (('Robert Jones',), ('Library',), '2004-01-01'),
...         (('Roberto Gonzales',), ('Library',), '2004-01-04'),
...         (('Robert Jones',), ('Grocery Store',), '2004-01-02'),
...         (('Roberto Gonzales',), ('Bank',), '2004-01-05'),
...         (('Robert Jones',), ('Bank',), '2004-01-03'),
...         (('Roberto Gonzales',), ('Grocery Store',), '2004-01-06'),
...         (('Roberta Smith',), ('Grocery Store',), '2004-01-07'),
...         (('Rance Thomas',), ('Grocery Store',), '2004-01-08'),
...         ]
... """)
>>> db = t.db

Both schema modules and database engines expose all available Query classes in its Q namespace. For brevity, we will use an alias:

>>> Q = db.Q

Match queries

The simplest query is a match on a field. Use the schevo.query.Match class:

>>> person_name = Q.Match(db.Person, 'name', 'startswith')

To set the value of the match query, set the query’s value:

>>> person_name.value = u'Robert'

If you already have a search in mind, you can specify it in the query constructor:

>>> person_name = Q.Match(db.Person, 'name', 'startswith', u'Robert')

It is worth noting that at any point, you can get a human language version of the query by converting it to a string:

>>> print person_name
People where Name starts with Robert

Retrieving results

To retrieve the results of a query, call the Query instance to get an iterator over the results. For example, if we are using the person_name query defined above:

>>> results = person_name()
>>> for person in results:
...     print person
Robert Jones, born on 1965-02-03
Roberto Gonzales, born on 1976-04-05
Roberta Smith, born on 1987-06-07

Query results, at the minimum, must support iteration:

>>> results = person_name()
>>> i = iter(results)
>>> i 
<generator object ...>

To cache the query results, use whatever tools you like. For example, you can get a list of results in order to retrieve the length or perform slices:

>>> results = list(person_name())
>>> len(results)
3
>>> r0 = results[0]
>>> r0
<Person entity oid:1 rev:0>
>>> results[1:3]
[<Person entity oid:2 rev:0>, <Person entity oid:3 rev:0>]
>>> results.index(r0)
0

Compound queries

Continuing the example, let us also search for persons who have a birthdate before 1980-01-01, and who were most recently sighted at the grocery store.

Create the birthdate query:

>>> person_birthdate = Q.Match(db.Person, 'birthdate', '<', '1980-01-01')
>>> print person_birthdate
People where Birthdate < 1980-01-01
>>> for person in sorted(person_birthdate()): 
...     print person
Rance Thomas, born on 1954-12-01
Robert Jones, born on 1965-02-03
Roberto Gonzales, born on 1976-04-05

Combine the results of these queries using an intersection, so that we get the Person entities that match both queries:

>>> persons = Q.Intersection(person_name, person_birthdate)
>>> print persons 
the intersection of (People where Name starts with Robert,
People where Birthdate < 1980-01-01)
>>> for person in sorted(persons()): 
...     print person
Robert Jones, born on 1965-02-03
Roberto Gonzales, born on 1976-04-05

Get all the sightings for each person:

>>> person_sightings = Q.Match(db.Sighting, 'person', 'in', persons)
>>> print person_sightings 
Sightings where Person in the intersection of (People where Name
starts with Robert, People where Birthdate < 1980-01-01)
>>> for sighting in sorted(person_sightings()):
...     print sighting 
Robert Jones at Library on 2004-01-01
Roberto Gonzales at Library on 2004-01-04
Robert Jones at Grocery Store on 2004-01-02
Roberto Gonzales at Bank on 2004-01-05
Robert Jones at Bank on 2004-01-03
Roberto Gonzales at Grocery Store on 2004-01-06

Group the sightings by person. Since we are grouping the results of a query, which may be heterogeneous, we also specify a field class:

>>> by_person = Q.Group(person_sightings, 'person', db.Sighting.f.person)
>>> print by_person 
Sightings where Person in the intersection of (People where Name
starts with Robert, People where Birthdate < 1980-01-01), grouped
by Person
>>> for group in sorted(by_person()): 
...     print '----'
...     for result in group:
...         print result
----
Robert Jones at Library on 2004-01-01
Robert Jones at Grocery Store on 2004-01-02
Robert Jones at Bank on 2004-01-03
----
Roberto Gonzales at Library on 2004-01-04
Roberto Gonzales at Bank on 2004-01-05
Roberto Gonzales at Grocery Store on 2004-01-06

The results of a Group query is a list of results. Reduce each group to the result that has the maximum value for the when field. Again, we specify a field class since the query source may be heterogeneous:

>>> most_recent = Q.Max(by_person, 'when', db.Sighting.f.when)
>>> print most_recent 
results that have the maximum When in each (Sightings where Person
in the intersection of (People where Name starts with Robert, People
where Birthdate < 1980-01-01), grouped by Person)
>>> for sighting in sorted(most_recent()): 
...     print sighting
Robert Jones at Bank on 2004-01-03
Roberto Gonzales at Grocery Store on 2004-01-06

Finally, filter most_recent by the grocery store Location:

>>> grocery_store = db.Location.findone(name=u'Grocery Store')
>>> last_seen_shopping = Q.Match(
...     most_recent, 'location', '==', grocery_store, db.Sighting.f.location)
>>> print last_seen_shopping 
results that have the maximum When in each (Sightings where Person
in the intersection of (People where Name starts with Robert, People
where Birthdate < 1980-01-01), grouped by Person) where Location ==
Grocery Store
>>> for sighting in sorted(last_seen_shopping()):
...     print sighting 
Roberto Gonzales at Grocery Store on 2004-01-06

Default queries

The default query for an extent is an Intersection of Match queries against all non-calculated fields of an extent, with a default operator of any (which means “is any value”), and a default value of UNASSIGNED (which is ignored when using the any operator).

This query is the same as would be returned when calling db.Sighting.q.default():

>>> query = Q.Intersection(
...     Q.Match(db.Sighting, 'person', 'any'),
...     Q.Match(db.Sighting, 'location', 'any'),
...     Q.Match(db.Sighting, 'when', 'any'),
...     )
>>> print query
all Sightings

Without changing any of the operators, iterating over the results of the default query for an extent yields the same thing as iterating over the extent itself.

The Intersection query optimizes its execution plan to use the extent’s find method when all of the Match queries match on the same extent, when there is only one of any given field name, when the only operators used are either any or ==, and when all values are not queries.

As seen above, it also optimizes its label for brevity if all values have the any operator.

Field Containers

The following types of objects are field containers:

A field container obj implements the following:

  • They have a namespace, obj.f, that contains field objects.

    • The value of obj.f.field_name is a Field instance associated with obj, with the name field_name.

    • The value of obj.f['field_name'] is the same as obj.f.field_name.

    • Iterating over obj.f yields the names of the fields, in the order they were defined.

    • If allowed by the field container, you can remove a field named field_name from obj using the following:

      .. sourcecode:: pycon
      
        del obj.f.field_name
      
    • If allowed by the field container, you can add a new field named field_name and of type FieldClass to obj using either of the following:

      .. sourcecode:: pycon
      
        obj.f.field_name = existing_field_instance
        obj.f.field_name = f.field_class(...)
      

      The field is added to the end of the definition order. To re-order fields, you can get them from obj.f, delete them from obj.f, then reassign them to obj.f, as in this example:

      .. sourcecode:: pycon
      
        # Assume original declaration order was:
        #    foo = f.string()
        #    bar = f.string()
        #    baz = f.string()
      
        bar, baz = obj.f.bar, obj.f.baz
        del obj.f.bar, obj.f.baz
        obj.f.baz = baz
        obj.f.bar = bar
      
        # Now the order is the following:
        #    foo = f.string()
        #    baz = f.string()
        #    bar = f.string()
      
  • They have a method, obj.s.fields(), that returns a schevo.fieldspec.FieldMap ordered dictionary for the fields in obj.

    • Certain types of field decorators accept additional arguments to the call to obj.s.fields().

Iterators

Many objects in a Schevo database provide iterators.

Extents

Iterating over an extent results in a sequence of all entities in the extent, ordered by OID.

Query, view, and transaction namespaces

Iterating over a f, q, t, or v namespace results in a sequence of the names in that namespace, in no defined order.

Example:

>>> sorted(db.Person.q)
['by_example', 'exact']
>>> sorted(db.Person[1].t)
['clone', 'delete', 'update']

Entity sys namespaces

Iterating over an entity’s sys namespace has no defined result. It may become part of the Objects without iterators.

Objects without iterators

  • Database instances.
  • Entity instances.
  • Query instances.
  • Transaction instances.
  • View instances.
  • sys namespaces.

Icon plugin