Package

Exporting Component instances

Packaging Component instances for sharing, and using in other deployments and teams, is straightforward, by means of the Superduper-protocol, which creates a human-readable export, linked with large data-blobs.

Any component may be exported to disk with:

component.export('<path_to_export>')

The component may be reloaded, without any type of prior initialization, in another session or program:

from superduper import Component

component = Component.read('<path_to_export>')

The component may be then reused with db.apply.

Packaging multiple Component instances as an Application

Developers may want to ship several interconnected components, which may not have a single parent component. To this end, Superduper includes the Application component, which may be used to bundle the components together.

app = Application('my-app', components=[my_model, my_cdc, my_vector_index])

Once bundled as an Application, the components inside may be managed together.

For example, all components in the application may be shown:

db.show(application='my-app')

and all components may be removed in a single command:

db.remove('Application', 'my-app', recursive=True)

Creating Template instances from Component

In order to re-use Component instances with certain values replaced with new values, for example, the location of data, Superduper provides an additional helper component: Template.

Create the template:

t = Template('my-template', substitutions={'my_value': 'my_variable'}, templte=app)

t.export('<path_to_export>')

Load the template and reuse:

t = Template.read('<path_to_export>')

app = t(my_variable='new_value')

db.apply(app)

Leveraging plugins to make development code portable

If developers would like to include locally developed code in their Component, Application and Template instances and implementations, they can use the Plugin component.

from superduper import Plugin

plugin = Plugin('./my_local_file.py')

app = Application('my-app', components=[my_model, my_cdc, my_vector_index], upstream=[plugin])

db.apply(app)   # plugin is applied first (since in `upstream`) and saved as a loadable artifact
app.export('.')