Skip to main content

dataset

superduper.components.dataset

Source code

Dataset​

Dataset(self,
identifier: str,
db: dataclasses.InitVar[typing.Optional[ForwardRef('Datalayer')]] = None,
uuid: None = None,
*,
upstream: "t.Optional[t.List['Component']]" = None,
artifacts: 'dc.InitVar[t.Optional[t.Dict]]' = None,
select: 't.Optional[Query]' = None,
sample_size: 't.Optional[int]' = None,
random_seed: 't.Optional[int]' = None,
creation_date: 't.Optional[str]' = None,
raw_data: 't.Optional[t.Sequence[t.Any]]' = None,
pin: 'bool' = False) -> None
ParameterDescription
identifierIdentifier of the leaf.
dbDatalayer instance.
uuidUUID of the leaf.
artifactsA dictionary of artifacts paths and DataType objects
selectA query to select the documents for the dataset.
sample_sizeThe number of documents to sample from the query.
random_seedThe random seed to use for sampling.
creation_dateThe date the dataset was created.
raw_dataThe raw data for the dataset.
pinWhether to pin the dataset. If True, the dataset will load the datas from the database every time. If False, the dataset will cache the datas after we apply to db.

A dataset is an immutable collection of documents.

DataInit​

DataInit(self,
identifier: str,
db: dataclasses.InitVar[typing.Optional[ForwardRef('Datalayer')]] = None,
uuid: None = None,
*,
upstream: "t.Optional[t.List['Component']]" = None,
artifacts: 'dc.InitVar[t.Optional[t.Dict]]' = None,
data: 't.List[t.Dict]',
table: 'str') -> None
ParameterDescription
identifierIdentifier of the leaf.
dbDatalayer instance.
uuidUUID of the leaf.
artifactsA dictionary of artifacts paths and DataType objects

DataInit(identifier: str, db: dataclasses.InitVar[typing.Optional[ForwardRef('Datalayer')]] = None, uuid: None = None, *, upstream: "t.Optional[t.List['Component']]" = None, artifacts: 'dc.InitVar[t.Optional[t.Dict]]' = None, data: 't.List[t.Dict]', table: 'str')