Prerequisities

Before you start looking into pyblish-starter, it is recommended that you first familiarise yourself with Pyblish.


Target Audience

To make the most out of pyblish-starter, some knowledge and experience is assumed.

  minimal recommended
personality curious excited
title technical director pipeline technical director
experience 1 year in advertisements or games 5+ years in feature film
software windows, linux or macos maya





“Pyblish is elegantly designed and closely resembles our in-house production publishing tools here at Imageworks. The setup and documentation is straightforward and easy to understand, getting you setup with a production-quality asset publisher in no time.”

Dhruv Govil - Pipeline TD @ Sony Pictures Imageworks





Introduction

Welcome to pyblish-starter, a basic asset creation pipeline - batteries included.


Table of contents



Install

Starter takes the form of a Python package with embedded plug-ins.

$ pip install pyblish-starter

PyPI version



Usage

Starter is initialised by calling install() with an interface for your host.

>>> from pyblish_starter import install, maya
>>> install(maya)

From here, you model, rig and animate as per the contract below.



Description

Build your own ASSET CREATION PIPELINE, starting with the basics.

Overview

Asset creation covers aspects related to producing content used in the production of film. A film is typically partitioned into sequences and shots, where each shot consists of one or more assets.

This project includes plug-ins and tools for 3 common types of ASSETS in a typical production pipeline.

These illustrate how to (1) devise a contract - known as a family - for each kind of ASSET and (2) design their interface towards each other.

Batteries included

In addition to contracts, the project illustrates boundaries between publishing and setup. The definition of “setup” being anything that isn’t strictly output, such as creating geometry or assigning attributes.

It includes a series of graphical user interfaces to aid the user in conforming to these contracts. These interfaces represent placeholders for your pipeline to fill in; you are invited to either continue from where they finish, or build your own from scratch.

Limitations

Each of the concepts in this pipeline are compatible with pipelines of any complexity. In the interest of simplicity however, some concepts were excluded.



Batteries

In addition to providing a co-operative set of plug-ins, Starter also implements a minimal toochain for ASSET creation in a typical film pipeline.


Creator

Associate content with a family.

The family is what determins how the content is handled throughout your pipeline and tells Pyblish what it should look like when valid.

API

The creator respects families registered with Starter.

from pyblish_starter import api

api.register_family(
    name="my.family",
    help="My custom family"
)

For each family, a common set of data is automatically associated with the resulting instance.

{
    "id": "pyblish.starter.instance",
    "family": {chosen family}
    "name": {chosen name}
}

Additional common data can be added.

from pyblish_starter import api

api.register_data(
    key="myKey",
    value="My value",
    help="A special key"
)

Finally, data may be associated with a family.

from pyblish_starter import api

api.register_family(
    name="my.family",
    data=[
        {"key": "name", "value": "marcus", "help": "Your name"},
        {"key": "age", "value": 30, "help": "Your age"},
])



Loader

Visualise results from api.ls().

from pyblish_starter import api

for asset in api.ls():
    print(asset["name"])

API

The results from api.ls() depends on the currently registered root.

from pyblish_starter import api
api.register_root("/projects/gravity")

The chosen ASSET is passed to the load() function of the currently registered host.

from pyblish_starter import api, maya
api.register_host(maya)

A host is automatically registered on pyblish_starter.install().










Looking Ahead

Without publishing, in any shape or form, the following essential tools are but a dream.

Name Purpose Description
browser remove file-system dependence Search and clear presentation of available data relative a given project or task.
loader control what goes in Keep tabs on where data comes from so as to enable tracking and builds.
builder associate disparate ASSETS Automatic composition of data that changes independently but need to somehow stay associated.
manager stay up to date Notification and visualisation of loaded data.



API

Starter exposes a series of interrelated APIs to the end-user.

Name Purpose
Filesystem API Defines how the developer interact with data on disk
Starter API Defines how the developer interacts with Starter
Host API Defines how the host interacts with Starter



Terminology

Starter reserves the following words for private and public use. Public members are exposed to the user, private ones are internal to the implementation.

  Term Public Description Example
PROJECT X Root of information Gravity, Dr. Strange
ASSET X Unit of data Ryan, Bicycle, Flower pot
VERSION X An ASSET iteration v1, v034
REPRESENTATION   A data format Maya file, pointcache, thumbnail
FORMAT   A file extension .ma, .abc, .ico, .png
WORKSPACE X Private data Scenefile for v034 of Ryan
STAGE   Transient data Outgoing VERSION from scenefile
SHARED X Public data v034 of Ryan
PRODUCER   Creator of data You
CONSUMER   User of data Me



Filesystem API

Data is organised into files and folders.

Some files and folders have special meaning in Starter.

image


Information Hierarchy

The mental and physical model for files and folders look like this.

temp_03



Workspace, Stage and Share

During the course of the creation of any ASSET, data moves between 2 of 3 states.

temp


Workspace and Shared separation

A naive approach to content creation might be to refer to ASSETS straight from another artists workspace. Starter separates between data considered work-in-progress and data exposed to others.

Workspace data is highly mutable and typically private to an individual artist.

Shared data on the other hand is immutable, correct and impersonal.



Each ASSET reside within the top-level root directory as follows.

Hierarchy Example
hier hierex


Each ASSET contain 0 or more versions which in turn contain 0 or more representations.

Hierarchy Example
org exm


Every extraction is made into the staging directory, regardless of whether it integrates successfully or not.

Hierarchy Example
usr1 usr2

Each directory will contain everything that did extract successfully, along with its metadata, for manual inspection and debugging.


ls()

Communication with the filesystem is made through ls().

ls() returns available assets - relative the currently registered root directory - in the form of JSON-compatible dictionaries. Each dictionary is strictly formatted according to three distinct “schemas” (see below).

See below for a full list of members.

Example

from pyblish_starter import api

for asset in api.ls():
    for version in asset["versions"]:
        for representation in version["representations"]:
            pass


Schema

Schema

Available schemas are organised hierarchically, with the former containing the latter.


asset.json

A unit of data

Key Value Description
name str Name of directory
versions list 0 or more version.json


version.json

An ASSET iteration

Key Value Description
version int Number of this VERSION
path str Unformatted path
time str ISO formatted, file-system compatible time.
author str User logged on to the machine at time of publish.
source str Original file from which this VERSION was made.
representations list 0 or more representation.json


representation.json

A data FORMAT.

Key Value Description
format str File extension
path str Unformatted path



Starter API

pyblish-starter provides a stateful API.

State is set and modified by calling any of the exposed registration functions, prefixed register_*, or automatically on calling pyblish_starter.install().


Public members of pyblish_starter

Member Returns Description
install(host) str Install Starter into the current interpreter session
uninstall() str Revert installation


Public members of pyblish_starter.api

Member Returns Description
ls() generator List available assets, relative root
root() str Absolute path to current working directory
format_staging_dir(root, name) str Return absolute path or staging directory relative arguments
format_shared_dir(root) str Return absolute path of shared directory
format_version(version) str Return file-system compatible string of version
find_latest_version(versions) int Given a series of string-formatted versions, return the latest one
parse_version(version) str Given an arbitrarily formatted string, return version number
register_root(root)   Register currently active root
register_host(host)   Register currently active host
register_plugins()   Register plug-ins bundled with Pyblish Starter
deregister_plugins()    
registered_host() module Return currently registered host
registered_families() list Return currently registered families
registered_data() list Return currently registered data
registered_root() str Return currently registered root



Host API

A host must implement the following members.

Member Returns Description
ls() generator List loaded assets
create(name, family) dict Build fixture for outgoing data (see instance)
load(asset, version=-1) str Import external data into container


Information hierarchy

Loaded data is stored in a container. A container hosts a loaded asset along with metadata used to associate assets that use other assets, such as a Wheel asset used in a Car asset.

Host data relationship

Id

Internally, Pyblish instances and containers are distinguished from native content via an “id”. For example, in Maya, the id is a user-defined attribute.

Name Description Example
pyblish.starter.container Unit of incoming data ...:model_GRP, ...:rig_GRP
pyblish.starter.instance Unit of outgoing data Strange_model_default



Contract

Starter defines these families.

Family Definition Link
starter.model Geometry with deformable topology Spec
starter.rig An articulated starter.model for animators Spec
starter.animation Pointcached starter.rig for rendering Spec


starter.model

A generic representation of geometry.

aud Target Audience

req Requirements

dat Data

set Sets





starter.rig

The starter.rig contains the necessary implementation and interface for animators to animate.

aud Target Audience

req Requirements

dat Data

set Sets



starter.animation

Point positions and normals represented as one Alembic file.

aud Target Audience

req Requirements

dat Data

set Sets


Legend

  Title Description
aud Target Audience Who is the end result of this family intended for?
req Requirements What is expected of this ASSET before it passes the tests?
dat Data End-user configurable options
set Sets Collection of specific items for publishing or use further down the pipeline.



Homework

With an understanding of this asset creation pipeline, here are some suggestions for what to explore next.

Difficulty Task Description
easy Loader tooltip Pick any of the available data from an asset, and add it to the tooltip of the item.
hard Shots Think of and document how the kind of workflow exemplified here - (1) creating, (2) publishing and (3) loading assets - can be applied to a shot- and task-based pipeline.
challenging A Creator Help Page The creator interface is very limited. Try including a help-page and picture alongside each family illustrating what each are for.
hard Loader thumbnails The loader interface is even more limited, with room for expansion. Consider how you might go about extracting a thumbnail alongside each asset, and add it to the interface.



Contributing

pyblish-starter, as Pyblish itself, is an open source effort and contributions are welcome.

For example, you could fork Starter, expand upon the graphical user interfaces and either make it your own or submit a pull-request to have it merge with the official project.

For more information on this, contact me and let’s have a conversation!