A basic asset creation pipeline - batteries included.

Keywords: Film, games, content creation, pipeline
Objective:
1. Provide a unified framework within which artists at Mindbender may work efficiently
2. Enable extending of said framework for future improvements and unique projects
3. Inspire further expansion upon basic ideas
Mission: Make content creation at Mindbender more effective and less error-prone.
Requirements: Reliably output correct data with minimal impact on artist productivity.
Technology: Mindbender is built upon Pyblish, Python and bindings for Qt, and depends upon a Windows, Linux or MacOS operating system with Autodesk Maya.

Prerequisities

Before you start looking into pyblish-mindbender, it is recommended that you first familiarise yourself with Pyblish.

learn.pyblish.com

Target Audience

To make the most out of pyblish-mindbender, some knowledge and experience is assumed.

	minimal	recommended
`title`	technical director	pipeline technical director
`experience`	1 year in advertisements or games	5+ years in feature film
`software`	windows, linux or macos	maya

Introduction

Welcome to pyblish-mindbender, the production pipeline at Mindbender Animation Studio.

Table of contents

Install
Usage
Description
Batteries
- Creator
- Loader
Looking Ahead
API
Contract
Homework
Contributing
Help

Install

Mindbender takes the form of a Python package with embedded plug-ins.

# Not Yet Implemented
$ pip install pyblish-mindbender

Usage

Mindbender is initialised by calling install() with an interface for your host.

>>> from mindbender import api, maya
>>> api.install(maya)

Supported hosts

From here, you model, rig and animate as per the contract below.

How to read this guide

Here are a few of the conventions used throughout this guide.

bold words are used to augment important aspects of a sentence
UPPERCASE words are unique terminology, each detailed under Terminology below.
(1), (2) are used to division important aspects in a sentence.
code is used to highlight words that occur in code.

Description

Mindbender is a content creation studio focusing on photo-realistic, yet exaggerated cartoons. They work across continents in Sweden, Brazil and Spain and therefore require an underlying pipeline that facilitate these activities.

Pyblish Mindbender works exclusively with computer generated imagery - which means no focus is put on video, sound or the integration of 3d and live action footage. It has been designed for use both locally and remotely, to bridge the gap between artists working together but in different locations.

Overview

This pipeline covers the entire pipeline at Mindbender, including asset and shot creation.

A PROJECT is partitioned into SHOTs where each shot consists of one or more ASSETs.

This PROJECT includes tools and Pyblish plug-ins for 4 common types of ASSETs in a typical production pipeline.

Modeling
Rigging
Animation
Look

These implement (1) a contract - known as a family - for each kind of ASSET and (2) their interface towards each other.

Batteries

Pyblish Mindbender includes a series of graphical user interfaces that aid the user in conforming to the specified contracts.

Name	Purpose	Description
creator	control what goes out	Manage how data is outputted from an application.
loader	control what goes in	Keep tabs on where data comes from so as to enable tracking and builds.
builder	associate disparate ASSETs	Automatic composition of data that changes independently but need to somehow stay associated.
manager	stay up to date	Notification and visualisation of loaded data.

Creator

Associate content with a family.

The family is what determins how the content is handled throughout your pipeline and tells Pyblish what it should look like when valid.

API

The creator respects families registered with Mindbender.

from mindbender import api

api.register_family(
    name="my.family",
    help="My custom family",
)

For each family, a common set of data is automatically associated with the resulting instance.

{
    "id": "pyblish.mindbender.instance",
    "family": {chosen family}
    "name": {chosen name}
}

Additional common data can be added.

from mindbender import api

api.register_data(
    key="myKey",
    value="My value",
    help="A special key"
)

Data may be associated with a family.

from mindbender import api

api.register_family(
    name="my.family",
    data=[
        {"key": "name", "value": "marcus", "help": "Your name"},
        {"key": "age", "value": 30, "help": "Your age"},
])

Each family may be associated to one or more LOADERs.

from mindbender import api

def mb_loader(version):
    """A custom loader for Maya Ascii files"""
    subset = version["subset"]
    asset = subset["asset"]
    cmds.file(version["path"],
              reference=True,
              namespace=asset["name"] + "01_")


api.register_family(
    family="my.family",
    loader=mb_loader
)

Now when this asset is loaded via Loader, this custom function is called.

Loader

Visualise results from api.ls().

from mindbender import api

for asset in api.ls():
    print(asset["name"])

API

The results from api.ls() depends on the currently registered root.

from mindbender import api
api.register_root("/projects/gravity")

The chosen ASSET is passed to the load() function of the currently registered host.

from mindbender import api, maya
api.register_host(maya)

A host is automatically registered on api.install().

API

Mindbender exposes a series of interrelated APIs to the end-user.

Name	Purpose
Terminal API	Defines how the artist interacts with tasks
Filesystem API	Defines how the developer interact with data on disk
Mindbender API	Defines how the developer interacts with Mindbender
Host API	Defines how the host interacts with Mindbender

Terminology

Mindbender reserves the following words for private and public use. Public members are exposed to the user, private ones are internal to the implementation.

Term	Public	Description	Example
`PROJECT`	`X`	Root of information	Gravity, Dr. Strange
`ASSET`	`X`	Unit of data	Ryan, Bicycle, Flower pot
`VERSION`	`X`	An ASSET iteration	v1, v034
`REPRESENTATION`		A data format	Maya file, pointcache, thumbnail
`FORMAT`		A file extension	`.ma`, `.abc`, `.ico`, `.png`
`WORKSPACE`	`X`	Private data	Scenefile for v034 of Ryan
`STAGE`		Transient data	Outgoing VERSION from scenefile
`SHARED`	`X`	Public data	v034 of Ryan
`PRODUCER`		Creator of data	You
`CONSUMER`		User of data	Me

Terminal API

Applications are run via the terminal.

`mb.bat`

Main executable.

This is what artists launch at the start of any task and is from where applications are started, such as Maya and Nuke.

Environment Variable	Description
PYBLISH_MINDBENDER	Absolute path to pyblish-mindbender
PYBLISH_BASE	Absolute path to pyblish-base
PYBLISH_MAYA	Absolute path to pyblish-maya
PYBLISH_NUKE	Absolute path to pyblish-nuke
PYBLISH_QML	Absolute path to pyblish-qml
PYBLISH_LITE	Absolute path to pyblish-lite

Example

@echo off

:: Helper variables
set GIT=M:\f03_assets\include\pyblish\git
set MB=%GIT%\pyblish-mindbender\bin\_mb.bat

:: Establish requirements
set PYBLISH_BASE=%GIT%\pyblish-base
set PYBLISH_MAYA=%GIT%\pyblish-maya
set PYBLISH_NUKE=%GIT%\pyblish-nuke
set PYBLISH_QML=%GIT%\pyblish-qml
set PYBLISH_LITE=%GIT%\pyblish-lite
set PYBLISH_MINDBENDER=%GIT%\pyblish-mindbender

:: MB Tools
set MAYA_PLUG_IN_PATH=M:\f03_assets\include\maya\scripts\Plugins;%MAYA_PLUG_IN_PATH%
set PYTHONPATH=M:\f03_assets\include\maya\scripts;%PYTHONPATH%

:: Install Nuke-specific Pyblish environment
set NUKE_PATH=M:\f03_assets\include\pyblish\etc\nuke;%NUKE_PATH%

call %MB% M:\f01_projects %*

asset.bat

@echo off
call _mkproject %~dp0 %~n0 %1

start.bat

Upon starting any application, the artist is required to enter (1) PROJECT, (2) asset or shot, (3) task and finally (4) application.

$ p999_Gifts_for_Greta_Assets
$ characters\Greta
$ maya modeling
Launching Maya..

The layout is as follows.

{PROJECT} (with TAB-completion)
{ASSET} or Shot (with TAB-COMPLETION)
{APPLICATION} {TASK}

The given TASK is automatically created, unless it already exists.

Environment variables

Variable	Description
`PROJECT`	Nice name of PROJECT, e.g. Gifts for Greta
`PROJECTDIR`	Absolute path to PROJECT, e.g. m:\f01_projects\p999_Gifts_for_Greta
`ROOT`	Top level directory of either shot or asset, e.g. ..\Greta

Filesystem API

Data is organised into files and folders.

Some files and folders have special meaning in Mindbender.

Information Hierarchy

The mental and physical model for files and folders look like this.

temp_03

During the course of the creation of any ASSET, data moves between 2 of 3 states.

temp

Mindbender does not take into consideration the workspace and is therefore workspace-agnostic.
The staging area is both implicit and transparent to the PRODUCER and CONSUMER, except for debugging purposes. This is where automatic/scheduled garbage collection may run to optimise for space constraints.
The shared space is where ASSETs ultimately reside once published.

Workspace and Shared separation

A naive approach to content creation might be to refer to ASSETs straight from another artists workspace. Mindbender separates between data considered work-in-progress and data exposed to others.

Workspace data is highly mutable and typically private to an individual artist.

Mutable implies transient data that is likely to change at any given moment.
Private implies personal, highly irregular and likely invalid data.

Shared data on the other hand is immutable, correct and impersonal.

Immutable implies that the data may be dependent upon by other data.
Correct implies passing validation of the associated family.
Impersonal implies following strict organisational conventions.

Each ASSET reside within the top-level root directory as follows.

Hierarchy	Example

Each ASSET contain 0 or more versions which in turn contain 0 or more representations.

Hierarchy	Example

Every extraction is made into the staging directory, regardless of whether it integrates successfully or not.

Hierarchy	Example

Each directory will contain everything that did extract successfully, along with its metadata, for manual inspection and debugging.

`ls()`

Communication with the filesystem is made through ls().

ls() returns available assets - relative the currently registered root directory - in the form of JSON-compatible dictionaries. Each dictionary is strictly formatted according to three distinct “schemas” (see below).

See below for a full list of members.

Example

from mindbender import api

for asset in api.ls():
    for version in asset["versions"]:
        for representation in version["representations"]:
            pass

Schema

Available schemas are organised hierarchically, with the former containing the latter.

asset.json
- version.json
  - representation.json

`asset.json`

A unit of data

Key	Value	Description
`name`	`str`	Name of directory
`subsets`	`list`	0 or more `subset.json`

`subset.json`

A unit of data

Key	Value	Description
`name`	`str`	Name of directory
`versions`	`list`	0 or more `version.json`

`version.json`

An ASSET iteration

Key	Value	Description
`version`	`int`	Number of this VERSION
`path`	`str`	Unformatted path
`time`	`str`	ISO formatted, file-system compatible time.
`author`	`str`	User logged on to the machine at time of publish.
`source`	`str`	Original file from which this VERSION was made.
`representations`	`list`	0 or more `representation.json`

`representation.json`

A data FORMAT.

Key	Value	Description
`format`	`str`	File extension
`path`	`str`	Unformatted path

Mindbender API

pyblish-mindbender provides a stateful API.

State is set and modified by calling any of the exposed registration functions, prefixed register_*, or automatically on calling api.install().

Public members of mindbender

Member	Returns	Description
`install(host)`	`str`	Install Mindbender into the current interpreter session
`uninstall()`	`str`	Revert installation

Public members of mindbender.api

Member	Returns	Description
`ls()`	`generator`	List available assets, relative `root`
`root()`	`str`	Absolute path to current working directory
`format_staging_dir(root, name)`	`str`	Return absolute path or staging directory relative arguments
`format_shared_dir(root)`	`str`	Return absolute path of shared directory
`format_version(version)`	`str`	Return file-system compatible string of `version`
`find_latest_version(versions)`	`int`	Given a series of string-formatted versions, return the latest one
`parse_version(version)`	`str`	Given an arbitrarily formatted string, return version number
`register_root(root)`		Register currently active root
`register_host(host)`		Register currently active host
`register_plugins()`		Register plug-ins bundled with Pyblish Mindbender
`deregister_plugins()`
`registered_host()`	`module`	Return currently registered host
`registered_families()`	`list`	Return currently registered families
`registered_data()`	`list`	Return currently registered data
`registered_root()`	`str`	Return currently registered root

Host API

A host must implement the following members.

Member	Returns	Description
`ls()`	`generator`	List loaded assets
`create(name, family)`	`dict`	Build fixture for outgoing data (see instance)
`load(asset, version=-1)`	`str`	Import external data into container

Information hierarchy

Loaded data is stored in a container. A container hosts a loaded asset along with metadata used to associate assets that use other assets, such as a Wheel asset used in a Car asset.

Host data relationship

Internally, Pyblish instances and containers are distinguished from native content via an “id”. For example, in Maya, the id is a user-defined attribute.

Name	Description	Example
`pyblish.mindbender.container`	Unit of incoming data	`...:model_GRP`, `...:rig_GRP`
`pyblish.mindbender.instance`	Unit of outgoing data	`Strange_model_default`

Contract

Mindbender defines these families.

Family	Definition	Link
`mindbender.model`	Geometry with deformable topology	Spec
`mindbender.rig`	An articulated `mindbender.model` for animators	Spec
`mindbender.animation`	Pointcached `mindbender.rig` for rendering	Spec

`mindbender.model`

A generic representation of geometry.

aud Target Audience

Texturing
Rigging
Final render

req Requirements

All DAG nodes must be parented to a single top-level transform
Normals must be unlocked

dat Data

name (str, optional): Pretty printed name in graphical user interfaces

set Sets

geometry_SEL (geometry): Meshes suitable for rigging
aux_SEL (any, optional): Auxilliary meshes for e.g. fast preview, collision geometry

`mindbender.rig`

The mindbender.rig contains the necessary implementation and interface for animators to animate.

aud Target Audience

Animation

req Requirements

All DAG nodes must be parented to a single top-level transform
Must contain an objectSet for controls and cachable geometry

dat Data

name (str, optional): Pretty printed name in graphical user interfaces

set Sets

in_SEL (geometry, optional): Geometry consumed by this rig
out_SEL (geometry): Geometry produced by this rig
controls_SEL (transforms): All animatable controls
resources_SEL (any, optional): Nodes that reference an external file

`mindbender.animation`

Point positions and normals represented as one Alembic file.

aud Target Audience

Lighting
FX
Cloth
Hair

req Requirements

None

dat Data

name (str, optional): Pretty printed name in graphical user interfaces

set Sets

None

Legend

	Title	Description
	Target Audience	Who is the end result of this family intended for?
	Requirements	What is expected of this ASSET before it passes the tests?
	Data	End-user configurable options
	Sets	Collection of specific items for publishing or use further down the pipeline.

Homework

With an understanding of this asset creation pipeline, here are some suggestions for what to explore next.

Difficulty	Task	Description
	Loader tooltip	Pick any of the available data from an asset, and add it to the tooltip of the item.
	Shots	Think of and document how the kind of workflow exemplified here - (1) creating, (2) publishing and (3) loading assets - can be applied to a shot- and task-based pipeline.
	A Creator Help Page	The creator interface is very limited. Try including a help-page and picture alongside each family illustrating what each are for.
	Loader thumbnails	The loader interface is even more limited, with room for expansion. Consider how you might go about extracting a thumbnail alongside each asset, and add it to the interface.

Contributing

Rules for Maya:

Any function altering the currently selected nodes must restore the previous selection upon completion.

Fork Mindbender, expand upon the graphical user interfaces and either make it your own or submit a pull-request to have it merge with the official PROJECT.

For more information on this, contact me and let’s have a conversation!

Prerequisities

Target Audience

Introduction

Install

Usage

How to read this guide

Description

Batteries

Creator

Loader

API

Terminology

Terminal API

mb.bat

Filesystem API

Information Hierarchy

Workspace, Stage and Share

ls()

Schema

asset.json

subset.json

version.json

representation.json

Mindbender API

Host API

Contract

mindbender.model

mindbender.rig

mindbender.animation

Homework

Contributing

`mb.bat`

`ls()`

`asset.json`

`subset.json`

`version.json`

`representation.json`

`mindbender.model`

`mindbender.rig`

`mindbender.animation`