JMX :: Javascript Multimedia Crossover

JMX is a data-flow engine used to manipulate multiple media sources like audio and video signals, scripted in Javascript (V8 engine) and with a minimal control GUI.

Entities can be created to connect OS specific interfaces to anything that can be programmed in scripts.

For instance, JMX can run the signal of a webcam into filters and overlaying them with processing-js animations. And much more.

JMX is written in Objective-C released as free and open source (GNU GPLv3) and makes use of the Quicktime multimedia framework on Apple computers (OSX Darwin): CoreAudio, CoreVideo, CoreGraphics, CoreImage.

History

JMX takes birth from the FreeJ experience, growing as an independent codebase written and maintained by Xant.

GNU/Linux users should direct their interest to FreeJ because JMX code is so far too dependent from Apple/OSX.

FreeJ is the GNU/Linux codebase of visions shared among fellow developers and friends, maintained by Jaromil and gathering through the years contributions by Xant himself, Kysucix, Mr.Goil, Fred_99, Shammash and Godog, Caedes, Lluis, Rama, Acracia and Andy.

Source on code.dyne.org

…and Todo

Early release, version 0.6.3.

Download binary builds for OSX

More features will be added soon, for instance streaming to Icecast is on top of the list, after sorting out some issues with Node.JS. Anyone volunteering a better GUI is welcome.

Internals

Entities

Almost everything inside JMX is an entity.

Entities can be considered as atomic processing units which communicate between each other using Signals.

An Entity usually receives signals (input) and produces signals (output).

On the (provisional) graphical interface (GUI) the input pins are left side and the output is right side, left to right.

Pins

A pin is what takes care of transporting signals.

Entities are connected by pins. Signals are sent and received through pins.

Signals

A signal transports data.

Signals are used to transport any supported data type across entities, through pins.

Scripts

The JMX engine is programmable in javascript. All entities, pins and signals can be used and manipulated from javascript.

Using scripts one can represent and control any kind of ‘graph’ of Entities, also previously created using the graphical interface (GUI).

Eacy script running in JMX runs in its own context, but can export pins. Those pins can connect scripts to other Entities even if they are generated by the GUI.

Of course a script can contain multiple Entities that communicate within the script context, using javascript libraries. Such big scripts can be considered “macro Entities” that will export pins.

Graph of internals

JS API Overview

------------------------------------
Legend:

* = Class
+ = prototype method
- = accessor
< = input pin > = output pin
------------------------------------

* Point
+ Point(x, y)
- x
- y

* Size
+ Size(width, height)
- width
- heigth

* Rect
+ Rect(x, y, widht, height)
- x
- y
- width
- height

* Pin

* Entity
    inputPins() : array of strings (available input pins)
    outputPins() : array of strings (available output pins)
    intputPin(pinName) : Pin object
    outputPin(pinName) : Pin object
    * VideoEntity
        - brightness : float
        - saturation : float
        - contrast   : float
        - alpha      : float
        - rotation   : float
        - origin     : point
        < brightness : number
        < saturation : number
        < contrast   : number
        < alpha      : number
        < origin     : number
        < saturation : number
        < contrast   : number
        < alpha      : number         > frame      : number
        > frameSize  : size

        * Movie
            + Movie(path)
            + open(path)
            + start()
            + stop()
            - fps        : float
            > fps        : number

        * Text
            + Text(string)
            - text       : string
            - fps        : float
            > fps        : number

        * Image
            + Image(path)
            + open(path)
            - fps        : float
            > fps        : number

    * VideoFilter
        + VideoFilter(filtername)
        - filter  : string

    * AudioFile
        + AudioFile(path)
        + open(path)
        + start()
        + stop()
        > sample : audio

    * VideoOutput
        + VideoOutput(width, height)
        - width  : int
        - height : int
        < frame        : image
        < frameSize    : size         > currentFrame : image

    * AudioOutput
        < sample        : audio         > currentSample : audio