Concepts

These are some important ideas behind Punctual which are crucial to gain a good understanding of how all of this works.

In my own experience, the latter concepts can be difficult to grasp.

If you are new to Punctual, it’s advisable to focus only on the first three sections here, or even jump directly to the getting started section and come back later.

Coordinate System

Punctual uses a coordinate system where both the x and y axes span from -1 to 1 across the visible screen. Specifically, the x-axis ranges from -1 at the left side of the display to 1 at the right side, while the y-axis ranges from -1 at the bottom side of the display to 1 at the top side.

This has the following implications:

• The origin [0,0] is conveniently situated at the center of the screen.
• It’s immediate to use functions that go from -1 to 1, like the sine or cosine, to modulate coordinates. See Oscillators.
• Fragment dimensions are relative to the aspect ratio of the window they are displayed in. So, for example, in a full-screen window on a landscape monitor, a fragment is wider than it is high, so many shapes seem disproportionate: circle [0,0] 1 is an oval, or rect [0,0] [0.5,0.5] is not a square. If you resize only the width of the window and narrow it, the circle will gradually become circular (and if you keep going, taller than it is wide). See also Coordinates and the fit function.
• As the color system uses values from 0 to 1, it’s not immediate, but easy to use coordinates to change colors, or vice versa. See the examples in the next section Output Notations and Colors.

Output Notations

A Punctual statement needs to end with an output notation in order to produce a result.

Each output notation has its own way of interpreting the values of the statement. When using many output notations, each one constitutes a separate layer, and the final result is the combination of all of them.

The add Output

There are several possible outputs for visual statements, with the most common being add.

A gray screen:

0.5 >> add;

A red screen:

[1,0,0] >> add;

A screen that gets whiter as we go from left to right:

fx >> add;

fx is the x coordinate of the current fragment’s position (see Fragments and Cartesian coordinates), and goes from -1 to 1, but color goes from 0 to 1. Here, negative values are ignored, and that’s why the right half of the screen is black. See also unipolar in Scaling values.

Note how in some examples we have sent only one value to add and in another one a list (see more on lists in Notes on Haskell) with three values.

In general, any Punctual expression is composed of a number of Channels (1 or 3 in our examples). When those channels reach the output, they are interpreted in some way, depending on the output used.

For add output:

• Three channels are interpreted as red, green, and blue intensities.
• With only one channel, all red, green, and blue intensities use the same channel.
• Two channels are interpreted as red for the first channel, and green+blue (cyan) the second channel.

Any remaining channels are interpreted in the same way. So for example, five channels would be red, green, blue, red, and cyan.

A cyan screen:

[0,1] >> add

[0,1,1] >> add

[fx+fy,0,0] >> add

The amount of red color for each fragment is computed by adding its two coordinates.

Other possibilities are:

• rgb: similar to add, but any previous layer is ignored.

• blend: this adds a fourth channel to the video output corresponding to the alpha channel.

  By default, alpha is 1, so any drawing is completely opaque. With blend you can specify another value to create transparent or semi-transparent parts.

• rgba: similar to blend, but any previous layer is ignored.

• mul: this multiplies the color of the current layer by the color of the previous layer.

Layers

When using more than one output, each one is a layer that is drawn on top of the previous one. The final result is the combination of all of them. The output of a layer determines how it is combined with the previous one.

When the output is add, the color in each expression is simply added:

[0.3, 0, 0.5] >> add;
[0, 0.5, 0.2] >> add;

is equivalent to:

[0.3, 0.5, 0.7] >> add;

fx >> add;
fy >> add;

is equivalent to:

fx+fy >> add;

[0,1,0,0.5] >> blend;
[1,0,0] >> add;

is equivalent to:

[1, 1, 0] >> add;

Note how the alpha channel in the first expression is ignored when using add, and the rest of the channels are added.

When using the mul output, the color of the current layer is multiplied by the color of the previous layer:

[1, 0.5, 0] >> add;
[0.5, 0.5, 0.5] >> mul;

is equivalent to:

[0.5, 0.25, 0] >> add;

When using the blend output, signals are mixed taking into account the alpha channel of the second signal:

[1, 0, 0.7, 1] >> blend;
[0.5, 1, 0.3, 0.2] >> blend;

The result is computed as a weighted average of the two colors, taking the alpha value of the second color as the weight. In this example, the second layer has a weight of 0.2, so the first one has a weight of 1 - 0.2 = 0.8. The result is then:

[1*0.8+0.5*0.2, 0*0.8+1*0.2, 0.7*0.8+0.3*0.2, 1*0.8+0.2*0.2] = [0.9, 0.2, 0.62, 0.84]

Therefore, the blended color is [0.9, 0.2, 0.62, 0.84].

In a more general form, if we have two colors defined as:

[r1, g1, b1, a1] >> blend;
[r2, g2, b2, a2] >> blend;

The resulting blended color is computed like so:

[
r1*(1-a2)+r2*a2,
g1*(1-a2)+g2*a2,
b1*(1-a2)+b2*a2,
a1*(1-a2)+a2*a2
] >> blend;

In this computation, each component in the first color is weighted by 1-a2, and in the second color by a2, being a2 the alpha channel component of the second color.

If the first output lacks an alpha channel, it is assumed to be 1:

[1,0,0] >> add;
[0,1,0,0.5] >> blend;

is equivalent to ([1*0.5+0*0.5, 0*0.5+1*0.5, 0*0.5+0*0.5, 1*0.5+0.5*0.5] = [0.5, 0.5, 0, 0.75]):

[0.5,0.5,0,0.75] >> blend;

As stated before, rgb and rgba outputs ignore any previous layer:

[1,0,1] >> add;
[0,1,0] >> rgb;

is equivalent to:

[0,1,0] >> rgb;

And:

[1,0,1] >> add;
[0,1,0,0.5] >> rgba;

is equivalent to:

[0,1,0,0.5] >> rgba;

Note that there exist functions with the same name as some output notations: add, blend and mul. These functions can be used to combine channels using the same system as the corresponding output.

For example:

c1 << [1, 0, 0.7, 1];
c2 << [0.5, 1, 0.3, 0.2];
blend $ c1++c2 >> blend;

is equivalent to:

[1, 0, 0.7, 1] >> blend;
[0.5, 1, 0.3, 0.2] >> blend;

See Combining Channels.

When applying this in Estuary, using more than one cell to make visuals, cells are drawn in a left-right up-down order.

Bindings

When expressions get long and complex, you can assign a name to a part in order to simplify its readability.

For example:

c << circle 0 0.3;
move [0.5,0] c >> add;

Here, we give the circle the name c and refer to it on the next line. The result is equivalent to move [0.5,0] (circle 0 0.3) >> add, that is, a circle with a radius of 0.3 and its center at the [0.5,0] coordinates.

Note that when using more than one statement, you need to end each one (except the last one) with a semicolon.

Now, here is the catch. c is not a variable in the sense of popular programming languages. It’s more like a definition or a binding to an expression, and this has some implications.

c isn’t a circle it defines a circle. So you can use it more than once to create several circles:

c << circle 0 0.3;
move [0.5,0] c >> add;
move [-0.5,0] c >> add;

This is perfectly valid and creates two identical circles, one at [0.5,0] and the other one at [-0.5,0].

Fragments

A fragment is essentially a pixel in OpenGL (although a single pixel can be related to several fragments).

What’s important here is that Puntual uses WebGL under the hood, which is a web implementation of OpenGL. When you run a Punctual statement, the code is parsed, compiled, and converted to a shader that can be directly executed by the graphics card.

Graphics cards nowadays have multiple processors and can compute the same expression in a large number of fragments simultaneously, which explains how complex graphics can be rendered in such an efficient way.

And once again, this has implications on how we think the code we write.

For example, when you look up the definition of circle on the official Punctual documentation you find this: circle [x,y,...] [d] -- returns 1 when current fragment [is] within a circle at x and y with diameter d.

What that means is, as each fragment is processed independently from the others, all fragments whose coordinates are inside the circle will be painted white, while all the other fragments will be painted black. Of course, other operations in you code can change this, and that’s why the definition says “returns”.

This way of thinking may help to understand and build Punctual expressions.

abs (fx/fy) >> add;

In this example, for each fragment, we take its x-coordinate and its y-coordinate, divide them, and take the result as a positive number. This will give a number greater or equal to 0 for each pixel.

When fx is greater or equal than fy (ignoring sign), the result is a number greater or equal than 1, so that fragment is white. All the other fragments get some gray color, darker as the ration between fx and fy decreases, and black when fx approaches 0.

Graphs

A graph is any Punctual expression that can be converted into a shader. Most simple graphs are an integer number or lists of integer numbers. All graphics related functions get graphs as arguments and return graphs. This way, Punctual offers a big flexibility on what can be used as an argument to a function, or what expression can be combined between them, as all are graphs.

circle [0,0] 0.3 + 0.3 >> add;
circle [0,0] 0.5 * abs fx >> add;
circle [0,0] (1/fx) >> add;
circle [circle [0,0] 0.2,0] 0.3 >> add;

All these examples are valid Punctual expressions, although it may seem that they are combining incompatible types.

circle [0,0] 0.3 + 0.3 >> add;: we add 0.3 to all fragments, so fragments inside the circle get a value of 1.3, which is white, and all the other a value of 0.3, which is a dark gray.

circle [0,0] 0.5 * abs fx >> add;: all fragment values are multiplied by its x-coordinate without sign. Fragments outside the circle are black anyway. Fragments inside the circle get their 1 value from the circle multiplied by their unsigned x-coordinate. As this is a value from 0 to 1, the result is a gray gradient with the shape of the circle.

circle [0,0] (1/fx) >> add;: for each fragment, its x-coordinate is inverted, then we look if the fragment is inside a circle from the origin with radius this value, and if it is, we paint it white, otherwise black. For negative fx the result is always black, as it’s impossible to be inside a circle with negative radius.

circle [circle [0,0] 0.2,0] 0.3 >> add;: for each fragment, we first check if it is inside a circle at the origin with radius 0.2. This can give a 0 or a 1. Then, we look if the fragment is inside the resulting circle, which can have its center on [0,0] or on [1,0] depending on the previous result. Overall, fragments that are near the [0,0], get a 1 on the first operation, and then a 0 on the second, as they are far away from the [1,0] coordinate. Fragments that are at a distance from the [0,0] between 0.2 and 0.3 get a 0 on the first operation and a 1 on the second. Finally, fragments that are at a distance from the origin greater than 0.3 get a 0 on the first operation, and a 0 on the second operation. That explains the crown shape of the result.

Channels

All Punctual statements produce a number of channels. For example, 0.5, circle [0,0] 0.3 or fx are 1-channel expressions, while [0.5,0.2,0.8] is a 3-channel expression.

Note how in the official reference documentation most functions have ellipsis on their arguments. That means we can pass any number of arguments, and that this will increase the number of channels on our expression.

circle [-0.5,0,0.5,0] 0.2 >> add;

In this example, we are providing two center points for the circle, creating a two-channel signal. As explained in Output notations, when sending two channels at video, the first one is interpreted as the red channel in the RGB color space, and the second one is taken both as the green and the blue channels.

Note that circle [-0.5,0,0.5] 0.2 >> add is a valid expression, and the cyan circle will have [0.5, 0.5] as its center.

Now, as any number in our expression can be substituted by any graph, we can write:

circle [-0.5,0,0.5,0] [0.2,0.3] >> add;

How many channels has this signal? There are two centers and two radius, so in total there are 4 different combinations, so 4 channels.

In general, Punctual interprets expressions in a combinatorial way, so it creates channels for each possible combination of the input values.

Let’s try to understand the result. When add receives more than 3 channels, each group of 3 is interpreted as a RGB signal, and the rest follows the same rules explained before. In our case, we have a complete RGB group, and one more channel that will be interpreted as the whole RGB, i.e. white.

We have 4 circles:

• Center [-0.5,0], radius 0.2, color red.
• Center [-0.5,0], radius 0.3, color green.
• Center [0.5,0], radius 0.2, color blue.
• Center [0.5,0], radius 0.3, color white.

The intersection of the two first circles leads to yellow (red+green). The crown on the left is green. The blue circle on the right is invisible due to the bigger white circle.

[0.2,fy,fx] + [0.2,0.3] >> add;

Due to the combinatorial nature of Punctual, this signal has 6 channels:

• 0.2+0.2: 0.4 interpreted as global red.
• 0.2+0.3: 0.5 interpreted as global green.
• fy+0.2: increase the blue component as we go up (and decrease it on the bottom, counteracting the sixth channel).
• fy+0.3: increase the red component as we go up (and decrease it on the bottom, counteracting the first channel).
• fx+0.2: increase the green component as we go to the right (and decrease it on the left, counteracting the second channel).
• fx+0.3: increase the blue component as we go to the right (and decrease it on the left, counteracting the third channel).

With all these interactions, and considering how RGB components affect the final color output, the result is the gradient you see on the screen.

See also [Combining graphs](#Combining graphs) for more examples on how to mix distinct graphs together.