Shapes and textures

Basic geometric shapes, images and videos.

Most Punctual patterns are built from a geometric shape. There are a bunch of functions that allow to draw the most basic shapes. Alternatively, an image or video can be used as a starting point for a pattern.

Basic shapes

As many things in Punctual, there are two versions of the functions dedicated to create basic shapes: combinatorial, and pair-wise.

The combinatorial functions to create basic shapes are circle, rect, hline, vline, iline and line, while their pair-wise versions are circlep, rectp, hlinep, vlinep, ilinep and linep. Additionally, there is point, which doesn’t need two versions.

Many times, you can use one or the other indistinctly. For example, circle [0.2, 0.3, 0.5, 0.6] 0.1 is the same than circlep [0.2, 0.3, 0.5, 0.6] 0.1, and circle [0.2, 0.3] [0.1, 0.2] is the same than circlep [0.2, 0.3] [0.1, 0.2]. The difference is only significant when using multiple values for each argument. For example, circle [0.2, 0.3, 0.5, 0.6] [0.1, 0.2] is different from circlep [0.2, 0.3, 0.5, 0.6] [0.1, 0.2], as in the first case there would be four channels and four (overlapping) circles, and in the second one only two channels and circles, one of radius 0.1 at (0.2, 0.3) and the other of radius 0.2 at (0.5, 0.6).

Let’s see how each basic shape works, and some examples for each of them:

circle

  • circle [x,y] r: returns 1 when the fragment is inside the circle with center (x,y) and radius r.

A simple circle with center at (0.5, -0.2) and radius 0.1:

circle [0.5, -0.2] 0.1 >> add;

Basic shapes example 1

Two circles of radius 0.1, one red centered at (-0.5, -0.2) and the second one cyan centered at (0.5, 0.2):

circle [-0.5,-0.2, 0.5,0.2] 0.1 >> add;

Basic shapes example 2

Three circles of radius 0.05. Red at (0.1, 0.3), green at (-0.5, -0.2), and blue at (0.8, 0.7). Note the use of zip to combine both lists, and fit to make the circles truly circular:

x << [0.1,-0.5,0.8];
y << [0.3,-0.2,0.7];
fit 1 $ circle (zip x y) 0.05 >> add;

Basic shapes example 3

A vertically-centered circle, moving along the x axis. Color oscillates from red to purple:

co << [0.8, 0, unipolar $ osc 0.3];
c << fit 1 $ circle [osc 0.1 ,0] 0.2;
c*co >> add;

Basic shapes example 4

A distorted circle, with center coordinates varying depending on the fragment being evaluated. The center is defined in polar coordinates. The radius (of the center) is the radius of each fragment whose radius is less than 0.5. The angle (of the center) is the angle of the fragment modulus 0.1, multiplied by 10·π, and keeping the sign. Note that ft%0.1 is a number between 0 and 0.1, and when multiplied by 10·π, the result is a number between 0 and π. Then adding the sign, it results in a number between -π and π, covering the whole circumference:

r << (fr<0.5)*fr;
t << (sign ft)*10*pi*(ft%0.1);
fit 1 $ circle (rtxy [r, t]) 0.04 >> add;

Basic shapes example 5

point

  • point [x,y]: returns 1 when the fragment coordinates are very similar to (x,y). A point is really a circle with a small predefined radius.

A single point at (0,0):

point 0 >> add;

Basic shapes example 6

A set of points that define two mathematical functions. The first one, in red, is the graphic representation of y=x^2. The second one, in cyan, is the representation of y=sin 4πx. For each fragment, its (fx,fy) coordinates are taken, and it will be painted red only if (fx,fy) are near enough to (fx, fx^2), and cyan only if (fx,fy) are near enough to (fx, sin 4π·fx).

point [fx, [fx*fx, sin' (4*fx*pi)]] >> add;

Basic shapes example 7

rect

  • rect [x,y] [w,h]: returns 1 when the fragments coordinates are inside a rectangle of width w, height h, and centered at (x,y).

A rectangle centered at (0.5, 0.2), of dimensions 0.1 horizontal and 0.3 vertical:

rect [0.5, 0.2] [0.1, 0.3] >> add;

Basic shapes example 8

In the next audio-reactive example, we build a rectangle shape by subtracting one rectangle to a slightly bigger rectangle. The size of the rectangle depends on the incoming low frequencies.

Then, we apply some basic transformations: spin, tile and move to create a pattern of moving rectangles. We use hsv color space to keep the color changing without losing intensity. Finally, we add a good amount of feedback to complete the pattern:

r1 << rect 0 $ 2*ilo;
r2 << zoom 0.95 r1;
r << move [saw (cps/4), 0] $ tile [8,4] $ spin 0.25 $ r1-r2;
c << hsvrgb [saw (cps/4), 1, 1];
r*c >> add;
0.98 * fb fxy >> add;

Basic shapes example 9

hline

  • hline y w: returns 1 when the fragments coordinates are inside a horizontal line at vertical coordinate y and width w.

A single horizontal line, at height 0.5 and width 0.01:

hline 0.5 0.01 >> add;

Basic shapes example 10

A total of six horizontal lines, moving by pairs up and down. Each pair have a line wider than the other, giving a neon-like impression:

y << 0.5*tri [0.1,0.13,0.14];
hline y [0.01,0.03] >> add;
0.9*fb fxy >> add;

Basic shapes example 11

In this example, we recreate the effect of walking towards the horizon in very old computer games. This is done by drawing a set of horizontal lines, which are closer to each other the nearer they are to the horizon. Lines move downwards, faster as they are closer to the camera. In reality, each line moves only through a little portion of the screen, ending its travel just when the next line begins its, and starting again, giving the impression that the movement is continuos and that new lines are created on the horizon, and old lines disappear when they reach the bottom of the screen.

Lists l1 and l2 define the beginning and ending horizontal position of each line. o is the oscillator used to move all the lines. Then, y computes the position of each line at a particular moment. Note how y is the linear interpolation between l2 (when o is 0) and l1 (when o is 1). c defines the color, and h the lines (note the use of mono to keep all the lines on a single channel). Finally, we multiply h by c to give the lines the desired color.

This example is best viewed in Estuary, selecting QHD as resolution, to avoid aliasing.

l1 << (-1)/[1.2,1.4..6.2];
l2 << (-1)/[1,1.2..6];
o << unipolar $ saw 3;
y << o*l2 +: (1-o)*l1;
c << [0, abs fy, abs fy];
h << mono $ hline y 0.001;
h*c >> add;

Basic shapes example 12

In the next example, we convert horizontal lines into spirals by spinning them.

Let’s first understand this simplified version:

l << mono $ hline 0 0.05;
fit 1 $ spin (fr/20) l >> add;

Basic shapes example 13

Note how the line is slightly curved, but more curved as fr is bigger. Now, if we spin it by fr instead of fr/20, the curvature is much bigger, to the point where part of the line is outside the screen. Now, try to change the line’s vertical position. Instead of 0, try for example, 0.2 and 0.6. Note how the curvature changes when moving the line.

Now, let’s go for the whole version. We start by creating 6 vertical positions in y. Note that each one of them is created by multiplying two oscillators, so the movement of each one of them is somewhat irregular. We use these positions to define 6 horizontal lines and store them in l.

Then, we spin each of them by a multiple of fr. This effectively curves the lines, as seen before, but for a huge amount, to the point where each line gives several laps.

Finally, we define the color in c, with a red component that increases as the fragment is farther from the origin and a blue component that is largest in the upper right and bottom left parts of the screen.

y << (osc [0.03, 0.05]) * (osc [0.054, 0.023, 0.039]);
l << mono $ hline y 0.05;
s << fit 1 $ spin (fr*6) l;
c << [fr, 0, unipolar $ fx*fy];
s * c >> add;

Basic shapes example 14

vline

  • vline x w: returns 1 when the fragments coordinates are inside a vertical line at horizontal coordinate x and width w.

A single vertical line at position 0.5 and width 0.01:

vline 0.5 0.01 >> add;

Basic shapes example 15

Ten vertical sinusoidal lines move downward in the next example. The core pattern defining the horizontal position of each fragment is sin' (fy*20+pi*saw 0.5). Without the oscillator, it would be sin' (fy*20), producing stationary sinusoidal lines. The addition of pi*saw 0.5 introduces horizontal movement to each fragment. The oscillators goes from to π, completing a full lap, resulting in continuous horizontal movement and giving the impression of downward movement:

tile [10,1] $ vline (sin' (fy*20+pi*saw 0.5)) 0.05 >> add;

Basic shapes example 16

In the next example, we create a single vertical line and use feedback to avoid erasing the screen, therefore drawing a pattern as the line rotates and changes color.

Color is defined in the HSV color-space. Hue (color) is controlled by an oscillator, thus going through all tonalities, while saturation and value (brightness) are constant. Note how we add an alpha channel of 0.6 (using the ++ operator) to the color once we have translated it to RGB. Using transparency avoids getting a completely white screen when using high values of feedback:

c << hsvrgb [unipolar $ osc 0.08,0.8,1]++0.6;
l << spin (saw 0.103) $ vline 0 0.001;
l*c >> blend;
fb fxy >> add;

Basic shapes example 17

iline

  • iline [x0, y0] [x1, y1] w: returns 1 when the fragment coordinates are inside the infinite line defined by the points (x0,y0) and (x1,y1) with width w.

The line that pass through (-0.2, -0.3) and (0.4, 0.5):

p << [-0.2,-0.3];
q << [0.4,0.5];
iline p q 0.001 >> add;

Basic shapes example 18

A good example of the use of iline is the default code in the standalone version of Punctual:

x1 << osc $ 0.11*[1,2]; y1 << osc $ 0.08/[3,4];
x2 << osc $ 0.06/[5,6]; y2 << osc $ 0.04*[7,8];
lines << mono $ iline [x1,y1] [x2,y2] 0.002;
col << hsvrgb [osc 0.11,0.5 ~~ 1 $ osc 0.12, 1];
mask << prox 0 ** 8;
a << fit 1 $ lines * col  * mask;
gatep 0.1 (maxp a (fb fxy * 0.98)) >> add <> 5

Basic shapes example 19

Here, x1, x2, y1 and y2 have two channels each, all of them oscillating from -1 to 1 but at different frequencies. When combined in the iline bit, this yields a total of 16 lines, due to the combinatorial nature of Punctual. These lines are then put together in a single channel by using mono.

Color is defined in the HSV color space. Hue oscillates through all possible values, saturation moves from 0.5 to 1, and value is fixed at 1.

mask is used to attenuate the result at the borders. As the name implies, it acts as a mask, multiplying each fragment value by a number from 0 to 1. See prox, in Polar coordinates for an extended explanation.

Finally, the result is mixed with the feedback in the following way: first, for each fragment, the maximum between the current frame result and the last frame result (multiplied by 0.98) is taken. This way, getting a too bright image is avoided, as could happen if the feedback was directly added. Finally, gatep is used to erase all fragments whose values are near 0. Note that if ypou remove this part, due to rounding errors, there are a lot of fragments that are never completely erased (for example, note that multiplying 0.98 by 0.001 and then rounding to two decimals results in 0.001 again, so applying this calculation each frame won’t let to a completely black fragment).

Also note, that the final result is sent to the screen with a transition time of 5 seconds. See Crossfading.

line

  • line [x0, y0] [x1, y1] w: returns 1 when the fragment coordinates are inside the segment defined by the points (x0,y0) and (x1,y1) with width w.

A single segment joining points (0.5, 0.2) and (-0.3, 0.8):

line [0.5, 0.2] [-0.3, 0.8] 0.001 >> add;

Basic shapes example 20

In the next example, we create a set of segments using the coordinates x and y. By deriving y from x and utilizing both variables both for the first and second points of each segment, we achieve two benefits: the expression becomes more concise and faster to write, and maintain a geometric relationship with each other. Then, we duplicate the segments using the two-channels spin, increasing the pattern’s symmetry.

The second step is using feedback to build the complex patterns that arise when running the code. Here, it’s worth noting that even with an alpha channel set to 1, it contributes to reducing the overall brightness of the pattern.

In the final step, we manipulate the feedback to create intricate patterns. The zoom operation introduces irregular zooming based on a formula involving fr, resulting in a gaseous-like movement. As in the last example, we use gatep to get rid of artifacts resulting from rounding errors.

This is example is best viewed in Punctual standalone, or in Estuary by setting a high resolution and frame rate.

x << osc [0.1,0.125..0.2];
y << x*osc 0.1;
l << mono $ spin (saw [0.03, -0.03]) $ line (zip x y) (zip y x) 0.001;
c << hsvrgb [saw 0.3, 0.8, 0.8]++1;
fit 1 $ l*c >> blend;
gatep 0.2 $ spin 0.001 $ zoom (0.98~~1.02 $ fr-0.6) $ 0.99*fb fxy >> add;

Basic shapes example 21

When the number of given coordinates does not match, Punctual fills the gaps automatically.

For instance, consider the example line [0,0.1,0.2] [0,0.5] 0.01 >> add;. Here, there is a missing coordinate in the first argument and a missing point in the second one.

Punctual completes the missing coordinate in the first argument by repeating the last number. This behavior mirrors how circle 0 0.1 represents a circle centered at (0,0). Therefore, the example can be rewritten as line [0,0.1,0.2,0.2] [0,0.5] 0.01 >> add;.

Similarly, the missing point in the second set of coordinates is filled by repeating the last point. Thus, the updated example becomes line [0,0.1,0.2,0.2] [0,0.5,0,0.5] 0.01 >> add;. In this case, the result consists of two line segments: the first from (0, 0.1) to (0, 0.5) and the second from (0.2, 0.2) to (0, 0.5)."

linep

  • linep: the pairwise version of line.

Note the difference between these two expressions:

line [0,0, 0.5,0.5] [0,0.5, 0.5,0] 0.01 >> add;

Basic shapes example 22

linep [0,0, 0.5,0.5] [0,0.5, 0.5,0] 0.01 >> add;

Basic shapes example 23

The first one is combinatorial, and each point from one argument will be combined with all other points from the other. Here we have a total of 4 segments: (0, 0) to (0, 0.5), (0, 0) to (0.5, 0), (0.5, 0.5) to (0, 0.5), and (0.5, 0.5) to (0.5, 0).

The second one is pairwise, so each point will only be matched with the corresponding point on the other argument. We have only two segments: (0, 0) to (0, 0.5), and (0.5, 0.5) to (0.5, 0).

Multi-lines

chain

  • chain [x1,y1,x2,y2,x3,y3...] [w]

chain draws multiple chained segments, using the provided coordinates. All segment will have w width, and each one will be in a separate channel.

fit 1 $ chain [0,0, 0.5,0, 0.5,0.5, -0.5,0.5, -0.5,-0.5, 1,-0.5, 1,1] 0.01 >> add;

Multi-lines example 1

Here, starting from (0,0), a first segment is drawn to (0.5,0), then a second one from (0.5,0) to (0.5,0.5) and so on.

Let’s expand out spiral. We need to create a list of numbers following the pattern [0,0,0.1,0,0.1,0.1,-0.1,0.1,-0.1,-0.1,0.2,-0.1,0.2,0.2] and so on, until 1,1. Obviously, we don’t want to write all this numbers by hand, so we need to find some kind compact way to express this numbers set.

Let’s treat x and y coordinates separately (we know we can join them at the end with zip).

X-coordinates follow the pattern: [0,0.1,0.1,-0.1,-0.1,0.2,0.2,-0.2,-0.2...]. Except for the leading 0, we have the same number repeated 4 times, the first and second are positive, and the third and forth negative.

We can begin with a seed with all the numbers: seed << [0.1,0.2..1]. Now, we need to create four copies of each number, with the appropriate sign each one. This is easily done by using the combinatorial side of Punctual: coord << seed*[1,1,-1,-1];. Finally, we add a leading 0, and store the result in xs.

Let’s do the y-coordinates next. Y-coordinates follow the pattern [0,0,0.1,0.1,-0.1,-0.1,0.2,0.2,-0.2,-0.2...]. Apart from the two leading 0, we have the exact same pattern than before, so we can use the same coord variable as before, and add the two 0 by hand into ys.

Finally, we use zip to alternate X and Y-coordinates, and chain to draw the resulting spiral:

seed << [0.1,0.2..1];
coord << seed*[1,1,-1,-1];
xs << 0++coord;
ys << [0,0]++coord;
fit 1 $ chain [zip xs ys] 0.01 >> add;

Multi-lines example 2

chainp

  • chainp: is the pairwise equivalent to chain. When specifying more than one line and width, chain will combine them in all possible ways, while chainp will pair each segment with a width.

In the following example, this chainp feature is utilized to hide half of the segments of the chain by specifying a width of 0.

The example begins by defining the chain points in polar coordinates. The radius r is determined by three oscillators, organizing all the segments into three circles. The angle t is defined to create segments that will spin according to a saw oscillator. Multiplying the oscillator ensures that the distance between segments remains non-constant, causing them to continuously group and move apart. These coordinates are then converted into Cartesian coordinates.

The subsequent step involves creating the final shape by connecting all the points together in a chain. chainp is used here instead of chain, resulting in alternating segment widths of 0.008 and 0.

The following section deals with color. The objective is to alternate between three colors, but not the default red, green, and blue. Each base color is transformed, resulting in a, b, and c. mono is necessary as each color component is composed of multiple channels in sh. The sum of a, b, and c is then sent to the output. Additionally, note that the new color base includes an alpha channel to smooth the result of the feedback added in the final step.

r << unipolar $ osc [0.14, 0.19, 0.23];
t << [0,0.9..12]*saw 0.03;
coords << rtxy [r, t];
sh << fit 1 $ spin (saw [0.16, -0.16]) $ chainp coords [0.008,0];
a << [0.8, 0, 0.8, 1]*(mono $ rgbr sh);
b << [0.8, 0.3, 0, 1]*(mono $ rgbg sh);
c << [0, 0, 0.9, 1]*(mono $ rgbb sh);
a+:b+:c >> blend;
gatep 0.1 $ 0.98*fb fxy >> add;

Multi-lines example 3

mesh

  • mesh [x1,y1,x2,y2,...] [w]: returns 1 when current fragment is within w of a mesh of lines that go between every pair of (x1,y1), (x2,y2) etc; otherwise 0.

As mesh combines points in all possible combinations, it will fast lead to very complex patterns, hard to handle by the GPU.

In the next example, we utilize the mesh function to construct an intricate pentagonal shape based on polar coordinates. The variable t defines a set of five angles uniformly distributed along the perimeter, while r specifies two radii for each point. By combining these points using mesh, we create the edges of the pentagonal shape.

To introduce motion, we apply the spin function and gradually increase the spinning velocity over time. This is achieved by modulating the speed parameter s using an exponential function. The exponential argument ranges from -5 to 5 over 100 seconds, resulting in s increasing from nearly 0 to 148. As a result, the pentagonal shape spins faster and faster as time progresses.

Finally, we add a feedback effect to create a trailing purple tail behind the spinning shape. The feedback is applied by zooming into the feedback buffer and tinting the color to purple. This enhances the visual impact of the spinning motion, resulting in a dynamic and visually captivating pattern.

t << pi*bipolar [0,0.2..1];
r << [0.9,0.2];
m << mesh (rtxy [r,t]) 0.001;
s << exp ((etime%100-50)/10);
fit 1 $ spin s $ mono m >> add;
[0.4,0,0.5]*(mono $ zoom 1.03 $ fb fxy) >> add;

Multi-lines example 4

meshp

  • meshp: is the pairwise version of mesh. meshp still combines specified points in all possible ways, but will pair each segment with a single width, while mesh will pair each segment with all specified widths.

lines

  • lines [x0, y0, x1, y1] w: this function is like line but it takes all the points coordinates in a single list instead of a list for each point.

In this example, line and lines are equivalent:

p1 << [0, 0];
p2 << [0, 0.5];
q << [0.5, 0.5];
line (p1++p2) q 0.004 >> add;
lines (p1++q++p2) 0.004 >> add;

Multi-lines example 5

See how in lines we have reorganized the coordinates.

In the next example, we need to use linep to achieve the same result as lines:

p1 << [0, 0]; p2 << [0, 0.5];
q1 << [0.5, 0.5]; q2 << [-0.5, -0.5];
linep (p1++p2) (q1++q2) 0.004 >> add;
lines (p1++q1++p2++q2) 0.004 >> add;

Multi-lines example 6

lines is useful in cases where our points’ coordinates are all stored in a single list. In the next example, we take a sample of 64 frequency intensities and store them in f.

All coordinates in f are positive. When multiplied by [-1,1], we get a second set of the same coordinates, but negative.

In the next step, we define l by calling lines with p. This creates a set of segments that move according to the captured audio frequencies. As each segment is defined by coordinates that respond to similar frequencies, they tend to be short and distribute along the diagonal.

Next, we apply a 3-way symmetry by using step. As our pattern already had a symmetry, this creates hexagonal patterns.

In the last step we add feedback. While the pattern lines are white, the feedback is blue.

The resulting pattern creates audio-responsive shapes that resemble snow crystals due to the hexagonal symmetry and the chosen colors.

f << ifft $ [0..64]/64;
p << [-1,1]*f;
l << lines p 0.001;
mono $ fit 1 $ zoom 2 $ spin [0, 1/3, 2/3] l >> add;
(zoom 0.98 $ mono $ fb fxy)*[0, 0.3, 0.5] >> add;

Multi-lines example 7

linesp

  • linesp: pairwise version of lines.

This variation of the last example utilizes two widths for the lines.

With linesp, half of the lines will have a width of 0.001, and the other half will have a width of 0.01. If we used lines instead, all the segments would be duplicated with both widths, and only the wider ones would be visible.

f << ifft $ [0..64]/64;
p << [-1,1]*f;
l << linesp p [0.001, 0.01];
mono $ fit 1 $ zoom 2 $ spin [0, 1/3, 2/3] l >> add;
(zoom 0.98 $ mono $ fb fxy)*[0, 0.3, 0.5] >> add;

Multi-lines example 8

ilines

  • ilines [x0, y0, x1, y1] w: this function is similar to iline but takes all the point coordinates in a single list instead of a list for each end.

In this example, a set of 12 oscillators of different frequencies is stored in o. Then, they are used as coordinates for a set of 3 lines (4 coordinates per line).

Afterwards, the lines are deformed using spin: each fragment is rotated an amount that depends on its distance to the origin and the oscillators in o. As o is a multichannel signal, each line transforms into a set of 12 curves.

A dimmed version of the resulting lines is sent to the output along with the feedback. max is used to merge both signals (the current frame and the last one) to prevent excessive brightness.

o << osc $ 1/[9,10..20];
l << (mono $ spin (fr*o) $ ilines o 0.002) / 4;
gatep 0.1 $ max l $ 0.97*fb fxy >> add;

Multi-lines example 9

ilinesp

  • ilinesp: pairwise version of ilines.

This variation of the previous example utilizes different widths for each line:

o << osc $ 1/[9,10..20];
w << [2,8,24]/1000;
l << (mono $ spin (fr*o) $ ilinesp o w) / 4;
gatep 0.1 $ max l $ 0.97*fb fxy >> add;

Multi-lines example 10

Images and videos

Punctual allows you to incorporate external images and videos as textures for your patterns. However, there are certain limitations:

Limitations

  • Due to security reasons, web pages cannot access local files directly. To use local files, you may employ workarounds as explained below. Keep in mind that using local files is most suitable for solo performances; for collaborative jamming in Estuary, shared files must be accessible to all participants and hosted on a web server.

  • When fetching images and videos from an external web server, CORS (Cross-Origin Resource Sharing) must be enabled on that server. CORS is a security feature that prevents web pages from making requests to a different domain than the one serving the web page unless explicitly permitted. Note that popular platforms like YouTube or Vimeo do not allow this, but Wikimedia Commons does. Alternative options are discussed below.

  • To utilize the webcam, you only need to grant permission in your browser. However, each participant in a jam will see their own webcam feed.

img

  • img "https://url-to-image-file": Fetches a texture created from the specified image file, represented as a red-green-blue (3-channel signal). By default, the image will be stretched to fit the screen, potentially distorting its proportions. Refer to fit and aspect in the Cartesian coordinates section to adjust this behavior.

In this example, we start with an image from Wikimedia Commons and utilize three oscillators to blend its RGB channels in varying proportions over time. This creates the illusion of different tints being applied to the image:

i << img "https://upload.wikimedia.org/wikipedia/commons/9/9c/Catedral_de_Salzburgo%2C_Salzburgo%2C_Austria%2C_2019-05-19%2C_DD_30-32_HDR.jpg";
o1 << unipolar $ osc 0.13;
o2 << unipolar $ osc 0.19;
o3 << unipolar $ osc 0.3;
r << o1*rgbr i+(1-o1)*rgbg i;
g << o2*rgbg i+(1-o2)*rgbb i;
b << o3*rgbb i+(1-o3)*rgbr i;
[r, g, b] >> add;

Images and videos example 1

vid

  • vid "https://url-to-image-file": Similar to img, but with a video file.

In this example, we begin with a video from Wikimedia Commons, adding an alpha channel to it. Then, we introduce a colored and slightly distorted version of the feedback, resulting in an intriguing effect:

v << vid "https://upload.wikimedia.org/wikipedia/commons/transcoded/5/5f/Steamboat_Willie_%281928%29_by_Walt_Disney.webm/Steamboat_Willie_%281928%29_by_Walt_Disney.webm.1080p.vp9.webm";
v++0.8 >> blend;
f << mono $ tile [1.2,1.1] $ fb fxy;
f*[1,0,0.2,0.6] >> blend;

Images and videos example 2

cam

  • cam: captures the image from the webcam as an RGB texture.

When using cam, make sure to grant permission to your web browser to access the webcam.

In a collaborative setting like a jam in Estuary, each participant will view their own webcam, resulting in diverse outputs for each participant.

In the following example, transformations are applied to the webcam source image to create a kaleidoscopic effect.

c represents the original cam stream. From it, c2 is created, which is a horizontally mirrored version of c. t stores a tiled version of the average between c and c2. Subsequently, distortion is applied and stored in z by zooming t with a radius-dependent factor. Finally, the resulting image is duplicated, and each copy spun in opposite directions, enhancing the kaleidoscopic effect:

c << cam;
c2 << setfx ((-1)*fx) c;
t << tile 4 $ (c+:c2)/2;
z << zoom (1+fr*8) $ zoom 0.2 t;
spin (saw [0.02,-0.02]) $ z/2 >> add;

Using your own images and videos

To use your own images and videos, you must host them somewhere. This section only points out some ideas but doesn’t intend to be a complete tutorial on how to set up each solution.

There are several possibilities:

  • Using local resources (on your own computer).

As mentioned earlier, this approach is only suitable for solo performances.

To enable Punctual/Estuary to access your local resources, you need to use a local web server. The easiest way is to use a web server that doesn’t require any configuration, such as the ones included in some programming languages. The chosen web server must support enabling CORS.

In this case, I recommend using the web server included in Node.js. Install it by following the official instructions.

You’ll also need to allow Estuary/Punctual to access resources on non-encrypted HTTP pages (easier to configure, and no need to encrypt access to your own computer). In Chrome, after your page is loaded, you’ll see a configuration button on the left side of the address bar. Click on it, look for insecure content, and select allow.

Once this is done, open a terminal, navigate to the directory where your files are stored, and type npx serve --cors. This should make it.

  • Hosting resources on GitHub Pages.

GitHub Pages enables CORS by default, making it a suitable place to store resources. Note that all files stored this way are accessible to anyone.

To use GitHub Pages, create a GitHub repository and upload the images/videos you’d like to use. Navigate to Settings -> Pages and select main as the branch. In a few minutes, your page will be ready, and its address will be visible at the top of the page. It will be something like username.github.io/projectname/.

  • Hosting resources on your own server.

This is an advanced solution, and you need to have knowledge about using and configuring web servers.

The idea is to use your own server (for example, a cheap VPS is a good solution for this) and configure Apache or Nginx to serve your files.

Then, you should enable HTTPS (for example, using Let’s Encrypt) and enable CORS. In Apache, you can achieve this by adding the following lines in your virtual server configuration:

<IfModule mod_headers.c>
  Header set Access-Control-Allow-Origin "https://your-domain"
</IfModule>