Fixing some and , we can define the inner product of two tensors by
where indicates complex conjugate, if appropriate. If and are represented with separation ranks and using vectors and , then their inner product can also be computed via
which costs operations.
With this inner product we can define length with the norm
We can also define the angle between and relative to the origin with
realizing that this may be a complex number if or is complex.
The operations necessary for the visualization will be based on this inner product, and so are essentially coordinate-free. In particular, a separable change of coordinates does not effect the vizualization.
We now consider how to visualize a set of tensors. Since is defined as a subset of we cannot visualize it all at once. Instead we can intersect it with a simpler object and visualize this intersection. Any two distinct tensors and define a line, which can be represented by
Similarly, any three tensors , , and that do not lie on a line define a plane
By intersecting with one of these simpler objects we obtain something that we can visualize.
Unfortunately, in practice the straightforward intersection described above yields too few points to plot. If we proceed by generating points in , the probability that any of them lie in the given line/plane is virtually zero. If we proceed by generating points in the line/plane then we need to test their membership in . For the sets we are interested in this membership test can fail, and at best can produce a tensor in within some small distance of the tensor we are checking.
A star-shaped set is a set such that for any scalar . Such a is connected and simply connected. Instead of asking if is on our plotting line/plane, we can ask if is on our plotting line/plane for some , and if so, plot that . Thus we can increase the number of plotted points and remove the uninteresting scalar structure. Formally, we consider tensors that are positive scalar multiples of one another to be equivalent. Locally, this is similar to restricting to tensors of norm one. (Identifying negative scalar multiples as well does not seem to be worthwhile.)
We will add further information to the visualization by including points that are in but not on the plotting line/plane and coloring them to indicate how far from the plotting plane they are. Since we identify and we measure “how far” using an angle instead of a distance.
We will describe the the case for a plane, since the case for a line is a simple modification of it. From the three points , , and that define our plane, we define the matrix
Given , its projection onto the span of , , and is given by
with
Dividing by the scalar , we obtain the tensor
which lies on the plotting plane. To measure how much failed to be on the plane, we compute the angle
An angle of zero indicates that some scalar multiple of is on the plane and an angle of indicates that is orthogonal to the span of , , and . This projection process (neglecting ) is illustrated below:
To indicate this angle in the visualization, we color the plotted point with black indicting angle nearly zero, then through a spectrum of ever-lightening colors until light yellow indicates angles near . We use the colors
and bin the angles logarithmicly by
In cases where more than one tensor projects to the same plotting point, we plot the darker of the two. Since we plot points as small discs, in cases where the discs overlap we plot the darker disc on top.
If is orthogonal to the span of , , and we will have and cannot consruct . Thus should not appear in the visualization. (Numerically one produces a white point at a location determined by roundoff errors.)
For tensors over the real numbers, our tool uses three tensors yielding a plane, so we now consider that case in more detail. We arbitrarily choose to orient the visualization so that is at and is in the direction . The angle from to to is given by
so is in the direction . A tensor is at the point
This plotting geometry is illustrated below:
For tensors over the complex numbers, our tool uses two tensors yielding a complex line. We arbitrarily choose to orient the visualization so that is at and is in the direction . A tensor is at the complex point . This plotting geometry is illustrated below: