Visualization Theory

Inner Product, Norm, and Angle

Fixing some d and M, we can define the inner product of two tensors by

\langle T, U \rangle = \sum_{i_1=1}^M \sum_{i_2=1}^M\cdots
\sum_{i_d=1}^M T(i_1,i_2,\ldots,i_d)\bar{U}(i_1,i_2,\ldots,i_d) \,,

where \bar{U} indicates complex conjugate, if appropriate. If T and U are represented with separation ranks r and s using vectors V^l_k and W^l_k, then their inner product can also be computed via

\langle T, U \rangle
=\sum_{l=1}^r \sum_{m=1}^s \prod_{k=1}^d  \sum_{i=1}^MV^l_k(i)\bar{W}^m_k(i)
=\sum_{l=1}^r \sum_{m=1}^s \prod_{k=1}^d  \langle V^l_k, W^m_k
\rangle\,,

which costs \mathcal{O}(rsdM) operations.

With this inner product we can define length with the norm

\|T\|= \sqrt{\langle T, T \rangle}
= \left(\sum_{i_1=1}^M \sum_{i_2=1}^M\cdots
\sum_{i_d=1}^M | T(i_1,i_2,\ldots,i_d)|^2\right)^{1/2}\,.

We can also define the angle between T and U relative to the origin with

\angle(T,U)= \arccos\left(\frac{\langle T, U \rangle}{\|T\|\,\|U\|}\right)\,,

realizing that this may be a complex number if T or U is complex.

The operations necessary for the visualization will be based on this inner product, and so are essentially coordinate-free. In particular, a separable change of coordinates does not effect the vizualization.

Intersections

We now consider how to visualize a set S of tensors. Since S is defined as a subset of R^{M^d} we cannot visualize it all at once. Instead we can intersect it with a simpler object and visualize this intersection. Any two distinct tensors T_1 and T_2 define a line, which can be represented by

a_1 T_1+(1-a_1)T_2 \,.

Similarly, any three tensors T_1, T_2, and T_2 that do not lie on a line define a plane

a_1 T_1+a_2T_2+(1-a_1-a_2)T_3 \,.

By intersecting S with one of these simpler objects we obtain something that we can visualize.

Including Nearby Points

Unfortunately, in practice the straightforward intersection described above yields too few points to plot. If we proceed by generating points in S, the probability that any of them lie in the given line/plane is virtually zero. If we proceed by generating points in the line/plane then we need to test their membership in S. For the sets we are interested in this membership test can fail, and at best can produce a tensor in S within some small distance of the tensor we are checking.

A star-shaped set is a set S such that T\in S
\Rightarrow cT \in S for any scalar c. Such a S is connected and simply connected. Instead of asking if T\in S is on our plotting line/plane, we can ask if cT is on our plotting line/plane for some c, and if so, plot that cT. Thus we can increase the number of plotted points and remove the uninteresting scalar structure. Formally, we consider tensors that are positive scalar multiples of one another to be equivalent. Locally, this is similar to restricting to tensors of norm one. (Identifying negative scalar multiples as well does not seem to be worthwhile.)

We will add further information to the visualization by including points that are in S but not on the plotting line/plane and coloring them to indicate how far from the plotting plane they are. Since we identify T and cT we measure “how far” using an angle instead of a distance.

We will describe the the case for a plane, since the case for a line is a simple modification of it. From the three points T_1, T_2, and T_3 that define our plane, we define the matrix

G=
\left[\begin{array}{ccc}
\langle T_1,T_1\rangle & \langle T_2,T_1\rangle& \langle T_3,T_1\rangle\\
\langle T_1,T_2\rangle & \langle T_2,T_2\rangle& \langle T_3,T_2\rangle\\
\langle T_1,T_3\rangle & \langle T_2,T_3\rangle& \langle T_3,T_3\rangle
\end{array}\right]
\,.

Given T\in S, its projection onto the span of T_1, T_2, and T_3 is given by

\hat{T}=c_1 T_1 + c_2 T_2+ c_3 T_3

with

\left[\begin{array}{c}
c_1\\
c_2\\
c_3
\end{array}\right]
=G^{-1}
\left[\begin{array}{c}
\langle T,T_1\rangle\\
\langle T,T_2\rangle\\
\langle T,T_3\rangle
\end{array}\right]
\,.

Dividing by the scalar c_1+c_2+c_3, we obtain the tensor

U=b_1 T_1 + b_2 T_2+ b_3 T_3
=\frac{c_1}{c_1+c_2+c_3} T_1 + \frac{c_2}{c_1+c_2+c_3} T_2
+ \frac{c_3}{c_1+c_2+c_3} T_3\,,

which lies on the plotting plane. To measure how much T failed to be on the plane, we compute the angle

\angle(T,U)=\angle(T,\hat{T})
=\arccos\left(\frac{\langle T, \hat{T}\rangle}{\|T\|\,\|\hat{T}\|}\right)
=\arccos\left(\frac{\langle \hat{T},\hat{T}\rangle}{\|T\|\,\|\hat{T}\|}\right)
=\arccos\left(\frac{\|\hat{T}\|}{\|T\|}\right)\,.

An angle of zero indicates that some scalar multiple of T is on the plane and an angle of \pi/2 indicates that T is orthogonal to the span of T_1, T_2, and T_3. This projection process (neglecting T_3) is illustrated below:

\setlength{\unitlength}{2mm}
\begin{picture}(50,50)(-5,-5)
\put(0,0){\circle*{1}}
\put(-5,-3){\mbox{Origin}}
\put(28,0){\circle*{1}}
\put(25.8,-4){\mbox{$T_1$}}
\put(5,15){\circle*{1}}
\put(5,18){\mbox{$T_2$}}
\put(-1,19){\line(3,-2){35}}
\put(-5,25){\mbox{Plotting Plane}}
\put(0,24){\vector(0,-1){4}}
\put(0,0){\vector(2,1){30}}
\put(0,0){\vector(1,1){30}}
\put(24,24){\circle*{1}}
\put(20,24){\mbox{$T$}}
\put(15.8,8){\circle*{1}}
\put(14,3){\mbox{$U$}}
\put(24,24){\line(0,-1){12}}
\put(22,13){\line(0,-1){2}}
\put(22,13){\line(2,1){2}}
\put(24,12){\circle*{1}}
\put(23,8){\mbox{$\hat{T}$}}
\put(-5,6){\mbox{$\angle(T,U)$}}
\put(1,5){\vector(1,-1){2.5}}
\end{picture}

To indicate this angle in the visualization, we color the plotted point with black indicting angle nearly zero, then through a spectrum of ever-lightening colors until light yellow indicates angles near \pi/2. We use the colors

_images/spectrum.jpg

and bin the angles logarithmicly by

[0,\pi/2048],(\pi/2048,\pi/1024],\ldots,(\pi/8,\pi/4],(\pi/4,\pi/2].

In cases where more than one tensor projects to the same plotting point, we plot the darker of the two. Since we plot points as small discs, in cases where the discs overlap we plot the darker disc on top.

If T is orthogonal to the span of T_1, T_2, and T_3 we will have c_1=c_2=c_3=0 and cannot consruct U. Thus T should not appear in the visualization. (Numerically one produces a white point at a location determined by roundoff errors.)

Real Tensors, plotted on a real plane

For tensors over the real numbers, our tool uses three tensors yielding a plane, so we now consider that case in more detail. We arbitrarily choose to orient the visualization so that T_3 is at (0,0) and T_1 is in the direction (1,0). The angle from T_1 to T_3 to T_2 is given by

\theta=\arccos\left(\frac{\langle T_1-T_3, T_2-T_3\rangle}
{\|T_1-T_3\|\,\|T_2-T_3\|} \right)\,

so T_2 is in the direction (\cos(\theta),\sin(\theta)). A tensor T=a_1 T_1+a_2T_2+(1-a_1-a_2)T_3 is at the point

(a_1 \|T_1-T_3\| + \cos(\theta) a_2 \|T_2-T_3\|,
\sin(\theta) a_2 \|T_2-T_3\|).

This plotting geometry is illustrated below:

\setlength{\unitlength}{1.7mm}
\begin{picture}(50,50)(-5,-5)
\put(-5,0){\vector(1,0){50}}
\put(0,-5){\vector(0,1){50}}
\put(40,-5){\mbox{x}}
\put(-5,40){\mbox{y}}
\put(0,0){\circle*{1}}
\put(30,0){\circle*{1}}
\put(30,40){\circle*{1}}
\put(-4,-4){\mbox{$T_3$}}
\put(28,-4){\mbox{$T_1$}}
\put(25,40){\mbox{$T_2$}}
\put(0,0){\line(3,4){30}}
\put(5,2){\mbox{$\theta$}}
\put(25,2){\mbox{$\theta$}}
\put(0,0){\vector(1,0){20}}
\put(4,-3){\mbox{$a_1\|T_1-T_3\|$}}
\put(20,0){\vector(3,4){15}}
\put(30,10){\mbox{$a_2\|T_2-T_3\|$}}
\put(35,20){\circle*{1}}
\put(30,20){\mbox{$T$}}
\end{picture}

Complex Tensors, plotted on a complex line

For tensors over the complex numbers, our tool uses two tensors yielding a complex line. We arbitrarily choose to orient the visualization so that T_2 is at 0+0i and T_1 is in the direction 1+0i. A tensor T=a_1 T_1+(1-a_1)T_2 is at the complex point a_1. This plotting geometry is illustrated below:

\setlength{\unitlength}{1.7mm}
\begin{picture}(50,40)(-5,-5)
\put(-5,0){\vector(1,0){50}}
\put(0,-5){\vector(0,1){40}}
\put(40,-5){\mbox{real}}
\put(2,30){\mbox{imaginary}}
\put(0,0){\circle*{1}}
\put(30,0){\circle*{1}}
\put(-4,-4){\mbox{$T_2$}}
\put(28,-4){\mbox{$T_1$}}
\put(0,0){\vector(3,4){15}}
\put(9,8){\mbox{$a_1$}}
\put(15,20){\circle*{1}}
\put(13,21){\mbox{$T$}}
\end{picture}