This page is part of the Dynamical Systems on Tensor Approximations project.
These visualizations were started by S. Elaine Hale as a project for MATH 6940 Project in Computational Mathematics during Spring 2015.
We are studying approximations for tensors \(T\) of the form \[ T(j_1,j_2,\dots,j_d) \approx G(j_1,\dots,j_d) =\sum_{l=1}^{r} G^l(j_1,\dots,j_d) = \sum_{l=1}^r \prod_{i=1}^{d} G_i^l(j_i) \quad\text{for}\quad j_i=1,2,\dots,M_i \,, \] or equivalently \[ T \approx G =\sum_{l=1}^{r} G^l = \sum_{l=1}^r \bigotimes_{i=1}^{d} G_i^l \,, \] where \(d\) is the "dimension" (number of variables). As a particular test case, we consider the rank-2 tensors \[ T = \left(1+2 z\cos^d(\phi)+z^2\right)^{-1/2} \left(\bigotimes_{i=1}^{d} \mathbf{u}(0) +z\bigotimes_{i=1}^{d} \mathbf{u}(\phi)\right) \,, \] where
If we choose \(r=1\), then our approximation can be written as \[ G = a \bigotimes_{i=1}^{d} \mathbf{u}(\alpha_i) \,. \] Given the angles \(\{\alpha_i\}_{i=1}^d\), the optimal value for the scalar \(a\) can be easily determined, so we assume \(a\) is set optimally. We will consider the symmetric case, when \(\alpha_i=\alpha\) for all \(i\). We will visualize two aspects of this approximation. In both cases we choose values for \(d\), \(z\), and \(\lambda\) and plot with \(0\le\phi\le\pi/2\) on the horizontal axis and \(-\pi/2 \le \alpha-\phi/2 \le \pi/2\) on the vertical axis. The downward and upward slanted lines give the positions of the first and second summands in the target.
\(d=\) | \(\lambda=\) | \(z=\) |
If we choose \(r=2\), then our approximation can be written as \[ G = a \bigotimes_{i=1}^{d} \mathbf{u}(\alpha_i) + b \bigotimes_{i=1}^{d} \mathbf{u}(\beta_i) \,. \] Given the angles \(\{\alpha_i\}_{i=1}^d\) and \(\{\beta_i\}_{i=1}^d\), the optimal values for the scalars \(a\) and \(b\) can be easily determined, so we assume \(a\) and \(b\) are set optimally. We will consider the symmetric case, when \(\alpha_i=\alpha\) and \(\beta_i=\beta\) for all \(i\). We will visualize two aspects of this approximation. In both cases we choose values for \(d\), \(\phi\), \(z\), and \(\lambda\) and plot with \(-\pi/2 \le \alpha-\phi/2 \le \pi/2\) on the horizontal axis and \(-\pi/2 \le \beta-\phi/2 \le \pi/2\) on the vertical axis. The horizontal and vertical lines indicate the positions of the first and second summands in the target, at angles \(0\) and \(\phi\).
\(d=\) | \(\lambda=\) | \(z=\) | \(\phi=\) |
If we choose \(r=3\), then our approximation can be written as \[ G = a \bigotimes_{i=1}^{d} \mathbf{u}(\alpha_i) + b \bigotimes_{i=1}^{d} \mathbf{u}(\beta_i) + c \bigotimes_{i=1}^{d} \mathbf{u}(\gamma_i) \,. \] Given the angles \(\{\alpha_i\}_{i=1}^d\), \(\{\beta_i\}_{i=1}^d\), and \(\{\gamma_i\}_{i=1}^d\), the optimal values for the scalars \(a\), \(b\), and \(c\) can be easily determined, so we assume they are set optimally. We will consider the symmetric case, when \(\alpha_i=\alpha\), \(\beta_i=\beta\), and \(\gamma_i=\gamma\) for all \(i\). We plot with \(-\pi/2 \le \alpha-\phi/2 \le \pi/2\) on one axis, \(-\pi/2 \le \beta-\phi/2 \le \pi/2\) on the second axis, and \(-\pi/2 \le \gamma-\phi/2 \le \pi/2\) on the third axis; by symmetry it does not matter which axis is which. The thin white tubes mark \((\alpha,\beta,\gamma)=(0,\phi,\text{any})\) and its permutations, where two terms in the approximation match the two terms in the target.
Each video shows the contours of \(E_\lambda(G)\) as the value moves from 1 to 0. The video then rotates to show the same moving contours from a different angle. Note that when the contour moves slowly it indicates large gradient so the gradient flow would move quickly. When the contour moves quickly, the gradient flow would move slowly.
\(\phi=\pi/2\) | \(\phi\approx 0.27\pi\) | \(\phi=\pi/8\) |
---|---|---|
\(\phi=\pi/2\) | \(\phi\approx 0.27\pi\) | \(\phi=\pi/8\) |
---|---|---|
\(\phi=\pi/2\) | \(\phi\approx 0.39\pi\) | \(\phi=\pi/8\) |
---|---|---|
\(\phi=\pi/2\) | \(\phi\approx 0.39\pi\) | \(\phi=0\) |
---|---|---|
When \(z=-1\), plugging in \(\phi=0\) yields an indeterminate form \(0/0\), but we can compute the limit \[ \lim_{\phi\rightarrow 0^+} \left(2-2\cos^d(\phi)\right)^{-1/2} \left( \bigotimes_{i=1}^{d} \mathbf{u}(0) -\bigotimes_{i=1}^{d} \mathbf{u}(\phi) \right) =\lim_{\phi\rightarrow 0^+} \left(2-2\cos^d(\phi)\right)^{-1/2} \left(v(0)-v(\phi)\right) \,, \] and find (after some work) that it equals \[ L=\frac{-1}{\sqrt{d}}\sum_{j=1}^{d} \left(\bigotimes_{i=1}^{j-1}\mathbf{e}_1\right) \otimes \mathbf{e}_2 \otimes \left(\bigotimes_{i=j+1}^{d}\mathbf{e}_1\right) \,. \] Although \(L\) is rank \(d\), it is the limit of rank-2 tensors, which makes it a strange and interesting example. It has the same structure as the \(d\)-dimensional Laplacian, which also makes it important.
Below is an attempt to illustrate how such a strange thing can happen. The blue vector shows \(L\) and the green vector labeled v(0) shows the location of the fixed vector \(v(0)\), which is orthogonal to \(L\). The green curve gives the possible positions of \(-v(t)\) as \(t\) varies and the green vector labeled -v(t) shows the location of the vector \(-v(t)\) for the current \(t\). The red vector shows \(T\) for the current \(t\). The purple plane shows the span of \(\{v(0),v(t)\}\) and the red segment shows the (error of the) orthogonal projection of \(L\) onto this span; these were not described above but give another way to think about this example. As \(t\rightarrow 0^+\) the error goes to zero but if \(t=0\) then the span collapses to a line orthogonal to \(L\) and so the best approximation of \(L\) is 0.