From 355bc711c2118918e53bda224e363d6bdf2ebc13 Mon Sep 17 00:00:00 2001 From: Alper Ahmetoglu Date: Mon, 27 Feb 2017 11:24:04 +0300 Subject: [PATCH 1/2] 3.6.a partially --- hw3.ipynb | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 hw3.ipynb diff --git a/hw3.ipynb b/hw3.ipynb new file mode 100644 index 0000000..0e7b16a --- /dev/null +++ b/hw3.ipynb @@ -0,0 +1,94 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Homework 3" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 3.6.a" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In order to prove $\\|\\cdot\\|'$ is a norm, we have to prove the following:\n", + "1. $\\| \\mathbf{x} \\|'=0$ if and only if $\\mathbf{x}=0$, \n", + "2. $\\| \\mathbf{x} + \\mathbf{y}\\|' \\leq \\| \\mathbf{x} \\|' + \\| \\mathbf{y} \\|'$\n", + "3. $\\| \\alpha \\mathbf{x} \\| = \\left| \\alpha \\right| \\| \\mathbf{x} \\|$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\\|\\mathbf{x}\\|' = \\sup_{\\|y\\|=1} \\left| \\mathbf{y}^* \\mathbf{x} \\right| $$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "1. If $\\mathbf{x}=0$, $\\left| \\mathbf{y}^* \\mathbf{x} \\right| = 0$ for every $\\|\\mathbf{y}\\|=1$. \n", + "If $\\left| \\mathbf{y}^* \\mathbf{x} \\right|=0$, $\\mathbf{x}=0$ should hold. Take a non-zero vector $\\mathbf{x}$. We can always choose the $\\mathbf{y}$ such that $\\mathbf{y}=\\frac{x}{\\|x\\|}$. In this case $\\| \\mathbf{x} \\|' >0$. Therefore $\\|x\\|'=0, x\\neq0$ is not true. \n", + "\n", + "2. \n", + "\n", + "3. $$\\sup_{\\|y\\|=1}\\left|\\mathbf{y}^*\\alpha\\mathbf{x}\\right|$$ \n", + "$$\\sup_{\\|y\\|=1}\\left|y^*_1 \\alpha x_1 + y^*_2 \\alpha x_2 + \\dots + y^*_m \\alpha x_m \\right|$$ \n", + "$$\\sup_{\\|y\\|=1} \\left| \\left( \\alpha \\right) \\left( y^*_1 x_1 + y^*_2 x_2 + \\dots + y^*_m x_m \\right) \\right|$$ \n", + "$$\\sup_{\\|y\\|=1}\\left|\\alpha\\right| \\left| y^*_1 x_1 + y^*_2 x_2 + \\dots + y^*_m x_m \\right|$$ \n", + "$$\\left|\\alpha\\right| \\sup_{\\|y\\|=1} \\left| \\mathbf{y}^* \\mathbf{x} \\right|$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Octave", + "language": "octave", + "name": "octave" + }, + "language_info": { + "file_extension": ".m", + "help_links": [ + { + "text": "GNU Octave", + "url": "https://www.gnu.org/software/octave/support.html" + }, + { + "text": "Octave Kernel", + "url": "https://github.com/Calysto/octave_kernel" + }, + { + "text": "MetaKernel Magics", + "url": "https://github.com/calysto/metakernel/blob/master/metakernel/magics/README.md" + } + ], + "mimetype": "text/x-octave", + "name": "octave", + "version": "4.2.0" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 8e7605a2cefb71c42e51e671eb1a118801aaf50e Mon Sep 17 00:00:00 2001 From: Alper Ahmetoglu Date: Fri, 3 Mar 2017 21:12:08 +0300 Subject: [PATCH 2/2] partially finished --- hw3.ipynb | 636 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 624 insertions(+), 12 deletions(-) diff --git a/hw3.ipynb b/hw3.ipynb index 0e7b16a..ee98a18 100644 --- a/hw3.ipynb +++ b/hw3.ipynb @@ -2,21 +2,30 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "deletable": true, + "editable": true + }, "source": [ "## Homework 3" ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "deletable": true, + "editable": true + }, "source": [ "### 3.6.a" ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "deletable": true, + "editable": true + }, "source": [ "In order to prove $\\|\\cdot\\|'$ is a norm, we have to prove the following:\n", "1. $\\| \\mathbf{x} \\|'=0$ if and only if $\\mathbf{x}=0$, \n", @@ -26,19 +35,27 @@ }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "deletable": true, + "editable": true + }, "source": [ "$$\\|\\mathbf{x}\\|' = \\sup_{\\|y\\|=1} \\left| \\mathbf{y}^* \\mathbf{x} \\right| $$" ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "deletable": true, + "editable": true + }, "source": [ "1. If $\\mathbf{x}=0$, $\\left| \\mathbf{y}^* \\mathbf{x} \\right| = 0$ for every $\\|\\mathbf{y}\\|=1$. \n", - "If $\\left| \\mathbf{y}^* \\mathbf{x} \\right|=0$, $\\mathbf{x}=0$ should hold. Take a non-zero vector $\\mathbf{x}$. We can always choose the $\\mathbf{y}$ such that $\\mathbf{y}=\\frac{x}{\\|x\\|}$. In this case $\\| \\mathbf{x} \\|' >0$. Therefore $\\|x\\|'=0, x\\neq0$ is not true. \n", + "If $\\left| \\mathbf{y}^* \\mathbf{x} \\right|=0$, $\\mathbf{x}=0$ should hold. Take a non-zero vector $\\mathbf{x}$. We can always choose the $\\mathbf{y}$ such that $\\mathbf{y}=\\frac{\\mathbf{x}}{\\|\\mathbf{x}\\|}$. In this case $\\| \\mathbf{x} \\|' >0$. Therefore $\\|\\mathbf{x}\\|'=0, \\mathbf{x}\\neq0$ is not true. \n", "\n", - "2. \n", + "2. $$\\sup_{\\|y\\|=1}\\left|\\mathbf{y}^*\\left(\\mathbf{x}+\\mathbf{z}\\right)\\right|$$ \n", + "$$\\sup_{\\|y\\|=1}\\left|\\mathbf{y}^*\\mathbf{x} + \\mathbf{y}^*\\mathbf{z}\\right|$$ \n", + "Let us say $\\mathbf{y}^M$ is the vector which maximises $\\|\\mathbf{x}+\\mathbf{z}\\|$. Since $\\left|\\mathbf{x}+\\mathbf{z}\\right| \\leq \\left|\\mathbf{x}\\right| + \\left|\\mathbf{z}\\right|$, $\\left|\\mathbf{y}^{M*} \\mathbf{x} + \\mathbf{y}^{M*}\\mathbf{z}\\right| \\leq \\left| \\mathbf{y}^{M*}\\mathbf{x}\\right| + \\left|\\mathbf{y}^{M*}\\mathbf{z}\\right|$ is true. \n", "\n", "3. $$\\sup_{\\|y\\|=1}\\left|\\mathbf{y}^*\\alpha\\mathbf{x}\\right|$$ \n", "$$\\sup_{\\|y\\|=1}\\left|y^*_1 \\alpha x_1 + y^*_2 \\alpha x_2 + \\dots + y^*_m \\alpha x_m \\right|$$ \n", @@ -47,19 +64,614 @@ "$$\\left|\\alpha\\right| \\sup_{\\|y\\|=1} \\left| \\mathbf{y}^* \\mathbf{x} \\right|$$" ] }, + { + "cell_type": "markdown", + "metadata": { + "deletable": true, + "editable": true + }, + "source": [ + "### 5.3" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### a." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$ A = U\\Sigma V^* $$ \n", + "Since $U$ and $V^*$ are unitary, $V^*=V^-$\n", + "$$ A^*A = V\\Sigma^* U^*U\\Sigma V^* $$ \n", + "$$ A^*A = V\\Sigma^2 V^* $$ \n", + "$$ AA^* = U\\Sigma V^*V\\Sigma^* U^* $$ \n", + "$$ AA^* = U\\Sigma^2 U^* $$ \n", + "We find eigenvalues and eigenvectors of $A^*A$ and $AA^*$. Eigenvalues give $\\Sigma^2$, left eigenvectors of $A^*A$ give $V$, left eigenvectors of $AA^*$ give $U$." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "eigenvalues =\n", + "\n", + " 200\n", + " 50\n", + "\n", + "eigenvector_1 =\n", + "\n", + " 0.60000\n", + " -0.80000\n", + "\n", + "eigenvector_2 =\n", + "\n", + " 0.80000\n", + " 0.60000\n", + "\n", + "V =\n", + "\n", + " 0.60000 -0.80000\n", + " 0.80000 0.60000\n", + "\n", + "eigenvector_1 =\n", + "\n", + " -0.70711\n", + " -0.70711\n", + "\n", + "eigenvector_2 =\n", + "\n", + " -0.70711\n", + " 0.70711\n", + "\n", + "U =\n", + "\n", + " -0.70711 -0.70711\n", + " -0.70711 0.70711\n", + "\n", + "S =\n", + "\n", + " 14.14214 0.00000\n", + " 0.00000 7.07107\n", + "\n", + "B =\n", + "\n", + " -2.0000 -11.0000\n", + " -10.0000 -5.0000\n", + "\n" + ] + } + ], + "source": [ + "A = [-2,11;-10,5];\n", + "A_ta = A'*A;\n", + "eigenvalues = roots([1, -A_ta(1,1)-A_ta(2,2) , A_ta(1,1)*A_ta(2,2)-A_ta(1,2)*A_ta(2,1)])\n", + "eigenvector_1 = null(A_ta-eye(2)*eigenvalues(1))\n", + "eigenvector_2 = null(A_ta-eye(2)*eigenvalues(2))\n", + "V = [eigenvector_1'; eigenvector_2']\n", + "\n", + "Aa_t = A*A';\n", + "eigenvector_1 = null(Aa_t-eye(2)*eigenvalues(1))\n", + "eigenvector_2 = null(Aa_t-eye(2)*eigenvalues(2))\n", + "\n", + "U = [eigenvector_1, eigenvector_2]\n", + "\n", + "S = [sqrt(eigenvalues(1)), 0; 0, sqrt(eigenvalues(2))]\n", + "\n", + "B = U * S * V'" + ] + }, { "cell_type": "markdown", "metadata": {}, - "source": [] + "source": [ + "#### b." + ] + }, + { + "cell_type": "code", + "execution_count": 147, + "metadata": { + "collapsed": false, + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "r = 1\n" + ] + }, + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 5 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tu vectors\n", + "\t\n", + "\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\tgnuplot_plot_3a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "x = 0:0.01:2*pi;\n", + "r = 1\n", + "xx = r .* cos(x);\n", + "yy = r .* sin(x);\n", + "plot(xx, yy, \"linewidth\", 3, \"b\")\n", + "\n", + "hold on\n", + "compass(V(1,1), V(1,2))\n", + "compass(V(2,1), V(2,2))\n", + "title(\"u vectors\")" + ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 148, "metadata": { - "collapsed": true + "collapsed": false }, - "outputs": [], - "source": [] + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 5 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tv * sigma vectors\n", + "\t\n", + "\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\tgnuplot_plot_3a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "K = A * [xx ; yy];\n", + "\n", + "plot(K(1,:), K(2,:), \"linewidth\", 3)\n", + "hold on\n", + "\n", + "compass(U(1,1) * S(1,1), U(2,1) * S(1,1))\n", + "compass(U(1,2) * S(2,2), U(2,2) * S(2,2))\n", + "title(\"v * sigma vectors\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### c." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "By theorem 5.3, $\\|A\\|_2 =\\sigma_1$ and $\\|A\\|_F = \\sqrt{\\sigma_1^{2} + \\sigma_2^{2} + \\dots + \\sigma_2^{r}}$." + ] + }, + { + "cell_type": "code", + "execution_count": 164, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans = 14.142\n", + "ans = 14.142\n" + ] + } + ], + "source": [ + "# 2-norm\n", + "max(max(S))\n", + "\n", + "norm(A,2)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans = 15.811\n", + "ans = 15.811\n" + ] + } + ], + "source": [ + "# Frobenius norm\n", + "sqrt(sum(eigenvalues))\n", + "\n", + "norm(A, p=\"fro\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$\\|A\\|_1 = \\max{c_i}$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### d." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Since $U$ and $V$ are unitary matrices, $A^-=V\\Sigma^-U^*$" + ] + }, + { + "cell_type": "code", + "execution_count": 149, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "C =\n", + "\n", + " 0.050000 -0.110000\n", + " -0.100000 0.020000\n", + "\n", + "ans =\n", + "\n", + " 0.050000 -0.110000\n", + " 0.100000 -0.020000\n", + "\n" + ] + } + ], + "source": [ + "C = V * inv(S) * U'\n", + "\n", + "inv(A)" + ] + }, + { + "cell_type": "code", + "execution_count": 156, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans = 100.00\n" + ] + } + ], + "source": [ + "S(1,1)*S(2,2)" + ] } ], "metadata": {