Tutorial_4 - The usage of  name_variable_scope.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# name/variable_scope 的作用\n",
    "@author: huangyongye<br/>\n",
    "@creat_date: 2017-03-08<br/><br/>\n",
    "refer to: https://www.tensorflow.org/programmers_guide/variable_scope <br/>\n",
    "** 起因：在运行 RNN LSTM 实例代码的时候出现 ValueError。 ** <br/>\n",
    "在 TensorFlow 中，经常会看到这两个东东出现，这到底是什么鬼，是用来干嘛的。在做 LSTM 的时候遇到了下面的错误：<br/>\n",
    " <font color='red'>ValueError: Variable rnn/basic_lstm_cell/weights already exists, disallowed.</font><br/>\n",
    "然后谷歌百度都查了一遍，结果也不知是咋回事。我是在 jupyter notebook 运行的示例程序，第一次运行的时候没错，然后就总是出现上面的错误。后来才知道是 get_variable() 和 variable_scope() 搞的鬼。 <br/>\n",
    "下面就来分析一下 TensorFlow 中到底用这来干啥。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "# 设置GPU按需增长\n",
    "config = tf.ConfigProto()\n",
    "config.gpu_options.allow_growth = True\n",
    "sess = tf.Session(config=config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vs.name=vs1\n",
      "a: <tf.Variable 'vs1/var1:0' shape=(2, 4) dtype=float32_ref>\n",
      "b: <tf.Variable 'vs1/var2:0' shape=(3, 4) dtype=float32_ref>\n",
      "c: <tf.Variable 'vs1/var3:0' shape=(1, 4) dtype=int32_ref>\n"
     ]
    }
   ],
   "source": [
    "with tf.variable_scope('vs1') as vs:\n",
    "    a = tf.get_variable('var1', shape=[2,4], dtype=tf.float32)\n",
    "    b = tf.get_variable('var2', shape=[3,4], dtype=tf.float32)\n",
    "#     c = tf.get_variable('var2', shape=[3,4], dtype=tf.float32) # 报错，tf.get_variable() 是根据 name 来管理变量名的，所以 name 一定不能重复。\n",
    "    c = tf.Variable([[0,0,0,0]], name='var3')\n",
    "    print('vs.name=%s' % vs.name)\n",
    "    \n",
    "print('a: %s' % a)\n",
    "print('b: %s' % b)\n",
    "print('c: %s' % c)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vs.name=vs1\n",
      "d: <tf.Variable 'vs1/var1:0' shape=(2, 4) dtype=float32_ref>\n",
      "e: <tf.Variable 'vs1_1/var4:0' shape=(1, 5) dtype=float32_ref>\n"
     ]
    }
   ],
   "source": [
    "with tf.variable_scope('vs1', reuse=True) as vs:\n",
    "#     d = tf.get_variable('var1', shape=[3,4], dtype=tf.float32)    # 报错， d 和 a 共用变量，所以shape一定是相同的\n",
    "    d = tf.get_variable('var1', shape=[2,4], dtype=tf.float32)      # 正确, d 和 a 共用一个变量\n",
    "#     e = tf.get_variable('var3', shape=[1,5], dtype=tf.float32)    # 报错，在 vs1/var3 还没有存在\n",
    "    e = tf.Variable([[1,2,3,4,5]], dtype=tf.float32, name='var4')   # 正确，tf.variable_scope 不影响使用 Variable 方式定义变量     \n",
    "    print('vs.name=%s' % vs.name)\n",
    "    \n",
    "print('d: %s' % d)\n",
    "print('e: %s' % e)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面可以看出，无论是 tf.get_variable 还是 tf.Variable, **创建的都是 tf.Variable 变量。** 把 a,b,c,d 成为变量名，把 'vs1/var1:0','vs1/var2:0' 称为内存变量名称（注意：这样的称呼只是我自己便于理解，不是专业的叫法）。不同变量名可以指向同一个内存变量，这就是所谓的变量共享。\n",
    "\n",
    "可以看出 tf.variable_scope() 功能是要比 name_scope 更加强大的，下面看看 tf.name_scope 就没这么强大了。它主要就是提供命名管理罢了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(<type 'str'>, 'ns1/')\n"
     ]
    },
    {
     "ename": "AttributeError",
     "evalue": "'str' object has no attribute 'name'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-4-b8801d905540>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname_scope\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'ns1'\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mns\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m     \u001b[0;32mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m     \u001b[0;32mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'ns.name=%s'\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0mns\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname_scope\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'ns1'\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mns\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAttributeError\u001b[0m: 'str' object has no attribute 'name'"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('ns1') as ns:\n",
    "    print(type(ns), ns)\n",
    "    print('ns.name=%s' % ns.name)\n",
    "\n",
    "with tf.name_scope('ns1') as ns:\n",
    "    print('ns.name=%s' % ns.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 1. 首先看看比较简单的 tf.name_scope('scope_name'). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "tf.name_scope 主要结合 tf.Variable() 来使用，方便参数命名管理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "conv1/weights:0\n",
      "conv2/weights:0\n"
     ]
    }
   ],
   "source": [
    "'''\n",
    "Signature: tf.name_scope(*args, **kwds)\n",
    "Docstring:\n",
    "Returns a context manager for use when defining a Python op.\n",
    "'''\n",
    "# 也就是说，它的主要目的是为了更加方便地管理参数命名。\n",
    "# 与 tf.Variable() 结合使用。简化了命名\n",
    "with tf.name_scope('conv1') as scope:\n",
    "    weights1 = tf.Variable([1.0, 2.0], name='weights')\n",
    "    bias1 = tf.Variable([0.3], name='bias')\n",
    "\n",
    "# 下面是在另外一个命名空间来定义变量的\n",
    "with tf.name_scope('conv2') as scope:\n",
    "    weights2 = tf.Variable([4.0, 2.0], name='weights')\n",
    "    bias2 = tf.Variable([0.33], name='bias')\n",
    "\n",
    "# 所以，实际上weights1 和 weights2 这两个引用名指向了不同的空间，不会冲突\n",
    "print weights1.name\n",
    "print weights2.name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "conv1_1/weights:0\n",
      "conv2_1/weights:0\n"
     ]
    }
   ],
   "source": [
    "# 注意，这里的 with 和 python 中其他的 with 是不一样的\n",
    "# 执行完 with 里边的语句之后，这个 conv1/ 和 conv2/ 空间还是在内存中的。这时候如果再次执行上面的代码\n",
    "# 就会再生成其他命名空间\n",
    "with tf.name_scope('conv1') as scope:\n",
    "    weights1 = tf.Variable([1.0, 2.0], name='weights')\n",
    "    bias1 = tf.Variable([0.3], name='bias')\n",
    "\n",
    "with tf.name_scope('conv2') as scope:\n",
    "    weights2 = tf.Variable([4.0, 2.0], name='weights')\n",
    "    bias2 = tf.Variable([0.33], name='bias')\n",
    "\n",
    "print weights1.name\n",
    "print weights2.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.下面来看看 tf.variable_scope('scope_name') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "tf.variable_scope() 主要结合 tf.get_variable() 来使用，实现 变量共享。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vs1/Weights:0\n",
      "vs1/Weights:0\n"
     ]
    }
   ],
   "source": [
    "# 这里是正确的打开方式~~~可以看出，name 参数才是对象的唯一标识\n",
    "with tf.variable_scope('vs1') as scope:\n",
    "    Weights1 = tf.get_variable('Weights', shape=[2,3])\n",
    "    bias1 = tf.get_variable('bias', shape=[3])\n",
    "    \n",
    "# 下面来共享上面已经定义好的变量\n",
    "# note: 在下面的 scope 中的变量必须已经定义过了，才能设置 reuse=True，否则会报错\n",
    "with tf.variable_scope('vs1', reuse=True) as scope:\n",
    "    Weights2 = tf.get_variable('Weights')\n",
    "\n",
    "print Weights1.name\n",
    "print Weights2.name\n",
    "# 可以看到这两个引用名称指向的是同一个内存对象"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "也可以结合 tf.Variable() 一块使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vs2/Weights:0\n",
      "vs2/bias:0\n",
      "vs2/Weights:0\n",
      "vs2_1/bias:0\n"
     ]
    }
   ],
   "source": [
    "# 注意， bias1 的定义方式\n",
    "with tf.variable_scope('vs2') as scope:\n",
    "    Weights1 = tf.get_variable('Weights', shape=[2,3])\n",
    "    bias1 = tf.Variable([0.52], name='bias')\n",
    "    \n",
    "# 下面来共享上面已经定义好的变量\n",
    "# note: 在下面的 scope 中的get_variable()变量必须已经定义过了，才能设置 reuse=True，否则会报错\n",
    "with tf.variable_scope('vs2', reuse=True) as scope:\n",
    "    Weights2 = tf.get_variable('Weights')\n",
    "    bias2 = tf.Variable([0.52], name='bias')\n",
    "\n",
    "print Weights1.name\n",
    "print bias1.name\n",
    "print Weights2.name\n",
    "print bias2.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果 reuse=True 的scope中的变量没有已经定义，会报错！！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vs3/Weights:0\n",
      "vs3/bias:0\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "Variable vs3/bias does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-15957a828b48>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvariable_scope\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'vs3'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreuse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mTrue\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mscope\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m     \u001b[0mWeights2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_variable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Weights'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m     \u001b[0mbias2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_variable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'bias'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# bias\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0;32mprint\u001b[0m \u001b[0mWeights2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/common/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)\u001b[0m\n\u001b[1;32m   1063\u001b[0m       \u001b[0mcollections\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcollections\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1064\u001b[0m       \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1065\u001b[0;31m       use_resource=use_resource, custom_getter=custom_getter)\n\u001b[0m\u001b[1;32m   1066\u001b[0m get_variable_or_local_docstring = (\n\u001b[1;32m   1067\u001b[0m     \"\"\"%s\n",
      "\u001b[0;32m/home/common/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)\u001b[0m\n\u001b[1;32m    960\u001b[0m           \u001b[0mcollections\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcollections\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    961\u001b[0m           \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 962\u001b[0;31m           use_resource=use_resource, custom_getter=custom_getter)\n\u001b[0m\u001b[1;32m    963\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    964\u001b[0m   def _get_partitioned_variable(self,\n",
      "\u001b[0;32m/home/common/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc\u001b[0m in \u001b[0;36mget_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)\u001b[0m\n\u001b[1;32m    365\u001b[0m           \u001b[0mreuse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mreuse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrainable\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrainable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcollections\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcollections\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    366\u001b[0m           \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpartitioner\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpartitioner\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 367\u001b[0;31m           validate_shape=validate_shape, use_resource=use_resource)\n\u001b[0m\u001b[1;32m    368\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    369\u001b[0m   def _get_partitioned_variable(\n",
      "\u001b[0;32m/home/common/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc\u001b[0m in \u001b[0;36m_true_getter\u001b[0;34m(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource)\u001b[0m\n\u001b[1;32m    350\u001b[0m           \u001b[0mtrainable\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrainable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcollections\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcollections\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    351\u001b[0m           \u001b[0mcaching_device\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcaching_device\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidate_shape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvalidate_shape\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 352\u001b[0;31m           use_resource=use_resource)\n\u001b[0m\u001b[1;32m    353\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    354\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mcustom_getter\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/common/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc\u001b[0m in \u001b[0;36m_get_single_variable\u001b[0;34m(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource)\u001b[0m\n\u001b[1;32m    680\u001b[0m       raise ValueError(\"Variable %s does not exist, or was not created with \"\n\u001b[1;32m    681\u001b[0m                        \u001b[0;34m\"tf.get_variable(). Did you mean to set reuse=None in \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 682\u001b[0;31m                        \"VarScope?\" % name)\n\u001b[0m\u001b[1;32m    683\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_fully_defined\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0minitializing_from_value\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    684\u001b[0m       raise ValueError(\"Shape of a new variable (%s) must be fully defined, \"\n",
      "\u001b[0;31mValueError\u001b[0m: Variable vs3/bias does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?"
     ]
    }
   ],
   "source": [
    "# 注意， bias1 的定义方式\n",
    "with tf.variable_scope('vs3') as scope:\n",
    "    Weights1 = tf.get_variable('Weights', shape=[2,3])\n",
    "    bias1 = tf.Variable([0.52], name='bias')\n",
    "    \n",
    "print Weights1.name\n",
    "print bias1.name\n",
    "    \n",
    "# 下面来共享上面已经定义好的变量\n",
    "# note: 在下面的 scope 中的get_variable()变量必须已经定义过了，才能设置 reuse=True，否则会报错\n",
    "with tf.variable_scope('vs3', reuse=True) as scope:\n",
    "    Weights2 = tf.get_variable('Weights')\n",
    "    bias2 = tf.get_variable('bias', [1])  # 报错，vs3/bias does not exist, or was not created with tf.get_variable().\n",
    "\n",
    "print Weights2.name\n",
    "print bias2.name\n",
    "\n",
    "# 这样子的话就会报错\n",
    "# Variable v_scope/bias does not exist, or was not created with tf.get_variable()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}