首页 > 代码库 > 不要怂,就是GAN (生成式对抗网络) (五):无约束条件的 GAN

不要怂,就是GAN (生成式对抗网络) (五):无约束条件的 GAN

GAN 这个领域发展太快,日新月异,各种 GAN 层出不穷,前几天看到一篇关于 Wasserstein GAN 的文章,讲的很好,在此把它分享出来一起学习:https://zhuanlan.zhihu.com/p/25071913。相比 Wasserstein GAN ,我们的 DCGAN 好像低了一个档次,但是我们伟大的教育家鲁迅先生说过:“合抱之木,生于毫末;九层之台,起于累土;千里之行,始于足下”,(依稀记得那大概是我 7 - 8 岁的时候,鲁迅先生依偎在我身旁,带着和蔼可亲切的口吻对我说的这句话,他当时还加了一句话,小伙子你要记住,如果一句名言,你不知道是谁说的,那就是鲁迅说的)。所以我们的基础还是要打好的, DCGAN 是我们的基础,有了 DCGAN 的代码经验,相信写起 Wasserstein GAN 就顺手很多,所以,我们接下来继续来研究我们的无约束条件 DCGAN。

在上一篇文章中,我们用 MNIST 手写字符训练 GAN,生成网络 G 生成了相对比较好的手写字符,这一次,我们换个数据集,用 CelebA 人脸数据集来训练我们的 GAN,相比于手写字符,人脸数据集的分布更加复杂多样,长头发短头发,黄种人黑种人,戴眼镜不戴眼镜,男人女人等等,看看我们的生成网络 G 能否成功的检验出人脸数据集的分布。

首先准备数据:从官网分享的百度云盘连接 https://pan.baidu.com/s/1eSNpdRG#list/path=%2FCelebA%2FImg 下载 img_align_celeba.zip,在 /home/your_name/TensorFlow/DCGAN/data 文件夹下解压,得到 img_align_celeba 文件夹,里面有 20600 张人脸图片,在 /home/your_name/TensorFlow/DCGAN/data 文件夹下新建 img_align_celeba_tfrecords 文件夹,用来存放 tfrecords 文件,然后,在 /home/your_name/TensorFlow/DCGAN/ 下新建 convert_data.py,编写如下的代码,把人脸图片转化成 tfrecords 形式:

import os
import time
from PIL import Image

import tensorflow as tf

# 将图片裁剪为 128 x 128
OUTPUT_SIZE = 128
# 图片通道数,3 表示彩色
DEPTH = 3


def _int64_feature(value):
    return tf.train.Feature(int64_list = tf.train.Int64List(value =http://www.mamicode.com/ [value]))
def _bytes_feature(value):
    return tf.train.Feature(bytes_list = tf.train.BytesList(value =http://www.mamicode.com/ [value]))
    

def convert_to(data_path, name):
    
    """
    Converts s dataset to tfrecords
    """
    
    rows = 64
    cols = 64
    depth = DEPTH
    # 循环 12 次,产生 12 个 .tfrecords 文件
    for ii in range(12):
        writer = tf.python_io.TFRecordWriter(name + str(ii) + .tfrecords)
        # 每个 tfrecord 文件有 16384 个图片
        for img_name in os.listdir(data_path)[ii*16384 : (ii+1)*16384]:
            # 打开图片
            img_path = data_path + img_name
            img = Image.open(img_path)
            # 设置裁剪参数
            h, w = img.size[:2]
            j, k = (h - OUTPUT_SIZE) / 2, (w - OUTPUT_SIZE) / 2
            box = (j, k, j + OUTPUT_SIZE, k+ OUTPUT_SIZE)
            # 裁剪图片
            img = img.crop(box = box)
            # image resize
            img = img.resize((rows,cols))
            # 转化为字节
            img_raw = img.tobytes()
            # 写入到 Example 
            example = tf.train.Example(features = tf.train.Features(feature = {
                                        height: _int64_feature(rows),
                                        weight: _int64_feature(cols),
                                        depth: _int64_feature(depth),
                                        image_raw: _bytes_feature(img_raw)}))
            writer.write(example.SerializeToString())
        writer.close()


if __name__ == __main__:
    
    current_dir = os.getcwd()    
    data_path = current_dir + /data/img_align_celeba/
    name = current_dir + /data/img_align_celeba_tfrecords/train
    start_time = time.time() 
    
    print(Convert start)   
    print(\n * 2)
    
    convert_to(data_path, name)
    
    print(\n * 2)
    print(Convert done, take %.2f seconds % (time.time() - start_time))

运行之后,在 /home/your_name/TensorFlow/DCGAN/data/img_align_celeba_tfrecords/ 下会产生 12 个 .tfrecords 文件,这就是我们要的数据格式。

 

数据准备好之后,根据前面的经验,我们来写无约束条件的 DCGAN 代码,在 /home/your_name/TensorFlow/DCGAN/ 新建 none_cond_DCGAN.py 文件敲写代码,为了简便起见,代码中没有加注释并且把所有的代码总结到一个代码中,从代码中可以看到,我们自己写了一个 batch_norm 层,解决了 evaluation 函数中 is_train = False 的问题,并且可以断点续训练(只需要将开头的 LOAD_MODEL 设置为 True);此外该程序在开头采用很多的宏定义,可以方便的改为 tf.app.flags 定义的命令行参数,进而在命令行终端进行训练,还可以进行类的拓展,例如:

 

class DCGAN(object):
    def __init__(self):
        self.BATCH_SIZE = 64
        ...
    def bias(self):
        ...

    ...

 

关于类的拓展,这里不做过多说明。

 

在 none_cond_DCGAN.py 文件中敲写如下代码:

技术分享
import os
import numpy as np
import scipy.misc
import tensorflow as tf

BATCH_SIZE = 64
OUTPUT_SIZE = 64
GF = 64             # Dimension of G filters in first conv layer. default [64]
DF = 64             # Dimension of D filters in first conv layer. default [64]
Z_DIM = 100
IMAGE_CHANNEL = 3
LR = 0.0002         # Learning rate
EPOCH = 5
LOAD_MODEL = False  # Whether or not continue train from saved model。
TRAIN = True
CURRENT_DIR = os.getcwd()

def bias(name, shape, bias_start = 0.0, trainable = True):
    
    dtype = tf.float32
    var = tf.get_variable(name, shape, tf.float32, trainable = trainable, 
                          initializer = tf.constant_initializer(
                                                  bias_start, dtype = dtype))
    return var


def weight(name, shape, stddev = 0.02, trainable = True):
    
    dtype = tf.float32
    var = tf.get_variable(name, shape, tf.float32, trainable = trainable, 
                          initializer = tf.random_normal_initializer(
                                              stddev = stddev, dtype = dtype))
    return var


def fully_connected(value, output_shape, name = fully_connected, with_w = False):
    
    shape = value.get_shape().as_list()
    
    with tf.variable_scope(name):
        weights = weight(weights, [shape[1], output_shape], 0.02)
        biases = bias(biases, [output_shape], 0.0)
        
    if with_w:
        return tf.matmul(value, weights) + biases, weights, biases
    else:
        return tf.matmul(value, weights) + biases

    
def lrelu(x, leak=0.2, name = lrelu):
    
    with tf.variable_scope(name):
        return tf.maximum(x, leak*x, name = name)
        
        
def relu(value, name = relu):
    with tf.variable_scope(name):
        return tf.nn.relu(value)
    
    
def deconv2d(value, output_shape, k_h = 5, k_w = 5, strides =[1, 2, 2, 1], 
             name = deconv2d, with_w = False):
    
    with tf.variable_scope(name):
        weights = weight(weights, 
                         [k_h, k_w, output_shape[-1], value.get_shape()[-1]])
        deconv = tf.nn.conv2d_transpose(value, weights, 
                                        output_shape, strides = strides)
        biases = bias(biases, [output_shape[-1]])
        deconv = tf.reshape(tf.nn.bias_add(deconv, biases), deconv.get_shape())
        if with_w:
            return deconv, weights, biases
        else:
            return deconv
            
            
def conv2d(value, output_dim, k_h = 5, k_w = 5, 
            strides =[1, 2, 2, 1], name = conv2d):
    
    with tf.variable_scope(name):
        weights = weight(weights, 
                         [k_h, k_w, value.get_shape()[-1], output_dim])
        conv = tf.nn.conv2d(value, weights, strides = strides, padding = SAME)
        biases = bias(biases, [output_dim])
        conv = tf.reshape(tf.nn.bias_add(conv, biases), conv.get_shape())
        
        return conv


def conv_cond_concat(value, cond, name = concat):
    
    """
    Concatenate conditioning vector on feature map axis.
    """
    value_shapes = value.get_shape().as_list()
    cond_shapes = cond.get_shape().as_list()
    
    with tf.variable_scope(name):        
        return tf.concat(3,
                 [value, cond * tf.ones(value_shapes[0:3] + cond_shapes[3:])])
  
        
def batch_norm(value, is_train = True, name = batch_norm, 
               epsilon = 1e-5, momentum = 0.9):
    
    with tf.variable_scope(name):
        
        ema = tf.train.ExponentialMovingAverage(decay = momentum)
        shape = value.get_shape().as_list()[-1]
        beta = bias(beta, [shape], bias_start = 0.0)
        gamma = bias(gamma, [shape], bias_start = 1.0)
        
        if is_train:

            batch_mean, batch_variance = tf.nn.moments(
                value, [0, 1, 2], name = moments)

            moving_mean = bias(moving_mean, [shape], 0.0, False)
            moving_variance = bias(moving_variance, [shape], 1.0, False)
            
            ema_apply_op = ema.apply([batch_mean, batch_variance])
            
            assign_mean = moving_mean.assign(ema.average(batch_mean))
            assign_variance =                 moving_variance.assign(ema.average(batch_variance))
            
            with tf.control_dependencies([ema_apply_op]):
                mean, variance =                     tf.identity(batch_mean), tf.identity(batch_variance)
            
            with tf.control_dependencies([assign_mean, assign_variance]):
                return tf.nn.batch_normalization(
                    value, mean, variance, beta, gamma, 1e-5)
        
        else:
            mean = bias(moving_mean, [shape], 0.0, False)
            variance = bias(moving_variance, [shape], 1.0, False)

            return tf.nn.batch_normalization(
                value, mean, variance, beta, gamma, epsilon)


def generator(z, is_train = True, name = generator):
    
    with tf.name_scope(name):
    
        s2, s4, s8, s16 =                     OUTPUT_SIZE/2, OUTPUT_SIZE/4, OUTPUT_SIZE/8, OUTPUT_SIZE/16
    
        h1 = tf.reshape(fully_connected(z, GF*8*s16*s16, g_fc1), 
                        [-1, s16, s16, GF*8], name = reshap)
        h1 = relu(batch_norm(h1, name = g_bn1, is_train = is_train))
        
        h2 = deconv2d(h1, [BATCH_SIZE, s8, s8, GF*4], name = g_deconv2d1)
        h2 = relu(batch_norm(h2, name = g_bn2, is_train = is_train))
        
        h3 = deconv2d(h2, [BATCH_SIZE, s4, s4, GF*2], name = g_deconv2d2)
        h3 = relu(batch_norm(h3, name = g_bn3, is_train = is_train))
        
        h4 = deconv2d(h3, [BATCH_SIZE, s2, s2, GF*1], name = g_deconv2d3)
        h4 = relu(batch_norm(h4, name = g_bn4, is_train = is_train))
        
        h5 = deconv2d(h4, [BATCH_SIZE, OUTPUT_SIZE, OUTPUT_SIZE, 3], 
                      name = g_deconv2d4)    
        
        return tf.nn.tanh(h5)
    
    
def discriminator(image, reuse = False, name = discriminator):
    
    with tf.name_scope(name):    
    
        if reuse:
            tf.get_variable_scope().reuse_variables()
        
        h0 = lrelu(conv2d(image, DF, name=d_h0_conv), name = d_h0_lrelu)
        h1 = lrelu(batch_norm(conv2d(h0, DF*2, name=d_h1_conv),
                              name = d_h1_bn), name = d_h1_lrelu)
        h2 = lrelu(batch_norm(conv2d(h1, DF*4, name=d_h2_conv),
                              name = d_h2_bn), name = d_h2_lrelu)
        h3 = lrelu(batch_norm(conv2d(h2, DF*8, name=d_h3_conv),
                              name = d_h3_bn), name = d_h3_lrelu)
        h4 = fully_connected(tf.reshape(h3, [BATCH_SIZE, -1]), 1, d_h4_fc)
        
        return tf.nn.sigmoid(h4), h4
    
        
def sampler(z, is_train = False, name = sampler):
    
    with tf.name_scope(name):
        
        tf.get_variable_scope().reuse_variables()
        return generator(z, is_train = is_train)
    
    
def read_and_decode(filename_queue):
    
    """
    read and decode tfrecords
    """
    
    reader = tf.TFRecordReader()
    _, serialized_example = reader.read(filename_queue)
    
    features = tf.parse_single_example(serialized_example,features = {
                        image_raw:tf.FixedLenFeature([], tf.string)})
    image = tf.decode_raw(features[image_raw], tf.uint8)
    
    image = tf.reshape(image, [OUTPUT_SIZE, OUTPUT_SIZE, 3])
    image = tf.cast(image, tf.float32)
    image = image / 255.0
    
    return image
    

def inputs(data_dir, batch_size, name = input):
    
    """
    Reads input data num_epochs times.
    """
            
    with tf.name_scope(name):
        filenames = [
            os.path.join(data_dir,train%d.tfrecords % ii) for ii in range(12)]
        filename_queue = tf.train.string_input_producer(filenames)
        
        image = read_and_decode(filename_queue)
        
        images = tf.train.shuffle_batch([image], batch_size = batch_size, 
                                        num_threads = 4, 
                                        capacity = 20000 + 3 * batch_size, 
                                        min_after_dequeue = 20000)
        return images


def save_images(images, size, path):
    
    """
    Save the samples images
    The best size number is
            int(max(sqrt(image.shape[1]),sqrt(image.shape[1]))) + 1
    """
    img = (images + 1.0) / 2.0
    h, w = img.shape[1], img.shape[2]
    merge_img = np.zeros((h * size[0], w * size[1], 3))
    for idx, image in enumerate(images):
        i = idx % size[1]
        j = idx // size[1]
        merge_img[j*h:j*h+h, i*w:i*w+w, :] = image
        
    return scipy.misc.imsave(path, merge_img)    

    
def train():
        
    global_step = tf.Variable(0, name = global_step, trainable = False)

    train_dir = CURRENT_DIR + /logs_without_condition/
    data_dir = CURRENT_DIR + /data/img_align_celeba_tfrecords/

    images = inputs(data_dir, BATCH_SIZE)

    z = tf.placeholder(tf.float32, [None, Z_DIM], name=z)

    G = generator(z)
    D, D_logits  = discriminator(images)
    samples = sampler(z)
    D_, D_logits_ = discriminator(G, reuse = True)
    
    d_loss_real = tf.reduce_mean(
            tf.nn.sigmoid_cross_entropy_with_logits(D_logits, tf.ones_like(D)))
    d_loss_fake = tf.reduce_mean(
            tf.nn.sigmoid_cross_entropy_with_logits(D_logits_, tf.zeros_like(D_)))
    d_loss = d_loss_real + d_loss_fake
    g_loss = tf.reduce_mean(
            tf.nn.sigmoid_cross_entropy_with_logits(D_logits_, tf.ones_like(D_)))
                                                                      
    z_sum = tf.histogram_summary(z, z)
    d_sum = tf.histogram_summary(d, D)
    d__sum = tf.histogram_summary(d_, D_)
    G_sum = tf.image_summary(G, G)

    d_loss_real_sum = tf.scalar_summary(d_loss_real, d_loss_real)
    d_loss_fake_sum = tf.scalar_summary(d_loss_fake, d_loss_fake)
    d_loss_sum = tf.scalar_summary(d_loss, d_loss)                                                
    g_loss_sum = tf.scalar_summary(g_loss, g_loss)
    
    g_sum = tf.merge_summary([z_sum, d__sum, G_sum, d_loss_fake_sum, g_loss_sum])
    d_sum = tf.merge_summary([z_sum, d_sum, d_loss_real_sum, d_loss_sum])

    t_vars = tf.trainable_variables()
    d_vars = [var for var in t_vars if d_ in var.name]
    g_vars = [var for var in t_vars if g_ in var.name]

    saver = tf.train.Saver()
    
    d_optim = tf.train.AdamOptimizer(LR, beta1 = 0.5)                 .minimize(d_loss, var_list = d_vars, global_step = global_step)
    g_optim = tf.train.AdamOptimizer(LR, beta1 = 0.5)                 .minimize(g_loss, var_list = g_vars, global_step = global_step)
        
    
    os.environ[CUDA_VISIBLE_DEVICES] = str(0)
    config = tf.ConfigProto()
    config.gpu_options.per_process_gpu_memory_fraction = 0.2
    sess = tf.InteractiveSession(config=config)
     
    writer = tf.train.SummaryWriter(train_dir, sess.graph)    
    
    sample_z = np.random.uniform(-1, 1, size = (BATCH_SIZE, Z_DIM))
    
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(sess = sess, coord = coord)
    init = tf.initialize_all_variables()  
    sess.run(init)

    start = 0
    if LOAD_MODEL:        
        print(" [*] Reading checkpoints...")
        ckpt = tf.train.get_checkpoint_state(train_dir)        

        if ckpt and ckpt.model_checkpoint_path:
            ckpt_name = os.path.basename(ckpt.model_checkpoint_path)
            saver.restore(sess, os.path.join(train_dir, ckpt_name))
            global_step = ckpt.model_checkpoint_path.split(/)[-1]                                                    .split(-)[-1]
            print(Loading success, global_step is %s % global_step)
            
        start = int(global_step)
        
    for epoch in range(EPOCH):
        
        batch_idxs = 3072
        
        if epoch:
            start = 0
            
        for idx in range(start, batch_idxs):

            batch_z = np.random.uniform(-1, 1, size = (BATCH_SIZE, Z_DIM))
            
            
            _, summary_str = sess.run([d_optim, d_sum], feed_dict = {z: batch_z})
            writer.add_summary(summary_str, idx+1)

            # Update G network
            _, summary_str = sess.run([g_optim, g_sum], feed_dict = {z: batch_z})
            writer.add_summary(summary_str, idx+1)

            # Run g_optim twice to make sure that d_loss does not go to zero
            _, summary_str = sess.run([g_optim, g_sum], feed_dict = {z: batch_z})
            writer.add_summary(summary_str, idx+1)
            
            errD_fake = d_loss_fake.eval({z: batch_z})
            errD_real = d_loss_real.eval()
            errG = g_loss.eval({z: batch_z})
            if idx % 20 == 0:
                print("[%4d/%4d] d_loss: %.8f, g_loss: %.8f"                             % (idx, batch_idxs, errD_fake+errD_real, errG))
            
            if idx % 100 == 0:
                sample = sess.run(samples, feed_dict = {z: sample_z})
                samples_path = CURRENT_DIR + /samples_without_condition/
                save_images(sample, [8, 8], 
                            samples_path +                             sample_%d_epoch_%d.png % (epoch, idx))

                print \n*2
                print(===========    %d_epoch_%d.png save down    =========== 
                                                                %(epoch, idx))
                print \n*2
                
            if (idx % 512 == 0) or (idx + 1 == batch_idxs):
                checkpoint_path = os.path.join(train_dir, 
                                               my_dcgan_tfrecords.ckpt)
                saver.save(sess, checkpoint_path, global_step = idx+1)
                print *********    model saved    *********

        print ******* start with %d ******* % start
    
    coord.request_stop()    
    coord.join(threads)
    sess.close()
    
    
    
def evaluate():
    eval_dir = CURRENT_DIR + /eval/
    
    checkpoint_dir = CURRENT_DIR + /logs_without_condition/
    
    z = tf.placeholder(tf.float32, [None, Z_DIM], name=z)
    
    G = generator(z, is_train = False)
    
    sample_z1 = np.random.uniform(-1, 1, size=(BATCH_SIZE, Z_DIM))
    sample_z2 = np.random.uniform(-1, 1, size=(BATCH_SIZE, Z_DIM))
    sample_z3 = (sample_z1 + sample_z2) / 2
    sample_z4 = (sample_z1 + sample_z3) / 2
    sample_z5 = (sample_z2 + sample_z3) / 2    
    
    print("Reading checkpoints...")
    ckpt = tf.train.get_checkpoint_state(checkpoint_dir)
    
    saver = tf.train.Saver(tf.all_variables())
    
    os.environ[CUDA_VISIBLE_DEVICES] = str(0)
    config = tf.ConfigProto()
    config.gpu_options.per_process_gpu_memory_fraction = 0.2
    sess = tf.InteractiveSession(config=config)
    
    if ckpt and ckpt.model_checkpoint_path:
        ckpt_name = os.path.basename(ckpt.model_checkpoint_path)
        global_step = ckpt.model_checkpoint_path.split(/)[-1].split(-)[-1]        
        saver.restore(sess, os.path.join(checkpoint_dir, ckpt_name))
        print(Loading success, global_step is %s % global_step)
    
    eval_sess1 = sess.run(G, feed_dict = {z: sample_z1})
    eval_sess2 = sess.run(G, feed_dict = {z: sample_z4})
    eval_sess3 = sess.run(G, feed_dict = {z: sample_z3})
    eval_sess4 = sess.run(G, feed_dict = {z: sample_z5})
    eval_sess5 = sess.run(G, feed_dict = {z: sample_z2})
    
    print(eval_sess3.shape)
    
    save_images(eval_sess1, [8, 8], eval_dir + eval_%d.png % 1)
    save_images(eval_sess2, [8, 8], eval_dir + eval_%d.png % 2)
    save_images(eval_sess3, [8, 8], eval_dir + eval_%d.png % 3)
    save_images(eval_sess4, [8, 8], eval_dir + eval_%d.png % 4)
    save_images(eval_sess5, [8, 8], eval_dir + eval_%d.png % 5)
    
    
    sess.close()


if __name__ == __main__:
    
    if TRAIN:
        train()
    else:
        evaluate()
View Code

 

完成后,运行代码,网络开始训练,大致需要 1~2 个小时,训练就可以完成,在训练的过程中,可以看出 sampler 采样的生成结果越来越好,最后得到了一个如下图所示的结果,由于人脸的数据分布比手写数据分布复杂多样,所以生成器不能完全抓住人脸的特征,下图所示的第 6 行第 7 列就是一个很糟糕的生成图像。

 

技术分享

 

训练完成后,我们用 tensorboard 打开网络的 graph,看看经过我们的精心设计,网络结构变成了什么样子:

技术分享

 

 可以看出来,这次的结构图,比之前的顺眼多了,简直是处女座的福音啊有木有。

 

至此,我们完成了 DCGAN 的代码,下一篇文章,我们来说说 Caffe 那点事。

 

 

参考文献:

1. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/how_tos/reading_data/convert_to_records.py

2. https://github.com/carpedm20/DCGAN-tensorflow

 

不要怂,就是GAN (生成式对抗网络) (五):无约束条件的 GAN